Underwater image dual-path enhancement method based on wavelet domain diffusion and spatial domain cooperation
By employing a dual-path enhancement method that combines wavelet domain diffusion and spatial domain synergy, the problem of frequency domain differentiation in underwater image enhancement is solved, enabling refined enhancement and adaptive fusion of underwater images and improving image quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU UNIVERSITY
- Filing Date
- 2026-01-19
- Publication Date
- 2026-05-01
AI Technical Summary
Existing deep learning-based underwater image enhancement techniques fail to fully utilize the differentiated degradation characteristics of underwater images in different frequency domains, making it difficult to refine the features of each frequency band. Furthermore, they lack global semantic association and hierarchical processing strategies, and the fusion coefficients are difficult to adjust dynamically.
A dual-path enhancement method combining wavelet domain diffusion and spatial domain synergy is adopted. The image is decomposed by multi-level discrete wavelet transform, and frequency domain and spatial domain enhancement are performed respectively. An adaptive fusion mechanism is used to perform weighted fusion by combining pixel-by-pixel and global weights.
It achieves targeted processing of degradation features in different frequency domains, improves the color fidelity and visual harmony of underwater images, and enhances the adaptability and refinement of the effect.
Smart Images
Figure CN121961855A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image enhancement technology, and particularly relates to a dual-path enhancement method for underwater images based on wavelet domain diffusion and spatial domain synergy. Background Technology
[0002] Existing deep learning-based underwater image enhancement techniques mainly face the following bottlenecks: mainstream methods mostly perform end-to-end mapping in pixel space, failing to fully utilize the differentiated degradation characteristics of underwater images in different frequency domains (color distortion is concentrated in low frequencies, and detail blurring is concentrated in high frequencies), making it difficult to finely process features in each frequency band. Although frequency domain methods provide guidance for processing different components, they often process frequency domain information independently, lacking a connection with the global semantics of the spatial domain, and lack a systematic strategy for hierarchically processing degradation features at different scales.
[0003] In addition, existing fusion methods mostly rely on predefined static weights or simple splicing, making it difficult to dynamically and finely adjust the fusion coefficients according to the actual degradation of the image and the differentiated needs of different regions. Summary of the Invention
[0004] In view of the above-mentioned deficiencies of the prior art, this invention proposes a dual-path enhancement method for underwater images based on wavelet domain diffusion and spatial domain synergy. The technical solution designed in this invention includes the following steps: S1: Acquire the original underwater image and preprocess it. Perform multi-level discrete wavelet transform on each image to decompose it into low-frequency components and multi-level high-frequency components. S2: Perform frequency domain enhancement on the low-frequency component and the multi-layer high-frequency component respectively, and output a frequency domain enhanced image; S3: Input the original underwater image into the spatial domain residual mapping network for color and contrast correction to obtain a spatial domain enhanced image; S4: Generate pixel-wise weights and global weights by adaptively fusing the original underwater image, frequency domain enhanced image, and spatial domain enhanced image. Then, perform weighted fusion on the frequency domain enhancement result and the spatial domain enhancement result based on the pixel-wise weights and global weights to output the final enhanced image. S5: Load the trained model, and sequentially execute S2 to S4 on the input raw underwater image to output the final enhanced image.
[0005] Preferably, S1 includes: S11: Obtain paired or unpaired raw underwater images from public datasets, and filter and clean invalid samples; S12: Scale the original underwater image to a preset size and maintain the aspect ratio by filling; S13: Normalize the pixel values of the original underwater image; S14: Use the Haar wavelet basis to perform a two-level discrete wavelet transform on the normalized image to obtain low-frequency components and multiple high-frequency components.
[0006] Preferably, S2 includes: S21: The low-frequency components are restored in color and illumination by low-frequency reconstruction based on a diffusion model to obtain the enhanced low-frequency components; S22: The high-frequency enhancement branch based on the Transformer structure is used to reconstruct the detailed texture of the multi-layer high-frequency components to obtain the enhanced multi-layer high-frequency components; S23: Perform inverse wavelet transform on the enhanced low-frequency component and the enhanced multi-layer high-frequency component to reconstruct the frequency domain enhanced image.
[0007] Preferably, S21 includes: S211: Perform a forward diffusion process on the low-frequency components. Perform a forward diffusion process on the low-frequency components under conditional constraints. Gaussian noise is gradually added according to the preset noise scheduling table to obtain noisy low-frequency components at different time steps. S212: Construct a conditional noise prediction network based on U-Net. The input of the network includes noisy low-frequency components, time-step encoding, and conditional information corresponding to the low-frequency components. S213: The conditional noise prediction network is trained by minimizing the error function between the predicted noise and the actual noise to obtain a trained low-frequency conditional diffusion reconstruction model. S214: Using the trained low-frequency conditional diffusion reconstruction model, under the constraint of the conditional information, perform an inverse denoising sampling process on the low-frequency components to gradually remove noise and obtain the enhanced low-frequency components.
[0008] Preferably, S22 includes: S221: Channel splicing of multi-layer high-frequency components; S222: Map the concatenated data to high-dimensional features through overlapping Patch Embedding layers; S223: High-dimensional features are processed through an encoder-decoder structure, where both the encoder and decoder are composed of multiple cascaded wide-area Transformer blocks; S224: The wide-area Transformer block uses multi-scale deep separable convolution and multi-head self-attention mechanism for feature extraction, and fuses multi-scale features through skip connections; S225: The decoder outputs enhanced multi-layer high-frequency components.
[0009] Preferably, the spatial domain residual mapping network in S3 is a convolutional neural network containing multiple residual blocks. The input is the original underwater image, and the output is added to the input as a residual to obtain the spatial domain enhanced image.
[0010] Preferably, S4 includes: S41: The original underwater image, frequency domain enhanced image, and spatial domain enhanced image are concatenated along the channel dimension and input into a weight generation subnetwork to generate two pixel-wise weight maps corresponding to the frequency domain enhancement result and the spatial domain enhancement result, respectively. S42: Perform global average pooling on the three input images respectively, and calculate a spatial domain global weight and a frequency domain global weight through a fully connected layer; S43: Using the pixel-by-pixel weight map, spatial domain global weight, and frequency domain global weight, perform pixel-level weighted fusion of the frequency domain enhanced image and the spatial domain enhanced image to output the final enhanced image.
[0011] Beneficial effects: 1. This application constructs a space-frequency dual-path enhancement structure, focuses on restoring low-frequency color and illumination through a diffusion model, and enhances high-frequency texture details using a multi-scale Transformer, thereby achieving targeted processing and collaborative optimization of degradation features in different frequency domains.
[0012] 2. This application designs a lightweight spatial domain residual mapping network to directly learn the complex underwater color space transformation rules, effectively correcting uneven color shifts and contrast reduction, and improving color fidelity.
[0013] 3. By introducing a fusion mechanism that combines global weights and pixel-wise weights, this application can dynamically and precisely balance the contributions of the spatial and frequency enhancement results based on the global features and local regional characteristics of the input image, thereby improving the adaptability and visual harmony of the overall enhancement effect. Attached Figure Description
[0014] Figure 1 This is a flowchart illustrating a preferred embodiment of the present invention; Figure 2 This is a schematic diagram of a preferred embodiment of the present invention. Detailed Implementation The embodiments of the present invention will be described in detail below. The embodiments described below are implemented based on the technical solution of the present invention, and detailed implementation methods and specific operation processes are given. However, the protection scope of the present invention is not limited to the embodiments described below.
[0015] This invention designs a dual-path enhancement method for underwater images based on wavelet domain diffusion and spatial domain synergy. The technical solution includes the following steps, such as... Figure 1-2 As shown, it specifically includes: S1: Acquire the original underwater image and preprocess it. Perform multi-level discrete wavelet transform on each image to decompose it into low-frequency components and multi-level high-frequency components. S2: Perform frequency domain enhancement on the low-frequency component and the multi-layer high-frequency component respectively, and output the frequency domain enhanced image; S3: Input the original underwater image into the spatial domain residual mapping network for color and contrast correction to obtain a spatial domain enhanced image; S4: Generate pixel-wise weights and global weights by adaptively fusing the original underwater image, frequency domain enhanced image, and spatial domain enhanced image. Then, perform weighted fusion on the frequency domain enhancement result and the spatial domain enhancement result based on the pixel-wise weights and global weights to output the final enhanced image. S5: Load the trained model, and sequentially execute S2 to S4 on the input raw underwater image to output the final enhanced image.
[0016] Preferably, S1 includes: S11: Obtain paired or unpaired raw underwater images from public datasets, and filter and clean invalid samples; S12: Scale the original underwater image to a preset size and maintain the aspect ratio by filling; S13: Normalize the pixel values of the original underwater image; S14: Use the Haar wavelet basis to perform a two-level discrete wavelet transform on the normalized image to obtain low-frequency components and multiple high-frequency components.
[0017] Specifically, for S11, the public datasets include UIEB, LSUI, and U45; for S12, the preset size is 256×256 pixels, and images smaller than this size are padded with zero pixel values; for S13, the pixel value range of the image is linearly scaled to the [-1,1] interval; for S14, the first layer decomposition yields the low-frequency components. With high frequency components , , ,right Perform a second-level decomposition to obtain the second-level low-frequency components. With high frequency components , , Among them, low-frequency components specifically refer to For subsequent color and lighting restoration, multi-layer high-frequency components include , , , , , This is used for subsequent detailed texture reconstruction.
[0018] Preferably, S2 includes: S21: Low-frequency components are restored in color and illumination by low-frequency reconstruction based on diffusion model to obtain enhanced low-frequency components; S22: The high-frequency enhancement branch based on the Transformer structure is used to reconstruct the detailed texture of the multi-layer high-frequency components to obtain the enhanced multi-layer high-frequency components. S23: Perform inverse wavelet transform on the enhanced low-frequency components and the enhanced multi-layer high-frequency components to reconstruct the frequency domain enhanced image.
[0019] Preferably, S21 includes: S211: Perform a forward diffusion process on the low-frequency components. Perform a forward diffusion process on the low-frequency components under conditional constraints. Gaussian noise is gradually added according to the preset noise scheduling table to obtain noisy low-frequency components at different time steps. S212: Construct a conditional noise prediction network based on U-Net. The input of the network includes noisy low-frequency components, time-step encoding, and conditional information corresponding to the low-frequency components. S213: The conditional noise prediction network is trained by minimizing the error function between the predicted noise and the actual noise to obtain a trained low-frequency conditional diffusion reconstruction model. S214: Using the trained low-frequency conditional diffusion reconstruction model, under the constraint of the conditional information, perform an inverse denoising sampling process on the low-frequency components to gradually remove noise and obtain the enhanced low-frequency components.
[0020] Specifically, for S211, the intensity of the noise added at each step is controlled according to the preset noise scheduling table; for S213, the network parameters are trained by minimizing the mean square error between the real noise and the predicted noise.
[0021] Preferably, S22 includes: S221: Channel splicing of multi-layer high-frequency components; S222: Map the concatenated data to high-dimensional features through overlapping Patch Embedding layers; S223: High-dimensional features are processed through an encoder-decoder structure, where both the encoder and decoder are composed of multiple cascaded wide-area Transformer blocks; S224: The wide-area Transformer block uses multi-scale deep separable convolution and multi-head self-attention mechanism for feature extraction, and fuses multi-scale features through skip connections; S225: The decoder outputs enhanced multi-layer high-frequency components.
[0022] Specifically, for S223, the encoder expands the receptive field at each level through downsampling to extract multi-scale features; for S224, multi-scale depthwise separable convolutions use three different scales of convolution kernels in parallel: 3×3, 5×5 and 7×7.
[0023] Preferably, the spatial domain residual mapping network in S3 is a convolutional neural network containing multiple residual blocks. The input is the original underwater image, and the output is added to the input as a residual to obtain the spatial domain enhanced image.
[0024] Preferably, S4 includes: S41: The original underwater image, frequency domain enhanced image, and spatial domain enhanced image are concatenated along the channel dimension and input into a weight generation subnetwork to generate two pixel-wise weight maps corresponding to the frequency domain enhancement result and the spatial domain enhancement result, respectively. S42: Perform global average pooling on the three input images respectively, and calculate a spatial domain global weight and a frequency domain global weight through a fully connected layer; S43: Pixel-level weighted fusion of the frequency domain enhanced image and the spatial domain enhanced image is performed using a pixel-wise weight map, spatial domain global weight, and frequency domain global weight to output the final enhanced image.
[0025] Specifically, for S42, the two calculated global weights satisfy the constraint that their sum is 1; for S43, the final pixel-level weighted fusion can be expressed as:
[0026] In the formula, and This is the normalized pixel-by-pixel weight map. and For two-way image enhancement, and These are the global weights in the spatial domain and the frequency domain, respectively.
[0027] Additionally, for S5, loading the pre-trained model refers to loading the model parameters that performed optimally on the validation set during the model training phase. The processing flow for the input raw underwater image is as follows: First, perform a two-layer discrete wavelet transform decomposition as in S1; then, input the decomposed components into the pre-trained frequency domain enhancement path (execute S2) and spatial domain enhancement path (execute S3) for parallel processing; finally, input the two enhancement results and the original image into the adaptive fusion module (execute S4) to generate the final enhanced image and output it, thus completing the entire inference phase.
[0028] In addition, the hyperparameter settings and optimization strategies during training are as follows: the total number of iterations is initialized to 0, and its maximum value is set according to the size of different underwater datasets, usually not exceeding 300; the batch size during training is fixed at 4; the initial learning rate is set to 0.0001, and a cosine learning rate decay strategy is adopted to gradually decrease the learning rate during training. A training warm-up strategy is applied, using a smaller learning rate of 1 in the first epoch. To stabilize the initial training phase, the Adam algorithm was used as the optimizer for model training.
[0029] In addition, during model training, the optimal model is selected by monitoring the joint loss function curve on the validation set. When the validation loss converges and reaches its minimum value, the model parameters corresponding to that epoch are saved. Furthermore, model performance is evaluated using objective enhancement metrics, primarily Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Underwater Image Quality Assessment (UIQM). Ultimately, the model with the best overall performance is selected.
[0030] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A dual-path enhancement method for underwater images based on wavelet domain diffusion and spatial domain synergy, characterized in that, include: S1: Acquire the original underwater image and preprocess it. Perform multi-level discrete wavelet transform on each image to decompose it into low-frequency components and multi-level high-frequency components. S2: Perform frequency domain enhancement on the low-frequency component and the multi-layer high-frequency component respectively, and output a frequency domain enhanced image; S3: Input the original underwater image into the spatial domain residual mapping network for color and contrast correction to obtain a spatial domain enhanced image; S4: Generate pixel-wise weights and global weights by adaptively fusing the original underwater image, frequency domain enhanced image, and spatial domain enhanced image. Then, perform weighted fusion on the frequency domain enhancement result and the spatial domain enhancement result based on the pixel-wise weights and global weights to output the final enhanced image. S5: Load the trained model, and sequentially execute S2 to S4 on the input raw underwater image to output the final enhanced image.
2. The underwater image dual-path enhancement method based on wavelet domain diffusion and spatial domain synergy as described in claim 1, characterized in that, S1 includes: S11: Obtain paired or unpaired raw underwater images from public datasets, and filter and clean invalid samples; S12: Scale the original underwater image to a preset size and maintain the aspect ratio by filling; S13: Normalize the pixel values of the original underwater image; S14: Use the Haar wavelet basis to perform a two-level discrete wavelet transform on the normalized image to obtain low-frequency components and multiple high-frequency components.
3. The underwater image dual-path enhancement method based on wavelet domain diffusion and spatial domain synergy as described in claim 1, characterized in that, S2 includes: S21: The low-frequency components are restored in color and illumination by low-frequency reconstruction based on a diffusion model to obtain the enhanced low-frequency components; S22: The high-frequency enhancement branch based on the Transformer structure is used to reconstruct the detailed texture of the multi-layer high-frequency components to obtain the enhanced multi-layer high-frequency components; S23: Perform inverse wavelet transform on the enhanced low-frequency component and the enhanced multi-layer high-frequency component to reconstruct the frequency domain enhanced image.
4. The underwater image dual-path enhancement method based on wavelet domain diffusion and spatial domain synergy as described in claim 3, characterized in that, S21 includes: S211: Perform a forward diffusion process on the low-frequency components. Perform a forward diffusion process on the low-frequency components under conditional constraints. Gaussian noise is gradually added according to the preset noise scheduling table to obtain noisy low-frequency components at different time steps. S212: Construct a conditional noise prediction network based on U-Net. The input of the network includes noisy low-frequency components, time-step encoding, and conditional information corresponding to the low-frequency components. S213: The conditional noise prediction network is trained by minimizing the error function between the predicted noise and the actual noise to obtain a trained low-frequency conditional diffusion reconstruction model. S214: Using the trained low-frequency conditional diffusion reconstruction model, under the constraint of the conditional information, perform an inverse denoising sampling process on the low-frequency components to gradually remove noise and obtain the enhanced low-frequency components.
5. The underwater image dual-path enhancement method based on wavelet domain diffusion and spatial domain synergy as described in claim 3, characterized in that, S22 includes: S221: Channel splicing of multi-layer high-frequency components; S222: Map the concatenated data to high-dimensional features through overlapping Patch Embedding layers; S223: High-dimensional features are processed through an encoder-decoder structure, where both the encoder and decoder are composed of multiple cascaded wide-area Transformer blocks; S224: The wide-area Transformer block uses multi-scale deep separable convolution and multi-head self-attention mechanism for feature extraction, and fuses multi-scale features through skip connections; S225: The decoder outputs enhanced multi-layer high-frequency components.
6. The underwater image dual-path enhancement method based on wavelet domain diffusion and spatial domain synergy as described in claim 1, characterized in that, The spatial domain residual mapping network in S3 is a convolutional neural network containing multiple residual blocks. The input is the original underwater image, and the output is added to the input as a residual to obtain the spatial domain enhanced image.
7. The underwater image dual-path enhancement method based on wavelet domain diffusion and spatial domain synergy as described in claim 1, characterized in that, S4 includes: S41: The original underwater image, frequency domain enhanced image, and spatial domain enhanced image are concatenated along the channel dimension and input into a weight generation subnetwork to generate two pixel-wise weight maps corresponding to the frequency domain enhancement result and the spatial domain enhancement result, respectively. S42: Perform global average pooling on the three input images respectively, and calculate a spatial domain global weight and a frequency domain global weight through a fully connected layer; S43: Using the pixel-by-pixel weight map, spatial domain global weight, and frequency domain global weight, perform pixel-level weighted fusion of the frequency domain enhanced image and the spatial domain enhanced image to output the final enhanced image.