Underwater image super-resolution reconstruction method and system based on diffusion model

Abstract

The invention discloses an underwater image super-resolution reconstruction method and system based on a diffusion model, and belongs to the field of image reconstruction. A lighter network structure design is adopted, so that the model training and reasoning speed is effectively accelerated, and the overall calculation overhead of the network is remarkably reduced; a spectrum domain guiding mechanism is introduced into a model structure, accurate reconstruction of high-frequency image details is realized through residual prediction learning, and global spectrum features and local spatial features are fused, so that the model has good global structure modeling capability and local texture reduction capability; according to the proposed multi-scale detail guide training strategy, a discrete wavelet transform loss function is introduced, and high-frequency features of the image on multiple scales are fully mined, so that the detail retention capability and the visual perception effect of the generated image are improved. According to the method, the subjective visual quality of the underwater image is remarkably improved while the excellent performance of the evaluation index is kept.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image reconstruction, in particular to an underwater image super-resolution reconstruction method and system based on a diffusion model. BACKGROUND

[0002] Underwater images have important application value in the fields of marine resource detection, underwater vehicle navigation, underwater communication and environmental monitoring. However, due to the serious attenuation and scattering of light propagation in water, underwater images usually have problems such as low resolution, low contrast, serious color deviation and blurred details. Especially, the image quality is further reduced under complex water environment, which seriously affects the subsequent image analysis and recognition tasks.

[0003] Image super-resolution technology aims to recover high-resolution images from low-resolution images, providing a feasible path to improve underwater image quality. In existing research, convolutional neural networks (CNN), generative adversarial networks (GAN) and attention mechanisms are widely used in image super-resolution tasks, and certain progress has been made. However, these methods still have deficiencies in texture detail restoration and global color reconstruction when facing complex underwater imaging degradation processes, making it difficult to balance image quality and human visual perception.

[0004] In recent years, diffusion probabilistic models (DPMs) as a new emerging generative model have shown great potential in image generation and restoration. Through forward diffusion perturbation and reverse step-by-step recovery process, high-quality image generation is achieved. However, the application of existing diffusion models in the field of super-resolution is mostly focused on natural images, and their high-frequency detail restoration and global feature modeling capabilities for underwater images with special degradation characteristics still need to be further improved.

[0005] Therefore, there is an urgent need for a super-resolution method that combines the characteristics of underwater images and the advantages of diffusion models, which can effectively restore image details, improve color restoration, and balance visual perception quality and computational efficiency, suitable for various real underwater application scenarios. SUMMARY

[0006] The present application provides a super-resolution reconstruction method and system for underwater images based on a diffusion model, to realize super-resolution reconstruction of any low-resolution underwater image to a high-resolution image.

[0007] The first aspect of the present application provides a super-resolution reconstruction method for underwater images based on a diffusion model, comprising the following steps:

[0008] Obtaining a low-resolution underwater image to be reconstructed;

[0009] inputting the underwater low-resolution image to be reconstructed into a pre-trained image reconstruction model comprising a double-stage structure;

[0010] The low-resolution underwater image is enlarged and color corrected by a lightweight coarse recovery module in the image reconstruction model to obtain a coarse recovery image, a denoising diffusion probability model is used to extract global features of the coarse recovery image and a diffusion noise image in a frequency domain through a spectral modeling module, a residual prediction network with a U-Net structure is combined to predict residual information between a high-resolution image and the coarse recovery image, a residual prediction image is obtained, and the coarse recovery image and the residual prediction image are added to obtain a reconstructed high-resolution underwater image.

[0011] Optionally, in an embodiment of the present application, the method further comprises:

[0012] A training set of underwater images comprising pairs of low-resolution underwater images and corresponding high-resolution underwater images is constructed, the training set of underwater images is input into the image reconstruction model with a double-stage structure, pixel-level L1 loss, diffusion residual loss and high-frequency loss based on discrete wavelet transform are jointly used to optimize the image reconstruction model with a double-stage structure in an end-to-end manner, and an optimization algorithm is used to iteratively update parameters of the image reconstruction model with a double-stage structure until the image reconstruction model with a double-stage structure converges, thereby obtaining a trained image reconstruction model with a double-stage structure.

[0013] Optionally, in an embodiment of the present application, in the coarse recovery stage, the coarse recovery module is a lightweight convolutional neural network with a parameter amount of no more than 0.02M, comprising an up-sampling operation and a plurality of 3x3 convolutional layers, and is used to recover image basic structure and color information.

[0014] Optionally, in an embodiment of the present application, in the spectral domain guided stage, the spectral modeling module uses fast Fourier transform to convert feature maps from a spatial domain to a spectral domain, extracts global frequency features in the spectral domain, and then restores them back to the spatial domain through inverse fast Fourier transform.

[0015] Optionally, in an embodiment of the present application, the diffusion model adds Gaussian noise through a forward diffusion process based on a denoising diffusion probability model, and predicts a residual image by using a reverse diffusion process.

[0016] Optionally, in an embodiment of the present application, the residual prediction network adopts a U-Net structure, has down-sampling, bottleneck layer and up-sampling paths, and is used to extract and fuse feature information at different scales.

[0017] Optionally, in an embodiment of the present application, the pixel domain L1 loss is used for the reconstruction error between the coarse recovered image and the high resolution image, the diffusion residual L1 loss is used for optimizing the prediction of the residual information by the diffusion model, the high frequency loss of the discrete wavelet transform is used for measuring the recovery effect of the high frequency details of the image in different directions, and the high frequency loss of the discrete wavelet transform is:

[0018]

[0019] wherein H i , V i , D i represents the x r subband of the i-th down-sampling level, x r is the residual between the high resolution image and the coarse recovered image, corresponds to the subband of the same down-sampling level, is the noise residual generated by network inference, and J represents the total number of down-sampling levels; The joint loss loss total is:

[0020]

[0021]

[0022] wherein, is the L1 loss between the high resolution image and the coarse recovered image, is the L1 loss between the initial noise residual and the noise residual generated by network inference.

[0023] The second aspect of the present application provides an underwater image super-resolution reconstruction system based on a diffusion model, comprising:

[0024] An acquisition module is configured to acquire an underwater low-resolution image to be reconstructed.

[0025] An input module is configured to input the underwater low-resolution image to be reconstructed into a pre-trained image reconstruction model comprising a two-stage structure.

[0026] A reconstruction module is configured to use a lightweight coarse recovery module in the image reconstruction model to enlarge and color correct the underwater low-resolution image, to obtain a coarse recovered image, to use a denoising diffusion probability model to extract global features of the coarse recovered image and a diffusion noise image in a frequency domain through a spectral modeling module, to combine a residual prediction network with a U-Net structure to predict residual information between a high resolution image and the coarse recovered image, to obtain a residual prediction image, and to add the coarse recovered image and the residual prediction image to obtain a reconstructed underwater high-resolution image.

[0027] ​A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the underwater image super-resolution reconstruction method based on the diffusion model as described in the above embodiments.

[0028] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to perform the underwater image super-resolution reconstruction method based on a diffusion model as described in the above embodiments.

[0029] The underwater image super-resolution reconstruction method and system based on the diffusion model of this invention, compared with the prior art, adopts an extremely lightweight structure in the coarse restoration module to significantly reduce the computational resource requirements while maintaining basic reconstruction capabilities; the spectral domain guided diffusion module innovatively combines frequency domain feature extraction and residual diffusion prediction, and reduces computational complexity through frequency domain conjugate symmetry, effectively enhancing the ability to restore high-frequency details; the two-stage collaborative design significantly improves the visual perception quality of underwater images without increasing model complexity.

[0030] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0031] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0032] Figure 1 A flowchart of an underwater image super-resolution reconstruction method based on a diffusion model according to an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram illustrating the principle of the underwater image super-resolution reconstruction method based on the diffusion model of the present invention.

[0034] Figure 3 This invention relates to a coarse recovery network based on convolutional networks.

[0035] Figure 4 The loss function of this invention is illustrated in the figure.

[0036] Figure 5 This invention relates to a spectral domain-guided residual diffusion network based on convolutional networks.

[0037] Figure 6 This is a schematic diagram comparing the image generated by the present invention with images generated by other methods;

[0038] Figure 7This is an example diagram of an underwater image super-resolution reconstruction system based on a diffusion model according to an embodiment of the present invention;

[0039] Figure 8 A schematic diagram of the structure of an electronic device provided for an embodiment of the invention. Detailed Implementation

[0040] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0041] Figure 1 This is a flowchart illustrating an underwater image super-resolution reconstruction method based on a diffusion model, according to an embodiment of the present invention.

[0042] like Figure 1 As shown, the underwater image super-resolution reconstruction method based on the diffusion model includes the following steps:

[0043] In step S101, the underwater low-resolution image to be reconstructed is acquired.

[0044] In step S102, the underwater low-resolution image to be reconstructed is input into a pre-trained image reconstruction model containing a two-stage structure.

[0045] In step S103, the lightweight coarse restoration module in the image reconstruction model is used to enlarge and color correct the underwater low-resolution image to obtain a coarse restoration image. A denoising diffusion probability model is used to extract global features in the frequency domain from the coarse restoration image and the diffusion noise map through the spectrum modeling module. The residual prediction network with U-Net structure is combined to predict the residual information between the high-resolution image and the coarse restoration image to obtain a residual prediction map. The coarse restoration image and the residual prediction map are added together to obtain the reconstructed underwater high-resolution image.

[0046] In an embodiment of the present invention, a deep learning model with a two-stage structure is first constructed for underwater image super-resolution reconstruction. The two-stage deep learning model includes a coarse recovery stage and a spectral domain guidance stage. The coarse recovery stage receives the input low-resolution underwater image and uses a lightweight coarse recovery module to enlarge and color-correct the image to obtain a coarsely recovered image. The spectral domain guidance stage employs a denoising diffusion probability model, taking the coarsely recovered image and a diffusion noise map as input. A spectral modeling module extracts global features in the frequency domain and combines this with a U-Net structure to predict the residual information between the high-resolution image and the coarse image, which is used to reconstruct image details, resulting in a residual prediction map. The coarsely recovered image and the residual prediction map are then added to obtain the final super-resolution reconstructed image.

[0047] After constructing the two-stage deep learning model, the model is trained. An underwater image training set is constructed, which includes pairs of low-resolution underwater images and corresponding high-resolution underwater images. The underwater image training set is input into the two-stage image reconstruction model. Pixel-level L1 loss, diffusion residual loss, and high-frequency loss based on discrete wavelet transform are jointly used to optimize the two-stage image reconstruction model end-to-end. The optimization algorithm iteratively updates the parameters of the two-stage image reconstruction model until the two-stage image reconstruction model converges, resulting in the trained two-stage image reconstruction model.

[0048] Specifically, a dataset is constructed that includes pairs of low-resolution underwater images with their corresponding high-resolution images. The low-resolution images are then input into the constructed deep learning model, which sequentially goes through a coarse recovery stage and a spectrum-guided diffusion stage to obtain the predicted output image and the residual image. Based on the difference between the predicted image and the real high-resolution image, pixel-level L1 loss, diffusion residual loss, and frequency domain loss based on discrete wavelet transform are jointly calculated. The model parameters are then backpropagated and iteratively updated through an optimizer until the loss function converges, resulting in a trained underwater image super-resolution model.

[0049] Image inference is performed using the trained model. The input is a low-resolution image to be reconstructed. After coarse restoration, spectral guidance and diffusion residual reconstruction, the output is an enhanced high-resolution image.

[0050] In an embodiment of the present invention, the coarse recovery module is a lightweight convolutional neural network with no more than 0.02M parameters, including upsampling operations and multiple 3×3 convolutional layers, used to recover the basic structure and color information of the image.

[0051] In an embodiment of the present invention, the spectrum modeling module in the spectrum domain guidance stage uses Fast Fourier Transform (FFT) to transform the feature map from the spatial domain to the spectrum domain. After extracting global frequency features in the spectrum domain, it restores them back to the spatial domain through Inverse Fast Fourier Transform (iFFT).

[0052] In an embodiment of the present invention, the diffusion model is based on the Denoising Diffusion Probability Model (DDPM), which adds Gaussian noise through a forward diffusion process and uses a reverse diffusion process to predict the residual image, thereby achieving high-frequency detail reconstruction.

[0053] In an embodiment of the present invention, the residual prediction network adopts a U-Net structure, which has downsampling, bottleneck layers and upsampling paths, and is used to extract and fuse feature information at different scales.

[0054] In embodiments of the present invention, the joint loss function used in model training includes:

[0055] Pixel-domain L1 loss is used to assess the reconstruction error between the coarsely restored image and the high-resolution image; diffusion residual L1 loss is used to optimize the diffusion model's prediction of residual information; and Discrete Wavelet Transform (DWT) loss is used to measure the image's high-frequency detail recovery performance in different directions.

[0056]

[0057] Among them, H i V i D i x represents the i-th downsampling level r Sub-band, and Corresponding to the same downsampling level The subband, J represents the total number of reduced sampling levels;

[0058]

[0059] The following detailed description of the diffusion model-based underwater image super-resolution reconstruction method of the present invention is provided through specific embodiments.

[0060] Underwater image super-resolution reconstruction methods based on diffusion models, such as Figure 2 As shown, it includes the following steps:

[0061] Step 1, Training set preprocessing;

[0062] First, high-resolution underwater images are used as input, and low-resolution images with realistic degradation features are generated using Gaussian blur, bicubic downsampling, and style perturbation. A training set {x} is then constructed. L ,x H}, where x L For low-resolution images, x H This corresponds to a high-resolution image. The magnification factor used in the experimental setup is r∈{2,3,4,8}.

[0063] Step 2, construct the learning model, which includes the following steps:

[0064] Step 201, Coarse Recovery Module:

[0065] Inputting a low-resolution underwater image, a lightweight coarse-reconstruction module performs basic reconstruction. This module first uses a bilinear upsampling layer to upscale the spatial resolution to the target size, followed by feature extraction and color bias correction using three cascaded 3×3 convolutional layers. This design achieves efficient low-frequency reconstruction with only 0.02M parameters.

[0066] Step 202, Spectrum-guided diffusion module:

[0067] The preliminary recovered features from step 201 are input into a joint learning network for long and short dependencies based on a diffusion probability model. This network introduces a spectral domain transform (ST) module, which uses Fast Fourier Transform to extract global frequency domain features to enhance long-distance dependency modeling capabilities. Simultaneously, the backbone diffusion module progressively predicts the residuals using a U-Net structure to achieve high-frequency detail restoration.

[0068] Step 203, Residual Fusion Module;

[0069] The residual generated in step 202 is added to the coarsely restored image to generate the final high-resolution image.

[0070] Step 3: Train the deep learning model;

[0071] Low-resolution underwater images from pre-processed underwater training data are used as input to a deep learning model, while corresponding high-resolution underwater images are used as training targets. An end-to-end joint optimization strategy is employed to train the coarse recovery module and the spectral domain guided diffusion module within the model to obtain optimal weights and bias parameters. This results in a deep learning model capable of accurately reconstructing low-resolution underwater images from high-resolution images. In this example, the magnification factor r can be set to 2, 3, 4, or 8. The model magnifies the input low-resolution image by a factor of r and outputs a high-resolution image. During training, a joint objective function incorporating coarse recovery loss, diffusion loss, and DWT high-frequency loss is used for optimization, ensuring the model achieves a dual improvement in visual perception and quantization metrics on the USR-248 and UFO-120 datasets.

[0072] The specific training process is as follows: Steps A1 to A4:

[0073] A1: The original low-resolution underwater image is input into the coarse reconstruction module. In this module, bilinear upsampling is first performed to upscale the resolution to the target size. Then, three cascaded 3×3 convolutional layers are used for basic feature extraction and color deviation correction to generate a preliminary reconstructed image. This lightweight module contains only 0.02M parameters, significantly reducing computational load while achieving basic reconstruction.

[0074] A2: The initially reconstructed image is concatenated with the diffused noise and then input into the spectral domain guided diffusion module. First, multi-scale feature extraction is performed through the shrinking path of U-Net: convolution and downsampling operations are applied to gradually increase the number of feature channels and reduce the spatial resolution to capture the multi-scale representation of the image. A spectral domain transformation submodule (ST) is embedded in the intermediate layer to map the features to the frequency domain through fast Fourier transform. The computational efficiency is optimized by utilizing conjugate symmetry. Convolution, batch normalization, and ReLU activation are performed in the frequency domain to extract global contextual features, which are then restored to the spatial domain by inverse FFT. Finally, the spatial resolution is gradually restored through upsampling and convolution operations in the expansion path, and the predicted residual is output. This process effectively integrates local details and global dependencies.

[0075] A3: The predicted residual is added to the coarsely restored image to generate a high-resolution reconstruction result. This fusion operation compensates for high-frequency details, ensuring that the output image has rich texture information and accurate color representation.

[0076] A4: Train the entire deep learning model, calculating a joint loss function consisting of three parts: the L1 loss between the coarsely restored image and the true high-resolution image, the L1 loss between the predicted residual and the true residual, and the MSE loss of the horizontal / vertical / diagonal high-frequency subbands based on discrete wavelet transform. The Adam optimizer is used to update the model parameters until the loss function converges to a stable state, ultimately obtaining the optimized underwater image super-resolution model.

[0077] like Figure 3 and Figure 4 As shown, based on the above description of the loss function, in this embodiment, the underwater low-resolution image x is first processed... L The input is a coarse recovery module, which first extracts shallow features through a 3×3 convolutional layer to generate an initial feature map. This initial feature map is then input... Figure 3 The lightweight reconstruction network shown performs resolution enhancement and color correction through three sets of cascaded 3×3 convolutional layers (64 output channels per set, ReLU activated), ultimately outputting a preliminary reconstructed image.

[0078] X CRM =Conv3×3(3)(ReLU(Conv3×3(2)(ReLU(Conv3×3(1)(Finit))))

[0079] The module has only 0.02M parameters, which significantly reduces the computational load.

[0080] In such Figure 5 In the spectral domain guided diffusion stage shown, X CRM With noise x t The input U-Net architecture is concatenated, and the channels are compressed to 32 dimensions through 1×1 convolution. It is then mapped to the frequency domain through Fast Fourier Transform. Next, the frequency domain conjugate symmetry is used to process only half of the frequency points, and 3×3 convolution feature extraction is performed. Finally, the spatial features are restored through inverse FFT.

[0081] F spec =IFFT(Conv(FFT(F)))

[0082] Step 4: Use the trained learning model to perform underwater image super-resolution testing.

[0083] The underwater image super-resolution reconstruction system based on the diffusion model includes: a deep learning model building unit and a deep learning model training unit; the deep learning model built by the deep learning model building unit includes a coarse recovery module, a spectrum-guided diffusion reconstruction module, and a spectrum transformation module.

[0084] Coarse Restoration Module: This module receives low-resolution underwater images as input and employs a lightweight network structure consisting of three 3×3 convolutional layers. It first performs upsampling and color correction on the image and extracts shallow structural information. The output preliminary restored image not only possesses basic structure and texture but also provides conditional guidance information for subsequent diffusion modeling stages. This module has only 0.02M parameters, making it suitable for deployment on resource-constrained underwater platforms.

[0085] The spectrum-guided diffusion reconstruction module models the image residual through two stages: forward perturbation and backward generation. First, noise is injected into the target image to form a diffusion sequence. Then, a pre-trained U-Net structure is used to perform backward diffusion prediction, guided by a coarsely reconstructed image. In each diffusion inversion step, the network progressively learns the reconstruction characteristics of high-frequency information, thereby generating a clearer and more detailed image. This module is designed as a residual prediction, avoiding direct regression of the entire image and improving the stability and high-frequency representation capability of the reconstruction.

[0086] Spectrum Transformation Module: To further enhance the model's ability to model global image information, a spectrum transformation module is embedded between the bridging layer and the decoding layer of the diffusion network. This module uses Fast Fourier Transform (FFT) to map intermediate features from the spatial domain to the spectrum domain, extracts global contextual information through convolution operations, and then restores it to the spatial domain through Inverse Fourier Transform (IFFT) for fusion with local structural features. The introduction of the spectrum module enhances the model's ability to perceive long-range dependencies and texture alignment, effectively improving the realism and consistency of the recovered image.

[0087] Deep learning model training unit: This unit is used to train the network structure described above. During the training phase, it uses low-resolution images x containing pairs. L With high-resolution image x H The model utilizes a real underwater dataset. It employs a joint optimization of three loss functions: pixel reconstruction loss to align the coarse reconstruction output with the real image; diffusion residual loss to guide the diffusion model in accurately learning noise removal strategies; and high-frequency loss to calculate the high-frequency sub-band error of the image through discrete wavelet transform, enhancing texture detail restoration. The weighted sum of these three loss functions forms the total loss, which drives network convergence and obtains the optimal parameter configuration.

[0088] After system training, the inference phase begins, enabling super-resolution reconstruction of any input low-resolution underwater image. The inference process first generates a preliminary image through a coarse recovery module, then iteratively reconstructs it using a diffusion module, and finally performs spectral-guided enhancement in an intermediate layer, outputting a high-resolution image with higher visual quality. This method boasts advantages such as clear structure, strong reconstruction effect, and high computational efficiency, making it suitable for applications in real-world underwater environments, including image enhancement, target recognition, and underwater mapping.

[0089] In this embodiment, to verify the system's performance, the method of this invention is compared with existing typical underwater image super-resolution methods on the public dataset UFO-120. The peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) of the images are tested under scaling factors of 2, 3, and 4, respectively. The experimental results are shown in Table 1.

[0090]

[0091] Table 1: Comparison of PSNR and SSIM values ​​on the standard dataset UFO-120 with scaling factors r of 2, 3, and 4.

[0092] As shown in Table 1, the method of this invention performs excellently across all three scaling factors (×2, ×3, and ×4). At ×2 scaling, its PSNR is second only to SRGAN, but significantly better than other diffusion models. At ×3 scaling, the method of this invention achieves the highest PSNR overall. At ×4 scaling, the method of this invention is second only to Deep WaveNet. This indicates that the method of this invention leads in overall pixel-level fidelity, especially at medium scaling (×3).

[0093] To more intuitively demonstrate the advantages of the method of this invention in terms of visual quality, a visual comparison was conducted between the method of this invention and several mainstream underwater models on the UFO-120 test set under different test conditions with scale factors of 2, 3, and 4. The results are as follows: Figure 6 As shown in the figure. Visual comparisons demonstrate that the super-resolution images generated by this invention outperform other methods in terms of edge sharpness, texture detail restoration, and color fidelity, and their visual perception quality is closer to that of real high-resolution images.

[0094] Therefore, the experimental results fully demonstrate the effectiveness and advancement of the method of this invention in actual underwater scenarios, significantly improving the ability to restore high-frequency texture details. Meanwhile, user research further verifies that its subjective score exceeds 70%, effectively balancing the consistency between quantitative indicators and visual perception, and solving the problems of detail blurring and color distortion in traditional methods.

[0095] This invention proposes a diffusion-based underwater image super-resolution reconstruction method. It constructs a deep learning model with a spectral information-guided mechanism, including a coarse recovery module, a spectral transformation module, a residual prediction module, and a high-frequency detail loss function based on discrete wavelet transform. Through end-to-end joint training, the optimal network weights and bias parameters are obtained, thereby achieving the goal of restoring low-resolution underwater images to high-resolution images. This invention employs a more lightweight network structure design, including a coarse recovery module with fewer parameters and a spectral domain transformation module based on Fourier convolution. This not only effectively accelerates model training and inference speed but also significantly reduces the overall computational cost of the network. A spectral domain-guided mechanism is introduced into the model structure, achieving accurate reconstruction of high-frequency image details through residual prediction learning. Simultaneously, it integrates global spectral features and local spatial features, giving the model excellent global structure modeling and local texture restoration capabilities. The proposed multi-scale detail-guided training strategy, by introducing a discrete wavelet transform loss function, fully exploits the high-frequency features of the image at multiple scales, thereby improving the detail preservation and visual perception of the generated image. Experimental results show that the method of this invention significantly improves the subjective visual quality of underwater images while maintaining excellent performance on evaluation metrics.

[0096] Next, with reference to the accompanying drawings, we describe the underwater image super-resolution reconstruction system based on the diffusion model proposed in an embodiment of the present invention.

[0097] Figure 7 This is an example diagram of an underwater image super-resolution reconstruction system based on a diffusion model according to an embodiment of the present invention.

[0098] like Figure 7 As shown, the underwater image super-resolution reconstruction system 10 based on the diffusion model includes: an acquisition module 100, an input module 200, and a reconstruction module 300.

[0099] The system includes an acquisition module 100 for acquiring the underwater low-resolution image to be reconstructed; an input module 200 for inputting the underwater low-resolution image to be reconstructed into a pre-trained image reconstruction model with a two-stage structure; and a reconstruction module 300 for using a lightweight coarse restoration module in the image reconstruction model to enlarge and color-correct the underwater low-resolution image to obtain a coarse restoration image. Using a denoising diffusion probability model, the coarse restoration image and the diffusion noise map are combined with a spectrum modeling module to extract global features in the frequency domain. A residual prediction network with a U-Net structure is then used to predict the residual information between the high-resolution image and the coarse restoration image to obtain a residual prediction map. Finally, the coarse restoration image and the residual prediction map are added together to obtain the reconstructed underwater high-resolution image.

[0100] In the underwater image super-resolution reconstruction system based on the diffusion model, the deep learning model constructed by the image reconstruction model construction module adopts a two-stage structure, including a coarse recovery stage and a spectral domain guided diffusion stage.

[0101] The system comprises several stages: First, the coarse restoration stage upsamples and corrects color deviations in the input low-resolution underwater image using a coarse restoration module. It then extracts low-frequency features from the image using a lightweight convolutional structure to generate an initial restored image. Second, the spectrum-guided diffusion stage models and progressively restores the image residual using a diffusion probability model based on a U-Net structure. High-frequency details are reconstructed through iterative denoising of the residual space guided by the noise. Third, the spectrum transformation module maps the image signal to the spectrum domain using fast Fourier transform and inverse transform, extracting image information containing global context to enhance the modeling capability of the diffusion model and improve detail restoration accuracy. Finally, the system fuses the residual image with the coarse restored image to generate a high-resolution underwater image with realistic colors and rich details.

[0102] The image reconstruction model training module is used to train the aforementioned deep learning model. It employs a joint loss function consisting of three parts: first, a pixel-level L1 loss between the coarsely restored image and the true high-resolution image, used to optimize the initial upsampling effect; second, a difference loss between the diffusion model prediction residual and the true residual, used to improve the accuracy of residual modeling; and third, a high-frequency loss based on discrete wavelet transform (DWT), which improves the image edge and texture restoration capability by comparing the high-frequency subband errors in the horizontal, vertical, and diagonal directions. The training data uses paired low-resolution and high-resolution underwater image samples. Gaussian noise perturbation and multi-step diffusion sampling are introduced during training, combined with a conditional noise prediction network for optimization and iteration, ultimately obtaining an underwater image super-resolution model that can be efficiently executed during the inference stage. The system can be deployed on computing devices to execute the trained model for automatic super-resolution reconstruction of any low-resolution underwater image.

[0103] It should be noted that the foregoing explanation of the underwater image super-resolution reconstruction method based on the diffusion model also applies to the underwater image super-resolution reconstruction system based on the diffusion model in this embodiment, and will not be repeated here.

[0104] The underwater image super-resolution reconstruction system based on a diffusion model proposed in this embodiment of the invention features a coarse restoration module with an extremely lightweight structure that significantly reduces computational resource requirements while maintaining basic reconstruction capabilities. The spectral domain-guided diffusion module innovatively combines frequency domain feature extraction with residual diffusion prediction, reducing computational complexity through frequency domain conjugate symmetry and effectively enhancing high-frequency detail recovery capabilities. The two-stage collaborative design significantly improves the visual perception quality of underwater images without increasing model complexity.

[0105] Figure 8This is a schematic diagram of an electronic device provided in an embodiment of the present invention. The electronic device may include:

[0106] The memory 801, the processor 802, and the computer program stored on the memory 801 and capable of running on the processor 802.

[0107] When the processor 802 executes the program, it implements the underwater image super-resolution reconstruction method based on the diffusion model provided in the above embodiments.

[0108] Furthermore, electronic devices also include:

[0109] Communication interface 803 is used for communication between memory 801 and processor 802.

[0110] The memory 801 is used to store computer programs that can run on the processor 802.

[0111] The memory 801 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0112] If the memory 801, processor 802, and communication interface 803 are implemented independently, then the communication interface 803, memory 801, and processor 802 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0113] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 803 are integrated on a single chip, then the memory 801, processor 802, and communication interface 803 can communicate with each other through an internal interface.

[0114] The processor 802 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0115] This embodiment also provides a computer-readable storage medium storing a computer program, characterized in that the program, when executed by a processor, implements the above-described diffusion-based underwater image super-resolution reconstruction method.

[0116] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0117] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0118] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0119] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0120] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A method for super-resolution reconstruction of underwater images based on a diffusion model, characterized in that, Includes the following steps: Acquire low-resolution underwater images to be reconstructed; The underwater low-resolution image to be reconstructed is input into a pre-trained image reconstruction model with a two-stage structure. The underwater low-resolution image is magnified and color-corrected using the lightweight coarse restoration module in the image reconstruction model to obtain a coarse restoration image. A denoising diffusion probability model is used to extract global features in the frequency domain from the coarse restoration image and the diffusion noise map through the spectrum modeling module. The residual prediction network with U-Net structure is then used to predict the residual information between the high-resolution image and the coarse restoration image to obtain a residual prediction map. The coarse restoration image and the residual prediction map are then added together to obtain the reconstructed underwater high-resolution image.

2. The method according to claim 1, characterized in that, The method further includes: An underwater image training set is constructed, consisting of paired low-resolution underwater images and corresponding high-resolution underwater images. This training set is then input into the two-stage image reconstruction model. Pixel-level L1 loss, diffusion residual loss, and high-frequency loss based on discrete wavelet transform are jointly employed to perform end-to-end optimization of the two-stage image reconstruction model. The optimization algorithm iteratively updates the parameters of the two-stage image reconstruction model until it converges, resulting in a trained two-stage image reconstruction model.

3. The method according to claim 1, characterized in that, In the coarse restoration stage, the coarse restoration module is a lightweight convolutional neural network with no more than 0.02M parameters, which includes upsampling operations and multiple 3×3 convolutional layers to restore the basic structure and color information of the image.

4. The method according to claim 1, characterized in that, During the spectral domain guidance phase, the spectrum modeling module uses Fast Fourier Transform to transform the feature map from the spatial domain to the spectral domain. After extracting global frequency features in the spectral domain, it then restores them back to the spatial domain using Inverse Fast Fourier Transform.

5. The method according to claim 1, characterized in that, The diffusion model is based on a denoising diffusion probability model. Gaussian noise is added through a forward diffusion process, and the residual image is predicted by a reverse diffusion process.

6. The method according to claim 1, characterized in that, The residual prediction network adopts a U-Net structure, which has downsampling, bottleneck layers and upsampling paths, and is used to extract and fuse feature information at different scales.

7. The method according to claim 2, characterized in that, Pixel-domain L1 loss is used to assess the reconstruction error between the coarsely restored image and the high-resolution image. Diffusion residual L1 loss is used to optimize the diffusion model's prediction of residual information. Discrete wavelet transform high-frequency loss is used to measure the high-frequency detail restoration effect of the image in different directions. for: Among them, H i V i D i x represents the i-th downsampling level r Sub-band, x r The residual between the high-resolution image and the coarsely restored image. Corresponding to the same downsampling level sub-band, The noise residual generated for network inference, where J represents the total number of reduced sampling levels; Joint loss total for: in, The L1 loss is the difference between the high-resolution image and the coarsely restored image. The L1 loss is the difference between the initial noise residual and the noise residual generated by network inference.

8. An underwater image super-resolution reconstruction system based on a diffusion model, characterized in that, include: The acquisition module is used to acquire low-resolution underwater images to be reconstructed. The input module is used to input the underwater low-resolution image to be reconstructed into a pre-trained image reconstruction model containing a two-stage structure. The reconstruction module is used to enlarge and color correct the underwater low-resolution image using the lightweight coarse restoration module in the image reconstruction model to obtain a coarse restoration image. Using a denoising diffusion probability model, the coarse restoration image and the diffusion noise map are processed by a spectrum modeling module to extract global features in the frequency domain. The residual prediction network with a U-Net structure is then used to predict the residual information between the high-resolution image and the coarse restoration image to obtain a residual prediction map. Finally, the coarse restoration image and the residual prediction map are added together to obtain the reconstructed underwater high-resolution image.

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the underwater image super-resolution reconstruction method based on the diffusion model as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the underwater image super-resolution reconstruction method based on the diffusion model as described in any one of claims 1-7.

Images