Underwater Image Enhancement Method and Device Based on Multi-Resolution Diffusion and Color Correction

Abstract

The present invention discloses an underwater image enhancement method and device based on multi-resolution diffusion and color correction, belonging to the technical field of image processing, including: using the inverse diffusion process of the diffusion model to reconstruct the denoised image with the underwater image as the guiding condition, adopting a variable-resolution sampling scheme for the image during the denoising process of the full diffusion period to gradually improve the image resolution, and at the same time, in each time step, using a global color corrector to correct the color of the denoised reconstructed image to correct the global degradation color deviation phenomenon of the image during the denoising process. The color-corrected reconstructed image is used as the input for the next time step. In this way, through the variable-resolution sampling scheme in the full diffusion period combined with the global color correction in each diffusion time step, the technical problem that it is difficult to balance the global color consistency and local detail fidelity when processing images in different water environments is solved, and at the same time, the relationship between the calculation efficiency and the recovery quality can be balanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of graphic processing, and particularly relates to an underwater image enhancement method and device based on multi-resolution diffusion and color correction. Background Technique

[0002] Underwater images present serious quality degradation problems due to special optical environments, mainly manifested as color distortion, low contrast, detail loss, scattered light interference, etc. Traditional underwater image enhancement methods mainly include physical model-based methods and image processing-based methods. Physical model-based methods restore images by simulating underwater light propagation characteristics, but often require accurate optical parameter estimation, and it is difficult to obtain accurate parameters in complex underwater environments. Image processing-based methods such as histogram equalization and white balance are simple to operate, but it is often difficult to balance color accuracy and detail retention simultaneously when dealing with complex underwater scenes.

[0003] With the development of deep learning technology, methods based on generative adversarial networks (GANs) and convolutional neural networks (CNNs) have been introduced into the underwater image enhancement task. These methods improve the processing effect through end-to-end training. However, existing deep learning methods still face the technical problem of difficult to balance global color consistency and local detail fidelity when processing high-resolution underwater images. Especially in different water environments (such as turbid waters, green waters, or blue waters, etc.), due to significant differences in underwater optical characteristics, existing methods are difficult to adaptively handle the image enhancement requirements of different underwater environments.

[0004] Recently popular diffusion models have shown excellent performance in the field of image generation and restoration. However, there is still a lack of effective solutions on how to design a dedicated diffusion model structure for the special optical degradation characteristics of underwater images to achieve higher-quality underwater image enhancement effects. Summary of the Invention

[0005] In view of the above, the purpose of the present invention is to provide an underwater image enhancement method and device based on multi-resolution diffusion and color correction to solve the technical problem that it is difficult to balance global color consistency and local detail fidelity in existing underwater image enhancement technologies when processing images in different water environments. The method and device of the present invention can be applied to scenarios that require high-quality underwater images, such as ocean exploration, underwater robot vision, underwater cultural relics archaeology, etc.

[0006] To achieve the above invention purpose, an underwater image enhancement method based on multi-resolution diffusion and color correction provided by an embodiment includes the following steps:

[0007] The inverse diffusion process of the diffusion model uses the underwater image as a guiding condition for denoising image reconstruction. During the denoising process of the full diffusion cycle, a variable resolution sampling scheme is adopted for the image to gradually improve the image resolution. At the same time, at each time step, a global color corrector is used to correct the color of the denoised reconstructed image to correct the global degradation color deviation phenomenon of the image during the denoising process. The reconstructed image after color correction is used as the input for the next time step.

[0008] Preferably, the variable resolution sampling scheme adopted for the image during the denoising process of the full diffusion cycle includes:

[0009] By setting a downsampling scheduling sequence to control the image resolution during the denoising process. Specifically, at the initial stage of denoising, a low resolution is used to sample the image, and noise is removed from the sampled image. As the time step increases, the sampling resolution is gradually increased to reach the high resolution of the original image at the last time step.

[0010] Preferably, the global color corrector includes an image feature extraction module, a global feature extraction module, a local feature extraction module, and a fusion correction module;

[0011] The image feature extraction module is used to extract the latent space features of the image from the reconstructed image;

[0012] The global feature extraction module is used to extract global features based on the latent space features and the time step features after time position encoding;

[0013] The local feature extraction module is used to obtain local features by performing global vector modulation based on the latent space features and the reconstructed image;

[0014] The fusion correction module is used to perform color fusion and correction based on the global features and the local features to obtain the corrected reconstructed image.

[0015] Preferably, the image feature extraction module, the local feature extraction module, and the fusion correction module adopt convolutional neural networks.

[0016] Preferably, the global feature extraction module adopts a Transformer module.

[0017] Preferably, the denoising network and the global color corrector in the diffusion model are optimized before being applied. The corrected reconstructed image obtained at the last time step is used as the enhanced image. The loss function used during optimization includes constructing a loss function based on the difference between the input target image and the generated enhanced image.

[0018] To achieve the above-mentioned invention objectives, the embodiments further provide an underwater image enhancement device based on multi-resolution diffusion and color correction, including a memory and one or more processors. An executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the above-mentioned underwater image enhancement method based on multi-resolution diffusion and color correction.

[0019] To achieve the above-mentioned invention objectives, the embodiments further provide a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, it implements the above-mentioned underwater image enhancement method based on multi-resolution diffusion and color correction.

[0020] To achieve the above-mentioned invention objectives, the embodiments further provide a computer product, which includes a computer program. When the computer program is executed by a processor, it implements the above-mentioned underwater image enhancement method based on multi-resolution diffusion and color correction.

[0021] Compared with the prior art, the beneficial effects of the present invention at least include:

[0022] The present invention realizes progressive underwater image enhancement from low resolution to high resolution through a variable-resolution sampling scheme in the full diffusion period, effectively reducing the computational complexity; on this basis, combined with a global color corrector designed for the characteristics of underwater images, global color correction of the reconstructed image is performed at each diffusion time step, solving the problems of color distortion and scattered light interference in different water environments; adopting a variable-resolution sampling scheme enhances the model's ability to restore local details of underwater images; achieving a balance between global color consistency and local detail fidelity, the enhanced underwater images obtained have higher application value in underwater target recognition, marine resource investigation, etc. Description of the Drawings

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 is a flowchart of the underwater image enhancement method based on multi-resolution diffusion and color correction provided by the embodiment;

[0025] Figure 2 is a structural schematic diagram of the global color corrector provided by the embodiment;

[0026] Figure 3 is a training schematic diagram of the diffusion model and the denoising network provided by the embodiment. Detailed implementation manners

[0027] To make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific implementation manners described herein are only used to explain the present invention and do not limit the protection scope of the present invention.

[0028] The inventive concept of the present invention is: aiming at the technical problem that it is difficult to balance the global color consistency and local detail fidelity when processing images in different underwater environments by the existing underwater image enhancement technologies. Embodiments of the present invention provide an underwater image enhancement method and device based on multi-resolution diffusion and color correction, by designing a variable-resolution sampling scheme for the diffusion model and a global color corrector, to model and correct the special degradation characteristics of underwater images. The variable-resolution sampling scheme allows the model to capture image features at different resolutions, and the global color corrector effectively solves the color deviation and scattering problems of the image during the diffusion process. In this way, a high-quality underwater image enhancement effect that can maintain both global color consistency and retain local detail information can be achieved.

[0029] Based on the above inventive concept, as Figure 1 shown, an underwater image enhancement method based on multi-resolution diffusion and color correction provided by an embodiment includes the following steps:

[0030] Using the inverse diffusion process of the diffusion model to perform denoising image reconstruction with the underwater image as the guiding condition, and adopting a variable-resolution sampling scheme for the image during the denoising process of the full diffusion period to gradually increase the image resolution. At the same time, in each time step, a global color corrector is used to perform color correction on the denoised reconstructed image to correct the global degradation color deviation phenomenon of the image during the denoising process. The reconstructed image after color correction is used as the input for the next time step.

[0031] In the embodiment, the variable-resolution sampling method adopted for the image during the denoising process of the full diffusion period can gradually increase the image resolution by gradually increasing the sampling resolution. As Figure 1 shown, the scheme includes: controlling the image resolution during the denoising process by setting a downsampling scheduling sequence. Specifically, at the initial stage of denoising, a low resolution (such as 1 / 4 or 1 / 8 of the original image) is used to sample the image, and noise is removed from the sampled image. This can reduce the computational amount and focus on the global structure of the image. As the time step increases, the sampling resolution is gradually increased to reach the high resolution of the original image at the last time step to restore the detail information. This resolution progressive processing scheme can balance the relationship between computational efficiency and restoration quality.

[0032] In the embodiment, a global color corrector is constructed to correct the reconstructed image at each time step, so as to realize the correction of the global degradation color cast phenomenon during the diffusion process. As Figure 2 shown, the global color corrector includes an image feature extraction module, a global feature extraction module, a local feature extraction module, and a fusion correction module; among them, the image feature extraction module is used to extract the latent space features of the image from the reconstructed image; the global feature extraction module is used to extract global features based on the latent space features and the time step features after time position encoding; the local feature extraction module is used to perform global vector modulation on the latent space features and the reconstructed image to obtain local features; the fusion correction module is used to perform color fusion and correction based on the global features and the local features to obtain the corrected reconstructed image.

[0033] Among them, the image feature extraction module can preferably adopt a lightweight convolutional neural network to extract the latent space features of the two-dimensional image from the input reconstructed image. The local feature extraction module can preferably adopt a convolutional neural network to extract the local features corresponding to the local information based on the input latent space features and the reconstructed image. The global feature extraction module can preferably adopt a Transformer module to extract the global features corresponding to the global information based on the latent space features and the time step features after time position encoding. The fusion correction module can preferably adopt a convolutional layer to perform color correction based on the combined input global features and local features to obtain the corrected reconstructed image.

[0034] In the embodiment, both the denoising network and the global color corrector in the diffusion model need to be trained before being applied to optimize the parameters. The specific training process is as Figure 3 shown. A target image is constructed as a sample, and the target image is used as the input of the diffusion model. Random noise is added to the image at each time step during the forward diffusion process to obtain a noisy image, and a completely noisy image is obtained at the end of the full time step cycle. During the inverse diffusion process, a time step is randomly selected, and a reconstructed image is generated based on the denoising network. Moreover, during the denoising process, the underwater image and the time step are used as guiding conditions, and the reconstructed image is also reconstructed and color-corrected using the global color corrector after each time step. A loss function is constructed based on the RGB difference between the input target image and the corrected reconstructed image, and the parameters of the denoising network and the global color corrector are optimized using the loss function.

[0035] After the above training, when enhancing underwater images, random noise is used as the input, and at each time step, the underwater image and the time step are used as guiding conditions to predict the denoising noise through the trained denoising network, and the input image at each time step is denoised to obtain a reconstructed image. At the same time, at each time step, the trained global color corrector is used for color correction, and the color-corrected reconstructed image is used as the input for the next time step until the finally generated reconstructed image is used as the enhanced image of the underwater image.

[0036] Based on the same inventive concept, the embodiment also provides an underwater image enhancement device based on multi-resolution diffusion and color correction, including a memory and one or more processors. An executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the above-mentioned underwater image enhancement method based on multi-resolution diffusion and color correction, specifically including the following steps:

[0037] The inverse diffusion process of the diffusion model is used to reconstruct the denoised image with the underwater image as the guiding condition, and a variable-resolution sampling scheme is adopted for the image during the denoising process in the full diffusion period to gradually improve the image resolution. At the same time, at each time step, the global color corrector is used to perform color correction on the denoised reconstructed image to correct the global degradation color deviation phenomenon of the image during the denoising process, and the color-corrected reconstructed image is used as the input for the next time step.

[0038] For the underwater image enhancement device based on multi-resolution diffusion and color correction provided by the embodiment, at the hardware level, in addition to including a processor and a memory, it also includes an internal bus, a network interface, a memory, and other hardware required for other services. The memory is a non-volatile memory, and the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned underwater image enhancement method based on multi-resolution diffusion and color correction. Of course, in addition to the software implementation method, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or a logic device.

[0039] Based on the same inventive concept, the embodiment also provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, it implements the above-mentioned underwater image enhancement method based on multi-resolution diffusion and color correction, specifically including the following steps:

[0040] The inverse diffusion process of the diffusion model uses the underwater image as a guiding condition for denoising image reconstruction. During the denoising process of the full diffusion period, a variable-resolution sampling scheme is adopted for the image to gradually improve the image resolution. At the same time, in each time step, a global color corrector is used to correct the color of the denoised reconstructed image to correct the global degradation color bias phenomenon of the image during the denoising process. The color-corrected reconstructed image is used as the input for the next time step.

[0041] In an embodiment, the computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data.

[0042] Based on the same inventive concept, the embodiment also provides a computer product that includes a computer program. When the computer program is executed by a processor, it implements the above-mentioned underwater image enhancement method based on multi-resolution diffusion and color correction, including the following steps:

[0043] The inverse diffusion process of the diffusion model uses the underwater image as a guiding condition for denoising image reconstruction. During the denoising process of the full diffusion period, a variable-resolution sampling scheme is adopted for the image to gradually improve the image resolution. At the same time, in each time step, a global color corrector is used to correct the color of the denoised reconstructed image to correct the global degradation color bias phenomenon of the image during the denoising process. The color-corrected reconstructed image is used as the input for the next time step.

[0044] The above-described specific embodiments have detailed the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, supplements, equivalent replacements, etc. made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An underwater image enhancement method based on multi-resolution diffusion and color correction, characterized in that It includes the following steps: Using the inverse diffusion process of the diffusion model, the underwater image is used as a guiding condition for denoising image reconstruction. During the denoising process of the full diffusion cycle, a variable resolution sampling scheme is adopted for the image to gradually improve the image resolution. At the same time, in each time step, a global color corrector is used to correct the color of the denoised reconstructed image to correct the global degradation color deviation phenomenon of the image during the denoising process. The color-corrected reconstructed image is used as the input for the next time step. The variable resolution sampling scheme adopted for the image during the denoising process of the full diffusion cycle includes: controlling the image resolution during the denoising process by setting a downsampling scheduling sequence. Specifically, at the beginning of denoising, the image is sampled at a low resolution, and the noise of the sampled image is removed. As the time step increases, the sampling resolution is gradually increased to reach the high resolution of the original image at the last time step. The global color corrector includes an image feature extraction module, a global feature extraction module, a local feature extraction module, and a fusion correction module. Among them, the image feature extraction module is used to extract the latent space features of the image from the reconstructed image; the global feature extraction module is used to extract global features based on the latent space features and the time step features after time position encoding; the local feature extraction module is used to obtain local features by performing global vector modulation on the latent space features and the reconstructed image; the fusion correction module is used to perform color fusion and correction based on the global features and the local features to obtain the corrected reconstructed image.

2. The underwater image enhancement method based on multi-resolution diffusion and color correction according to claim 1, characterized in that The image feature extraction module, the local feature extraction module, and the fusion correction module adopt a convolutional neural network.

3. The underwater image enhancement method based on multi-resolution diffusion and color correction according to claim 1, characterized in that, The global feature extraction module adopts a Transformer module.

4. The underwater image enhancement method based on multi - resolution diffusion and color correction according to claim 1, characterized in that The denoising network and the global color corrector in the diffusion model are optimized before being applied. The corrected reconstructed image obtained at the last time step is used as the enhanced image. The loss function used during optimization includes constructing a loss function based on the difference between the input target image and the generated enhanced image.

5. An underwater image enhancement device based on multi - resolution diffusion and color correction, including a memory and one or more processors, wherein the memory stores executable code, and is characterized in that, When the one or more processors execute the executable code, it is used to implement the underwater image enhancement method based on multi-resolution diffusion and color correction described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, A program is stored thereon, and when the program is executed by a processor, it implements the underwater image enhancement method based on multi-resolution diffusion and color correction described in any one of claims 1-4.

7. A computer product, characterized in that, It includes a computer program, and when the computer program is executed by a processor, it implements the underwater image enhancement method based on multi-resolution diffusion and color correction described in any one of claims 1-4.

Images