A unified image inpainting and enhancement method with diffusion prior generation
Through the combination of the generative diffusion model prior and pre-trained diffusion model, the problem of image repair and enhancement in the prior art is solved, and high-quality repair and enhancement of linear, nonlinear degraded and blind images is achieved, with good generalization.
Patent Information
- Application Number
- CN202310030233.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-09
- Publication Date
- 2025-06-06
- Estimated Expiration
- 2043-01-09
AI Technical Summary
The prior art is difficult to effectively solve the image repair and enhancement problems caused by multiple complex degradation, especially in the case of nonlinear degradation and unknown degradation models.
Generative diffusion model priors are adopted to mine powerful prior information through pre-trained diffusion models, and design simple and effective sampling methods to generate high-quality images consistent with the damaged image content.
It realizes unified repair and enhancement of linear, nonlinear degradation and blind images, has good generalization, is not limited to specific data sets, and can recover images with superimposed forms of multiple damaged forms.
Smart Images
Figure CN116071256B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to digital image processing, and more particularly to a method for generating a diffusion prior for unified image restoration and enhancement. Background Art
[0002] Image quality usually degrades during acquisition, storage, transmission, and rendering. The purpose of image restoration and enhancement is to overcome image degradation and improve image quality. Typically, restoration and enhancement tasks can be divided into two categories: 1) linear inverse problems, such as image super-resolution (SR), deblurring, padding, colorization, etc., where the degradation model is usually linear and known; 2) nonlinear or blind problems, such as image low-light enhancement and HDR image restoration, where the degradation model is nonlinear and unknown. For a specific linear degradation model, the image restoration problem can be solved by end-to-end supervised training of neural networks. Nevertheless, corrupted images in the real world usually have multiple complex degradations, and fully supervised methods are difficult to generalize in these cases. Summary of the invention
[0003] In view of the existing technology, most of the image restoration and enhancement technologies rely on the traditional generative adversarial network (GAN), and the diffusion model (DDPM, Denoising Diffusion Probabilistic Model) has recently been proven to have performance that exceeds GAN in the field of image generation. The purpose of the present invention is to propose a generative diffusion model prior to solve the unified image restoration and enhancement problem, and the image restoration task of the existing DDPM is limited to the linearly degraded image restoration task. The present invention further designs a centralized, simple and effective sampling method to mine powerful prior information from the pre-trained DDPM, which can better generate high-quality images consistent with the damaged image content, and has very good generalization and is not limited to a certain data set.
[0004] In order to solve the above technical problems, the technical solution of the present invention is as follows.
[0005] In a first aspect, the present invention proposes a method for generating a diffusion prior for unified image restoration and enhancement, the method comprising the following steps:
[0006] The diffusion model is pre-trained by adding noise and removing noise from the diffusion model containing T steps in total;
[0007] Generate compliance Diffusion prior image of the distribution ;
[0008] Get the current step t=T;
[0009] S100, based on the current step The corresponding noise is calculated about The mean and variance Σ;
[0010] Set the boot image and based on the boot image and damaged images , calculate the denoising gradient loss ,D is the degradation model, is the original image;
[0011] Denoising Gradient Loss , generating a compliance Distribution , is the scaling factor that controls the boot size;
[0012] Take t-1 as the current step , return to step S100 to obtain a relatively damaged image Restoration and enhancement of original natural images .
[0013] In the above technical solution, the degradation model The types include: degradation models for deblurring and super-resolution, degradation models for image completion, and degradation models for image coloring. As an improvement of the technical solution, the degradation model in the method of the present invention can also be a blind image restoration model.
[0014] The degradation model for deblurring and super-resolution is as follows:
[0015]
[0016] in: is the Gaussian kernel function or point spread function, represents convolution, is the original image, is the damaged image, the scale factor Perform downsampling operation ;
[0017] The degradation model of the image completion is as follows:
[0018]
[0019] in: is a binary mask; is the Hadamard product;
[0020] The image colorization degradation model is to transform the color image Keep as grayscale image Grayscale change, H is the height of the image, W is the width of the image;
[0021] The blind image restoration model is as follows:
[0022]
[0023] in: is the light factor, Light mask, and Unknown, in the reverse process of the diffusion model, and are randomly initialized and optimized synchronously, is the image to be degraded, This is the image obtained after degradation processing.
[0024] In the above technical solution, in the controlled generation process of the method, the variance does not participate in the offset calculation of the sampling mean at each step, and a guide image is set to obtain a better image generation effect.
[0025] In the above technical solution, based on the guidance image and damaged images , calculate the denoising gradient loss One implementation is to Add guidance, in this case, the gradient The calculation formula is:
[0026]
[0027] In the above formula: is the distance metric, is the scaling factor that controls the boot size, For quality enhancement loss, is the scaling factor for adjusting image quality; It is optional.
[0028] As an improvement of the above technical solution, based on the guide image and the damaged image , denoising gradient loss Another implementation is to Add guidance as follows:
[0029] By estimating The noise in the image Predict a clean image ;
[0030] Determine the number of guide steps to add based on the task, and gradually Add guidance to get the final guidance image , and based on the guidance image and damaged images Compute denoising gradients ;
[0031]
[0032] in: Used to add guide steps.
[0033] In the above technical solution, by estimating The noise in the image Predict a clean image :
[0034]
[0035] in: , , is the weight; It is a noise prediction model.
[0036] In the above technical solution, it is characterized in that, according to the task, determining the number of guide pictures to be added includes:
[0037] If the restored image is an HDR image, the number of guide pictures is 3; otherwise, the number of guide pictures is 1.
[0038] As an improvement of the above technical solution, a blind image restoration model is used to generate multiple guide images, and the original natural image is generated with the help of multiple guide images. The improved technical solution can achieve image restoration and enhancement under the guidance of any damaged image.
[0039] As an improvement of the above technical solution, if the size of the damaged image is different from the size of the image generated by the pre-trained diffusion model, the method estimates the degradation model at low resolution through a block-based strategy and a hierarchical guidance strategy. The degradation model is used to restore the image after restoring its resolution through interpolation, so that pictures of any size can be repaired. The specific implementation is as follows:
[0040] Scaling the size of the damaged original image to obtain a scaled image, thereby obtaining degradation model parameters of the scaled image;
[0041] Enlarging the degradation model parameters to the original image size by interpolation;
[0042] Based on the degradation model parameters, a fixed-size image block is generated by using an image block-based generation method, and each image block is a guide image, so that the corresponding restoration and enhancement images can be generated by using a method of generating diffusion priors;
[0043] For each restored and enhanced image block, the restored and enhanced image of the original image is obtained by combining the image blocks.
[0044] In the above technical solution, the quality enhancement loss Q comprises exposure control loss, color consistency loss and lighting smoothness loss.
[0045] In a second aspect, the present invention provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute any of the above methods.
[0046] In summary, the present invention has the following beneficial technical effects:
[0047] (1) The method of the present invention can be applied not only to image restoration with linear degradation, multilinear degradation, and nonlinear degradation, but also to image restoration guided by multiple images;
[0048] (2) Through layer-by-layer guidance and image patch-based methods, images of any size can be restored. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0050] Figure 1 , a schematic diagram of image restoration using a Generative Diffusion Prior (GDP) method in a specific implementation;
[0051] Among them: (a) Schematic diagram of GDP for unified image restoration; two variants of GDP; (b) a variant of GDP , (c) Another variant of GDP ;
[0052] Figure 2 In a specific embodiment Pseudocode screenshot of implementation;
[0053] Figure 3 In a specific embodiment Pseudocode screenshot of implementation;
[0054] Figure 4 , a flowchart of image restoration based on an image block strategy and a layer-guided strategy in a specific implementation. DETAILED DESCRIPTION
[0055] The diffusion model to which the present invention relates is defined as follows:
[0056] Diffusion models distribute complex data Transformed into a simple noise distribution , and recover data from noise, where is a Gaussian distribution, is the unit matrix. The diffusion model (DDPM) mainly includes (1.1) diffusion process and (1.2) reverse process.
[0057] (1.1) Diffusion process
[0058] The diffusion process is a Markov chain that gradually destroys the data , until it is Diffusion time step close to Gaussian noise .from Sampling corrupted data , whose diffusion process is defined as a Gaussian distribution:
[0059]
[0060] in: represents the diffusion step, and is the weight, which can be fixed or learned by the network. An important feature of forward noise processing is that any step can be directly obtained from the following equation Medium sampling:
[0061]
[0062] in, , as well as , As the weight, the diffusion step can be obtained as:
[0063]
[0064] when Very big time Approaching 0, Approaching Gaussian distribution .
[0065] (1.2) Reverse process
[0066] The reverse process is also a Markov chain, iteratively denoising the sampled Gaussian noise to a clean image. Starting from the potential data distribution To clean data The reverse process is defined as:
[0067]
[0068] Inverse denoising distribution for:
[0069]
[0070] in: is the mean, is the variance.
[0071] The image restoration task of the existing diffusion model (DDPM) is limited to the task of linearly degraded image restoration. The present invention designs a centralized, simple and effective sampling strategy to mine powerful prior information from the pre-trained diffusion model (DDPM), which can better generate high-quality images consistent with the damaged image content, and has very good generalization and is not limited to a certain dataset.
[0072] The present invention proposes a controlled generation method GDP of a diffusion model (DDPM) using damaged images as guidance. GDP is the first to use a pre-trained diffusion model (DDPM) to solve linear degradation, nonlinear degradation, blind image restoration, guidance of any damaged image, and a unified image restoration and enhancement framework for restoration of images of any size. In the controlled generation process of the diffusion model (DDPM), the variance does not participate in the offset calculation of the sampling mean at each step, which will obtain better generation results; in the controlled generation process of the diffusion model (DDPM), the image or other content used as guidance is applied to each step of the diffusion model (DDPM). The estimated Compared with directly Guidance is added to achieve better generation effect; a hierarchical guided generation method is proposed to estimate the degradation model in the case of low resolution, and the degradation model is used to restore the image after restoring its resolution through interpolation.
[0073] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.
[0074] In a specific embodiment, a GDP-based image restoration and enhancement method is used, the goal of which is to use a trained diffusion model as an effective prior for unified image restoration and enhancement, especially for processing a variety of damaged images.
[0075] Assume the damaged image pass Get, where is the original natural image, is a degradation model. By using The statistical value of Search for the best match in the space The best ,Will Considered as The corrupted observations or guided images. Due to the limited performance and application scope of the Generative Adversarial Network (GAN) in reverse processing of images, in this implementation, we will focus on studying a more general image prior and training a diffusion model on large-scale natural images for image synthesis.
[0076] The inverse denoising process of the diffusion model (DDPM) can be used to degrade the image Specifically, the inverse denoising distribution in the above formula is Conditional distribution can be used .
[0077]
[0078] in, and . and Can be considered a constant. Can be considered as Denoising is The probability of a consistent reconstructed image can be approximately calculated using the following formula:
[0079]
[0080] in: is the distance metric, is the normalization factor, is the scaling factor that controls the bootstrap size. Optional quality enhancement losses to increase GDP flexibility, Scaling factor to adjust image quality.
[0081] The gradients on both sides are calculated as:
[0082]
[0083]
[0084] Thus, we have a Gaussian conditional distribution It can be approximated by the unconditional distribution .
[0085] After calculation, the shift of the mean of the conditional distribution is:
[0086]
[0087] During the specific implementation, it was found that the way of adding guidance and whether variance Σ was used in sampling had an impact on the reconstructed image. Next, these two situations are analyzed.
[0088] (2.1) The impact of variance Σ on conditional generated images
[0089] In the previous conditional diffusion model, the variance Σ is used for the mean shift in the sampling process. Through experimental analysis, the variance Σ may have a negative impact on the quality of the generated image. Therefore, in the conditional denoising process of this embodiment, the variance is removed to improve the quality of the generated image.
[0090] (2.2) Add guidance
[0091] Figure 1 The process of unified image restoration using GDP is shown in Figure 2, including two variants, one of which is called , another one is called .
[0092] (2.2.1)
[0093] exist In the implementation mode of Add guidance, but without Σ when sampling. The pseudocode is as follows Figure 2 As shown, the specific steps include:
[0094] By adding noise and removing noise to the diffusion model containing T steps, the diffusion model is pre-trained and each step is stored. The corresponding noise;
[0095] Generate compliance Diffusion prior image of the distribution ;
[0096] Get the current step t=T;
[0097] S100, based on the current step The corresponding noise is calculated about The mean and variance Σ;
[0098] Set the boot image and based on the boot image and damaged images , calculate the denoising gradient loss , D is the degradation model, is the original image. Specifically:
[0099] The guide image is in the image Add guide, gradient The calculation formula is:
[0100]
[0101] In the above formula: is the distance metric, is the scaling factor that controls the boot size, For quality enhancement loss, is the scaling factor for adjusting image quality; It is optional;
[0102] Denoising Gradient Loss , generating a compliance Distribution , is the scaling factor that controls the boot size;
[0103] Take t-1 as the current step , return to step S100 to obtain a relatively damaged image Restoration and enhancement of original natural images .
[0104] However, this Variants that add guidance may still produce less than satisfactory image quality. Intuitively, using the commonly used conditional diffusion model and The generated image is still blurred in the background. To this end, in the specific implementation, try to The conditional signal on.
[0105] (2.2.2)
[0106] Specifically, during the sampling process, the pre-trained diffusion model (DDPM) is usually first estimated by The noise in the image Predict a clean image Then use the predicted and For the next step We can sample in this intermediate variable Add guidance to control the generation process of the diffusion model (DDPM). The pseudo code is as follows Figure 2 As shown, the specific steps include:
[0107] By adding noise and removing noise to the diffusion model containing T steps, the diffusion model is pre-trained and each step is stored. The corresponding noise;
[0108] Generate compliance Diffusion prior image of the distribution ;
[0109] Get the current step t=T;
[0110] S100, based on the current step The corresponding noise is calculated about The mean and variance Σ;
[0111] Set the boot image and based on the boot image and damaged images , calculate the denoising gradient loss :
[0112] By estimating The noise in the image Predict a clean image :
[0113]
[0114] in: , , is the weight; is the noise prediction model;
[0115] According to the task, determine the number of guide pictures to be added. If the image to be restored is an HDR image, the number of guide pictures is 3; otherwise, the number of guide pictures is 1;
[0116] Gradually Add guidance to get the final guidance image , and based on the guidance image and damaged images Compute denoising gradients ;
[0117]
[0118] in: Used to add guide steps;
[0119] Denoising Gradient Loss , generating a compliance Distribution , is the scaling factor that controls the boot size;
[0120] Take t-1 as the current step , return to step S100 to obtain a relatively damaged image Restoration and enhancement of original natural images .
[0121] In the implementation of the method of the present invention, some tasks can be classified into types with known degradation functions, such as: (3.1) degradation models for deblurring and super-resolution, (3.2) degradation models for image completion, and (3.3) degradation models for image colorization. These degradation models are specifically described as follows.
[0122] (3.1) Degradation Model for Deblurring and Super-Resolution
[0123] The degradation model for image deblurring and super-resolution can be expressed as:
[0124]
[0125] in: is the Gaussian kernel function or point spread function, represents convolution, is the original image, is the damaged image, the scale factor Perform downsampling operation .
[0126] The degradation model for image deblurring and super-resolution assumes that the low-resolution (LR) image is obtained by first concatenating the high-resolution (HR) image with a Gaussian kernel function or a point spread function. Convolution is performed to obtain a blurred image , then use Perform downsampling operation .
[0127] (3.2) Degradation model for image completion
[0128] The goal of image completion is to restore the missing pixels in the image. The corresponding degenerate transformation is to multiply the original image with the binary mask : ,in is the Hadamard product.
[0129] (3.3) Degradation model for image colorization
[0130] The purpose of image colorization is to convert grayscale images Restore to have In order to get the color image Obtained , degenerate transformation It is a reserved Grayscale shift of brightness.
[0131] In the real world, many images experience complex degradation, in which the degradation model or the parameters of the degradation model are always unknown. The method of the present invention is also applicable to image restoration with such unknown degradation model, which is called blind image restoration model.
[0132] (3.4) Blind Image Restoration Model
[0133] In blind image restoration models, both the image and the parameters of the degradation model need to be estimated simultaneously. For example, low-light image enhancement and HDR restoration can be regarded as tasks where the degradation model is unknown. Here, a simple and effective degradation model is designed to simulate complex degradation, which can be expressed as:
[0134]
[0135] Among them: Light factor and light mask is unknown, which is considered as blind image restoration. Specifically, in the reverse process, and are initialized randomly and optimized synchronously.
[0136] In the blind image restoration model, the losses are respectively and Calculate the gradient, through this gradient from and Subtract to optimize and The blind image restoration model can be used to guide the generation of a single image, such as dark light enhancement tasks, or it can be used to generate multiple images guided by multiple blind image restoration models to generate a single image, such as using three LDR images to restore an HDR image.
[0137] Using multiple images can be used to guide the generation of a single image, which is more challenging than guiding a single image. For example, for HDR image restoration, in the specific implementation, the HDR-GDP model is proposed for multi-image guided HDR image restoration. The HDR image consists of three input LDR images, namely short exposure, medium exposure, and long exposure. Similar to dark light enhancement, the degradation model is also considered , where the parameters determining HDR recovery remain unknown is a blind problem. Figure 1 As shown, in the inverse process, there are three image priors (i = 3) to guide the generation, so that three pairs of blind parameters of three LDR images are randomly initialized and optimized.
[0138] The pre-trained diffusion model can only generate images of fixed size, and the image sizes restored from different images are different. Therefore, the GDP model is further improved, including , , and HDR-GDP, can all use image patch-based methods such as to solve this problem. Resize the damaged image to , C is the number of color channels in the image, is the height of the image, is the width of the image. Applying the patch-based generation method to the obtained scaled image, the illumination factor and the scaled illumination mask can be obtained. . Afterwards, the light mask By interpolating and enlarging to the original image size , represents the global illumination mask. After that, the illumination factor and the illumination mask are fixed and used to generate image patches, which are reassembled into the output image. In one embodiment, the method is as follows Figure 4 As shown, the damaged image shown in the figure has three color channels, and its repair process includes the following steps:
[0139] (3.4.1) Scale the damaged original image size 600×400 to the set small size 384×256;
[0140] (3.4.2) Apply the patch-based generation method on the scaled image to obtain the illumination factor and the scaled illumination mask , the size is 384×256, the same as the scaled image size;
[0141] (3.4.3) Interpolate the illumination mask Light mask enlarged to the original damaged image size of 600×400 ;
[0142] (3.4.4) Based on illumination factor and illumination mask , an image block generation method based on an image block generates an image block of a fixed size, and each image block is a guiding image;
[0143] (3.4.5) Using each guide image, generate the corresponding restoration and enhancement images by using the method of generating diffusion priors;
[0144] (3.4.6) The restored and enhanced images are combined to obtain the restored and enhanced images of the original image.
[0145] In summary, if the size of the damaged image is different from the size of the image generated by the pre-trained diffusion model, the block-based strategy and the layer-based guidance strategy can be used to extend GDP to restore images of arbitrary resolution, thereby improving the versatility of GDP. Specifically, the following steps are included:
[0146] Scaling the size of the damaged original image to obtain a scaled image, thereby obtaining degradation model parameters of the scaled image;
[0147] Enlarging the degradation model parameters to the original image size by interpolation;
[0148] Based on the degradation model parameters, a fixed-size image block is generated by using an image block-based generation method, and each image block is a guide image, so that the corresponding restoration and enhancement images can be generated by using a method of generating diffusion priors;
[0149] For each restored and enhanced image block, the restored and enhanced image of the original image is obtained by combining the image blocks.
[0150] In the GDP of this embodiment, the loss function can be divided into two main parts: reconstruction loss and quality enhancement loss. The purpose of reconstruction loss is to restore the information contained in the conditional signal, while the purpose of quality enhancement loss is to improve the quality of the final output. The reconstruction loss is mainly MSEloss, which is the gradient loss above. or The quality enhancement loss includes the exposure control loss. , color consistency loss And the loss of lighting smoothness , respectively expressed as:
[0151]
[0152]
[0153]
[0154] in:
[0155] Exposure Control Loss Used to measure the distance between the local area and the area with good exposure level. is the total number of non-overlapping local regions, is the local area in the kth enhanced image, This is the area of good exposure level;
[0156] Loss of color consistency Correct the enhanced image and establish the relationship between the three adjusted channels, represents the average intensity value of the m channel in the enhanced image, (m, n) represents a pair of channels, is a channel set. Taking 3 color channels as an example, ;
[0157] Lighting smoothness loss For mapping each curve parameter , N represents the number of iterations, n is the iteration variable, , represents horizontal and vertical gradient operations, Is a channel set Elements. Taking the three color channels as an example, , Possible element values They are red, green and blue channels.
[0158] Through the description of the above implementation modes, those skilled in the art can clearly understand that the present disclosure can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits, etc. However, for the present disclosure, software program implementation is a better implementation mode in most cases.
[0159] In summary, the present invention deeply explores the powerful prior contained in the pre-trained diffusion model (DDPM) for unified image restoration, which has better restoration details and better diversity than the generative adversarial network (GAN) method; the present invention is not limited to linearly degraded image restoration, but can also be applied to nonlinearly degraded, blind, arbitrary-size and multiple damaged-image-guided image restoration; the present invention is not limited to a specific data set, but can be generalized to any data set; the present invention can restore image restoration with multiple damaged forms superimposed, which is closer to real-world image restoration.
[0160] Although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields, and the above specific embodiments are only illustrative and instructive, rather than restrictive. A person of ordinary skill in the art can also make many forms under the guidance of this specification and without departing from the scope of protection of the claims of the present invention, all of which belong to the protection of the present invention.
Claims
1. A unified image restoration and enhancement method for generating diffusion priors, It is characterized in that The method comprises the following steps: The diffusion model is pre-trained by adding noise and removing noise from the diffusion model containing T steps in total; Generate compliance Diffusion prior image of the distribution ; Get the current step t=T; S100, based on the current step Corresponding noise, calculate the diffusion prior image The mean and variance Σ; Set the boot image and based on the boot image and damaged images , calculate the denoising gradient loss ,D is the degradation model, is the original image; Denoising Gradient Loss , generating a compliance Distribution , is the scaling factor that controls the boot size; Take t-1 as the current step , return to step S100 to obtain a relatively damaged image Restoration and enhancement of original natural images ; Among them, the degradation model Types include: degradation models for deblurring and super-resolution, degradation models for image completion, degradation models for image colorization, and blind image restoration models; The degradation model for deblurring and super-resolution is as follows: in: is the Gaussian kernel function or point spread function, represents convolution, is the original image, For the damaged image, the scale factor Perform downsampling operation ; The degradation model of the image completion is as follows: in: is a binary mask; is the Hadamard product; The image colorization degradation model is to transform the color image Keep as grayscale image Grayscale change, H is the height of the image, W is the width of the image; The blind image restoration model is as follows: in: is the illumination factor, Light mask, and Unknown, in the reverse process of the diffusion model, and are randomly initialized and optimized synchronously, is the image to be degraded, This is the image obtained after degradation processing.
2. The method according to claim 1, It is characterized in that Based on the guided image and damaged images , calculate the denoising gradient loss ,include: The guide image is in the image Add guide, gradient The calculation formula is: In the above formula: is the distance metric, is the scaling factor that controls the boot size, For quality enhancement loss, is the scaling factor for adjusting image quality; It is optional.
3. The method according to claim 1, It is characterized in that Based on the guided image and the damaged image , denoising gradient loss ,include: By estimating The noise in the image Predict a clean image ; Determine the number of guide pictures to add based on the task, and gradually Add guidance to get the final guidance image , and based on the guidance image and damaged images Compute denoising gradients ; in: Used to add guide steps.
4. The method according to claim 3, It is characterized in that By estimating The noise in the image Predict a clean image : in: , , is the weight; It is a noise prediction model.
5. The method according to claim 3, It is characterized in that Determine the number of guide pictures to add based on the task, including: If the restored image is an HDR image, the number of pictures is 3; otherwise, the number of guide pictures is 1.
6. The method according to claim 1, It is characterized in that Use the blind image restoration model to generate multiple guide images, and use multiple guide images to guide the generation of original natural images .
7. The method according to claim 1, It is characterized in that If the size of the damaged image is different from the size of the image generated by the pre-trained diffusion model, the method further comprises the following steps: Scaling the size of the damaged original image to obtain a scaled image, thereby obtaining degradation model parameters of the scaled image; Enlarging the degradation model parameters to the original image size by interpolation; Based on the degradation model parameters, a fixed-size image block is generated by using an image block-based generation method, and each image block is a guide image, so that the corresponding restoration and enhancement images can be generated by using a method of generating diffusion priors; For each restored and enhanced image block, the restored and enhanced image of the original image is obtained by combining the image blocks.
8. The method according to claim 2, It is characterized in that The quality enhancement loss Q consists of exposure control loss, color consistency loss, and lighting smoothness loss.
9. A computer-readable storage medium, Features: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 8.