An image restoration method and device, electronic equipment and storage medium

Abstract

This disclosure relates to an image restoration method, apparatus, electronic device, and storage medium, belonging to the field of image processing technology. It involves sampling a Gaussian random noise map based on a pre-trained diffusion model, obtaining estimated features corresponding to the current diffusion step size, decoding these features, and obtaining a restored image. The restored image and the estimated distribution corresponding to the estimated features are input into a preset physical coherence degradation model to obtain a degraded observation image. Based on the restored image, the degraded observation image, and the image to be processed as conditional input, image loss values ​​are calculated to adjust the network parameters of the preset physical coherence degradation model. An image quality score is obtained by evaluating the restored image based on a preset image evaluation algorithm. The restored image is then used as the target image and output when the difference between the image quality score of the current diffusion step size and the image quality score of the previous diffusion step size is less than a preset score difference threshold, thus achieving effective restoration of zero-shot images.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent positioning technology, and in particular to an image restoration method, apparatus, electronic device and storage medium. Background Technology

[0002] Image restoration is typically a fundamental visual task, recovering clean content from degraded observations. In the real world, images are frequently compromised during acquisition or transmission by noise, blur, haze, lighting, or artifacts, which severely degrade perceptual quality and the performance of downstream tasks. Therefore, effectively restoring images under diverse and unpredictable degradation remains an essential yet challenging problem.

[0003] Related technologies rely on deeper networks or pre-trained features to implicitly capture degradation features. This implicit modeling limits the ability to represent heterogeneous degradation and introduces a large amount of training cost as the types of tasks increase, resulting in poor efficiency and effectiveness of image restoration. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides an image restoration method, apparatus, electronic device and storage medium.

[0005] This disclosure provides an image restoration method, comprising: acquiring a Gaussian random noise map, sampling the Gaussian random noise map according to a preset diffusion step size based on a pre-trained diffusion model, acquiring estimated features corresponding to the current diffusion step size, and decoding the estimated features to obtain a restored image corresponding to the current diffusion step size; inputting the restored image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features into a preset physical coherence degradation model for processing to obtain a degraded observation image; calculating an image loss value based on the restored image corresponding to the current diffusion step size, the degraded observation image, and an image to be processed as a conditional input, and adjusting the network parameters of the preset physical coherence degradation model based on the total image loss value; evaluating the restored image corresponding to the current diffusion step size based on a preset image evaluation algorithm to obtain an image quality score of the restored image corresponding to the current diffusion step size, until the image quality score is less than or equal to a preset score difference threshold, and outputting the restored image corresponding to the current diffusion step size as the target image.

[0006] This disclosure also provides an image restoration apparatus, comprising: a sampling module for acquiring a Gaussian random noise map and sampling the Gaussian random noise map according to a preset diffusion step size based on a pre-trained diffusion model to obtain estimated features corresponding to the current diffusion step size; a decoding module for decoding the estimated features to obtain a restored image corresponding to the current diffusion step size; a processing module for inputting the restored image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features into a preset physical coherence degradation model for processing to obtain a degraded observation image; an adjustment module for calculating an image loss value based on the restored image corresponding to the current diffusion step size, the degraded observation image, and an image to be processed as a conditional input, and adjusting the network parameters of the preset physical coherence degradation model based on the total image loss value; and an evaluation output module for evaluating the restored image corresponding to the current diffusion step size based on a preset image evaluation algorithm to obtain an image quality score of the restored image corresponding to the current diffusion step size, until the image quality score is less than or equal to a preset score difference threshold, and then outputting the restored image corresponding to the current diffusion step size as the target image.

[0007] This disclosure also provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the image restoration method provided in this disclosure.

[0008] This disclosure also provides a computer-readable storage medium storing a computer program for performing the image restoration method provided in this disclosure.

[0009] This disclosure also provides a computer program product, including a computer program, wherein the computer program is executed by a processor as an image restoration method provided in the embodiments of this application.

[0010] Compared with the prior art, the technical solution provided in this disclosure has the following advantages: The image restoration scheme provided in this disclosure acquires a Gaussian random noise map, samples the Gaussian random noise map according to a preset diffusion step size based on a pre-trained diffusion model, obtains the estimated features corresponding to the current diffusion step size, and decodes the estimated features to obtain the restored image corresponding to the current diffusion step size; inputs the restored image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features into a preset physical coherence degradation model for processing to obtain a degraded observation image; calculates the image loss value based on the restored image corresponding to the current diffusion step size, the degraded observation image, and the image to be processed as a conditional input, and adjusts the network parameters of the preset physical coherence degradation model based on the total image loss value; evaluates the restored image corresponding to the current diffusion step size based on a preset image evaluation algorithm to obtain the image quality score of the restored image corresponding to the current diffusion step size, until the obtained image quality score is less than or equal to a preset score difference threshold, and then outputs the restored image corresponding to the current diffusion step size as the target image. Therefore, heterogeneous degradation is modeled as a uniform integrated distribution, which can be directly optimized in the latent space, thereby enabling principled solution exploration and effective timely adaptation. In addition, a dynamic quality improvement strategy is introduced, which can adaptively adjust the diffusion trajectory to achieve robust global optimal convergence and effective recovery of zero-shot images. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0012] Figure 1 A schematic flowchart of an image restoration method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of the image restoration framework provided in the embodiments of this disclosure; Figure 3 This is a schematic diagram of another image restoration device provided in an embodiment of the present disclosure. Detailed Implementation

[0013] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0014] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0015] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] In related technologies, task-specific image restoration methods have largely addressed individual degradation problems, including denoising, deblurring, dehazing, and even super-resolution, through task-tailored, often supervised architectures and objectives. Representative advances include classical prior and residual convolutional neural networks for denoising, multi-scale and adversarial formulations explicitly capturing motion blur dynamics, and prior / physical guidance constraints with frequency-aware cues (e.g., dark channels) for dehazing to reduce color shifts and artifacts. Despite their strong intra-domain performance, these pipelines assume fixed forward models and paired supervision, exhibit poor transfer between tasks, and remain vulnerable to mixed or entangled degradations. Additional task-specific adjustments further limit the scalability of unified multi-task restoration.

[0020] Furthermore, holistic image restoration approaches, to improve generalization across degradation types, construct a unified network that shares parameters and conditions on degradation cues. A typical design learns degradation embeddings from low-quality input, attaches cues or adapters to guide features, and uses a lightweight controller to route paths. Some methods identify degradation at test time and select sub-modules or statistics to match prediction conditions. Complementary lines employ zero-shot restoration with pre-trained diffusion priors, adapting inference without task-specific retraining by adjusting sampling plans, guiding signals, or latent condition estimates. However, degradation is often implicitly modeled, control signals are not physically grounded, and inference may require extensive sampling, limiting controllability and convergence.

[0021] Furthermore, physics-based degradation models provide explicit physical modeling of the image formation process to guide restoration, including noise statistics, atmospheric scattering of haze, and illuminance-reflectance decomposition based on image processing theory. Representative methods instantiate these priors in learning systems, such as extended scattering models with data-efficient training for structure-aware formulas for dehazing, and physics-based noise modeling for multispectral filter arrays; broader investigations integrate these priors across tasks. Analytical constraints improve interpretability and stability. However, the dependence on task-specific forward models and precise parameters makes these methods sensitive to model mismatches and difficult to generalize to heterogeneous or hybrid degradation, thus unidirectional image restoration under invisible conditions remains challenging.

[0022] This disclosure provides a flexible approach to handling various degradations without requiring task-specific training. However, existing methods typically rely on stacked layers or pre-trained features to enhance degradation representations while neglecting physically consistent priors; insufficient degradation cues in zero-shot diffusion introduce heavy training burdens and high sampling costs; furthermore, fixed inference trajectories often collapse into suboptimal solutions under complex erosion. Specifically, heterogeneous degradation can be reparameterized into a minimal set of physically coherent parameters for compact representation. The image restoration method of this disclosure models heterogeneous degradation as a uniform, unified distribution that can be directly optimized in the latent space, thereby enabling principled solution exploration and effective timely adaptation; furthermore, a dynamic quality improvement strategy is introduced that adaptively adjusts the diffusion trajectory to achieve robust global optimal convergence, enabling efficient restoration of zero-shot images.

[0023] Specifically, this disclosure shifts from implicit feature learning to explicit degradation distribution modeling. Instead of treating heterogeneous degradation as isolated or unknown disturbances, it uses a compact set of interpretable physical parameters to represent various disruptions from the real world. The transformed distribution establishes a unified solution space for integrated recovery. Therefore, this disclosure can directly optimize degradation behavior in the potential diffusion space by forcing distribution alignment, thereby achieving controlled and degradation-aware posterior sampling, rather than unguided diffusion. Furthermore, a dynamic quality improvement strategy evaluates recovery quality during inference and adaptively adjusts the sampling trajectory to escape local minima and move towards the global optimum. By combining explicit degradation distribution modeling with adaptive posterior improvement, physically interpretable null-shoot recovery is achieved, extending to unseen and mixed degradation, and consistently outperforming state-of-the-art methods in terms of quantitative metrics and perceptual fidelity.

[0024] In summary, the embodiments of this disclosure propose a unified physical parameterization for heterogeneous degradation, which enables explicit control over the potential spatial distribution of zero-shot recovery; the dynamic quality improvement strategy adaptively adjusts the diffusion trajectory in a self-evaluative manner to ensure global optimality with minimal inference cost; and the embodiments of this disclosure consistently outperform existing methods in both single and hybrid degradation of quantitative and sensing performance.

[0025] Specifically, Figure 1 This is a flowchart illustrating an image restoration method provided in an embodiment of this disclosure. The method can be implemented using software and / or hardware, and is generally integrated into an electronic device. Figure 1 As shown, the method includes: Step 101: Obtain the Gaussian random noise map, and sample the Gaussian random noise map according to the preset diffusion step size based on the pre-trained diffusion model, obtain the estimated features corresponding to the current diffusion step size, and decode the estimated features to obtain the restored image corresponding to the current diffusion step size.

[0026] In this embodiment, a pre-trained LDM (Latent Diffusion Model) is used as the pre-trained diffusion model to sample the Gaussian random noise map, and then denoising is performed stepwise in the latent space to obtain a high-quality output: (1) (2) (3) in, , , , The clean latent representation represents the magnitude of the noise introduced at the i-th time step. From The estimated characteristics; This represents the prediction noise, which is the only uncertain variable in the inverse process.

[0027] Furthermore, the estimated features are decoded to obtain the reconstructed image corresponding to the current diffusion step size; that is, the decoder is used to convert the predicted output into a new image. Decode to the pixel domain to obtain the corresponding restored image. .

[0028] For example, such as Figure 2 As shown, the Gaussian random noise map is sampled according to a preset diffusion step size to obtain the estimated features corresponding to each diffusion step size. to For example, processing the current diffusion step size, for example Figure 2 shown To obtain the estimated features Decode the image to obtain the recovered image. .

[0029] Step 102: Input the restored image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features into the preset physical coherence degradation model for processing to obtain the degraded observation image.

[0030] Understandably, heterogeneous degradation allows for minimal physical coherence parameterization in the potential diffusion space, providing a fundamental physical basis for modeling complex degradation processes. Specifically, for example... Figure 2 As shown, at each sampling step t, the predicted output is first obtained using the LDM decoder. Decode to the pixel domain to obtain the corresponding image. Then, the uniform degradation distribution GGD (Generalized Gaussian distribution) was applied. of That is, the estimated distribution corresponding to the estimated feature, and the distribution parameters. By using linear layer embedding, the degradation process can be represented as a structured probability distribution, thereby producing physically plausible images of degradation observations. : (4) Where f1, f2, and f3 represent convolutional layers, which can be minimized by the mixture loss function. To optimize the physical coherence degradation model.

[0031] (5) Where Lmse, Lpse, and Ladv represent mean squared error loss, perceptual loss, and adversarial loss, respectively.

[0032] Therefore, the physical coherence degradation model is useful for image recovery. Corresponding to its degradation An equivalent distribution mapping is established between them, thereby reformulating the degradation modeling into a distribution-level approximation problem.

[0033] In this embodiment of the disclosure, the restored image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features are input into a preset Physically Coherent Degradation Modeling (PCDM) for processing to obtain a degraded observation image. The process includes: processing the restored image based on a preset first processing module to obtain a first feature; processing the first feature based on a preset second processing module to obtain a second feature; processing the estimated features based on a preset third processing module to obtain a third feature; multiplying the first feature and the third feature element-wise and adding them to the second feature to obtain a fourth feature; and processing the fourth feature based on a preset fourth processing module to obtain a degraded observation image. The preset first processing module, preset second processing module, preset third processing module, and preset fourth processing module are each composed of a convolutional network, an activation function, and another convolutional network, respectively.

[0034] Specifically, such as Figure 2 As shown, restore the image The estimated distribution corresponding to the estimated features The images are input into the corresponding processing modules for processing and the degraded observation images are output. Each processing module consists of a convolutional network (Conv), an activation function (ReLU), and another convolutional network (Conv).

[0035] Understandably, the physical coherence representation first uses the generalized Gaussian distribution (GGD) to approximate the physical coherence. and It includes common distributions and adapts to the statistics of both natural and degraded images: (6) (7) (8) in, and Indicates scale and shape parameters.

[0036] Therefore, the GGD model is used to model the latent representation of degraded images, and parameters are estimated on a pre-defined dataset, such as the Kodak24 dataset, under the same experimental settings. and Therefore, a physically coherent representation can be obtained, which connects heterogeneous degenerate distributions and supports dynamic assimilation modeling.

[0037] Step 103: Calculate the image loss value based on the recovered image, the degraded observation image, and the image to be processed as the conditional input corresponding to the current diffusion step size, and adjust the network parameters of the preset physical coherence degradation model based on the total image loss value.

[0038] In this embodiment of the disclosure, the total image loss value is calculated based on the restored image, the degraded observation image, and the image to be processed as conditional input corresponding to the current diffusion step size. This includes: calculating the degradation alignment loss value of the degraded observation image and the image to be processed; calculating the mean square error loss value of the degraded observation image and the image to be processed; calculating the perceptual loss value of the degraded observation image and the image to be processed; calculating the adversarial loss value between the restored image, the degraded observation image, and the image to be processed; obtaining the luminance quality value and chrominance quality value of the restored image; and weighting and summing the degradation alignment loss value, mean square error loss value, perceptual loss value, adversarial loss value, luminance quality value, and chrominance quality value according to multiple preset weights to obtain the total image loss value.

[0039] In this embodiment of the disclosure, calculating the degradation alignment loss value of the degraded observation image and the image to be processed includes: encoding the image to be processed and the degraded observation image respectively to obtain the features to be processed and the degraded observation features, and processing the features to be processed and the degraded observation features based on a preset generalized Gaussian distribution to obtain the distribution to be processed and the degraded distribution; and calculating the distribution to be processed and the degraded distribution based on a preset degradation alignment loss formula to obtain the degradation alignment loss value.

[0040] In this embodiment of the disclosure, obtaining the luminance quality value and chrominance quality value of the restored image includes: obtaining the average luminance of the restored image in each chrominance channel, and calculating the luminance quality value based on the average luminance and a preset exposure standard value; and obtaining the chrominance quality value of the restored image as the chrominance quality value.

[0041] Specifically, in order to adaptively handle various degradations, the input image is... Incorporating the posterior sampling direction, the process is restated as follows: (9) Among them, the correction item , To fix the error, the drift Compared with observed values Coupled, and adjust the denoising direction according to the current degradation. Representing a given latent representation Generate degraded observations The probability can be used Approximately: (10) in, Let J represent the transformation from a high-quality image to a low-quality image, Q represent the image distance metric, and (Z, λ1, λ2) be the scaling and weighting factors controlling the guidance strength. The likelihood gradient provides degradation-aware, interpretable updates, improving robustness to mixed degradation. As mentioned above, conditional posterior sampling... This can be interpreted as estimating the probability of degradation. This depends on accurate modeling of the underlying degradation mechanism.

[0042] During posterior sampling, through image difference terms A novel degenerate alignment loss (DegLoss) was developed. It uses a uniform degenerate distribution as a well-defined and tractable optimization objective.

[0043] Specifically, considering PCDM and input Degraded observation images ,like Figure 2 As shown, an LDM encoder is first used to generate their latent representations. and Secondly, their parameter degradation distribution is modeled as and The degenerate alignment loss is defined as the Kullback-Leibler divergence (KLD) between the two distributions: (11) Furthermore, to improve reconstruction fidelity, the degradation alignment loss and mean square error J are compared. mse Perceived loss J pse and combat losses J adv Combined, it can be summarized as follows: (12) Among them, λ1, λ2, λ3, and λ4 control the contribution of each term. Furthermore, Brightness and chromaticity quality are defined as image quality terms. for: (13) in, This represents the average brightness of channel i. This is a natural exposure standard, where D represents a pixel pair in the chroma channel, δ(p, q) measures their color difference, and λ5 and λ6 are weighting factors. Through these designs, posterior sampling is effectively redirected to physically consistent zero-shot reconstruction, thereby enabling degradation perception and principled solution exploration.

[0044] Step 104: Evaluate the restored image corresponding to the current diffusion step size based on the preset image evaluation algorithm to obtain the image quality score of the restored image corresponding to the current diffusion step size. When the difference between the image quality score and the reference quality score is less than the preset score difference threshold, the restored image corresponding to the current diffusion step size is taken as the target image and output; wherein, the reference quality score is the restored image quality score corresponding to the previous diffusion step size relative to the current diffusion step size.

[0045] In some embodiments, if the score difference is greater than a preset score difference threshold and the current diffusion step size is greater than 0, then sampling is performed according to the preset diffusion step size to obtain the feature corresponding to the next diffusion step size as the estimated feature.

[0046] In some embodiments, if the score difference is greater than a preset score difference threshold, the current diffusion step size is equal to 0, and the Gaussian random noise map is reacquired.

[0047] The reference quality score is the quality score of the restored image corresponding to the previous diffusion step size relative to the current diffusion step size. When the current diffusion step size is the first expansion step size, the reference quality score is 0.

[0048] To mitigate suboptimal convergence under complex degradation during static diffusion inference, a no-reference image quality assessment model was integrated. This model was pre-trained on various degradations from the real world to dynamically adjust the posterior sampling strategy.

[0049] Specifically, according to Figure 2 The Dynamic Quality-Refinement Strategy (DQR) shown is used as the preset image evaluation algorithm, according to... Figure 2 The evaluation paradigm shown applies a pre-defined image evaluation algorithm, such as a no-reference image quality evaluation model, to calculate the reconstructed image at the current diffusion step size t at each step j (executed once every Δt = 100 steps). Image quality score Then compare it with the score from the previous iteration. The basis for forming an adaptive decision-making strategy through comparison is: (14) Here, η is a predefined threshold used to control the refinement sensitivity. If the score difference is below the threshold, the recovery quality is considered satisfactory, the inference terminates, and... The optimal output is considered; otherwise, further refinement is adaptively performed: for t>0, inference continues to follow standard diffusion updates; for t=0, noise is introduced into the current latent representation z0, and posterior sampling is restarted as follows: (15) in, Represents noise scale, noise Independent sampling from a standard Gaussian distribution.

[0050] Therefore, as Figure 2 As shown, this disclosure is a unified physical zero-shot recovery framework consisting of four main parts: (1) a degradation-aware posterior sampling scheme that operates in the latent diffusion space and conditionally adjusts the sampler according to degradation cues; (2) a physical coherent degradation modeling (PCDM) that maps heterogeneous degradation to a unified low-dimensional distribution that provides conditional signals; (3) a distribution alignment target that aligns the estimated latent distribution with physical parameters to enable controllable, degradation-aware inference; and (4) a dynamic quality improvement (DQR) strategy that adaptively adjusts the sampling trajectory using self-evaluation quality to promote global optimal convergence.

[0051] In other words, by unifying heterogeneous degradation into a compact distribution, this disclosure transforms diffusion into a controllable search through physically coherent degradation modeling, distribution alignment loss, and dynamic quality improvement strategies, producing physically consistent solutions between single and hybrid degradations. Experiments show significant improvements in accuracy and perception by reducing the uncertainty of fixed-trajectory diffusion, and ablation studies confirm complementary advantages in representation strength, principle guidance, and convergence robustness. For the first time, a physically interpretable image restoration paradigm is explored through explicit distribution-level controllability, which is practical for zero-shot deployments in the real world. As a flexible pipeline, the diffusion model can be tuned with parameter settings to robustly handle unseen real-world variability.

[0052] In summary, the image restoration scheme provided in this embodiment acquires a Gaussian random noise map, samples the Gaussian random noise map according to a preset diffusion step size based on a pre-trained diffusion model, obtains the estimated features corresponding to the current diffusion step size, and decodes the estimated features to obtain the restored image corresponding to the current diffusion step size. The restored image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features are input into a preset physical coherence degradation model for processing to obtain a degraded observation image. An image loss value is calculated based on the restored image corresponding to the current diffusion step size, the degraded observation image, and the image to be processed as a conditional input. The network parameters of the preset physical coherence degradation model are adjusted based on the total image loss value. The restored image corresponding to the current diffusion step size is evaluated based on a preset image evaluation algorithm to obtain an image quality score for the restored image corresponding to the current diffusion step size. The process continues until the image quality score is less than or equal to a preset score difference threshold, at which point the restored image corresponding to the current diffusion step size is used as the target image and output. Therefore, heterogeneous degradation is modeled as a uniform integrated distribution, which can be directly optimized in the latent space, thereby enabling principled solution exploration and effective timely adaptation. In addition, a dynamic quality improvement strategy is introduced, which can adaptively adjust the diffusion trajectory to achieve robust global optimal convergence and effective recovery of zero-shot images.

[0053] Figure 3 This is a schematic diagram of an image restoration device provided in an embodiment of this disclosure. The system can be implemented by software and / or hardware, and is generally integrated into an electronic device. Figure 3 As shown, the device includes: The sampling module 301 is used to acquire a Gaussian random noise map and sample the Gaussian random noise map according to a preset diffusion step size based on a pre-trained diffusion model to obtain the estimated features corresponding to the current diffusion step size. Decoding module 302 is used to decode the estimated features to obtain the restored image corresponding to the current diffusion step size; Processing module 303 is used to input the recovered image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features into a preset physical coherence degradation model for processing to obtain a degraded observation image; The adjustment module 304 is used to calculate the image loss value based on the recovered image corresponding to the current diffusion step size, the degraded observation image, and the image to be processed as a conditional input, and to adjust the network parameters of the preset physical coherence degradation model based on the total image loss value. The evaluation output module 305 is used to evaluate the restored image corresponding to the current diffusion step size based on a preset image evaluation algorithm, and obtain the image quality score of the restored image corresponding to the current diffusion step size. When the difference between the image quality score and the reference quality score is less than a preset score difference threshold, the restored image corresponding to the current diffusion step size is used as the target image and output; wherein, the reference quality score is the restored image quality score corresponding to the previous diffusion step size relative to the current diffusion step size.

[0054] Optionally, the processing module 303 is specifically used for: processing the restored image based on a preset first processing module to obtain a first feature; processing the first feature based on a preset second processing module to obtain a second feature; processing the estimated feature based on a preset third processing module to obtain a third feature; multiplying the first feature and the third feature element-wise and adding them to the second feature to obtain a fourth feature, and processing the fourth feature based on a preset fourth processing module to obtain the degraded observation image; wherein the preset first processing module, the preset second processing module, the preset third processing module, and the preset fourth processing module are respectively composed of a convolutional network, an activation function, and another convolutional network.

[0055] Optionally, the adjustment module 304 includes: The first computing unit is used to calculate the degradation alignment loss value of the degraded observation image and the image to be processed; The second calculation unit is used to calculate the mean square error loss value of the degraded observation image and the image to be processed; The third calculation unit is used to calculate the perceptual loss value of the degraded observation image and the image to be processed; The fourth calculation unit is used to calculate the adversarial loss value among the restored image, the degraded observation image, and the image to be processed; The acquisition unit is used to acquire the luminance quality value and chrominance quality value of the restored image; The weighted summation unit is used to perform weighted summation on the degradation alignment loss value, the mean square error loss value, the perceptual loss value, the adversarial loss value, the luminance quality value and the chrominance quality value according to multiple preset weights to obtain the total image loss value; The adjustment unit is used to adjust the network parameters of the preset physical coherence degradation model based on the total image loss value.

[0056] Optionally, the first computing unit is specifically used to: encode the image to be processed and the degraded observation image respectively to obtain features to be processed and degraded observation features, and process the features to be processed and the degraded observation features based on a preset generalized Gaussian distribution to obtain a distribution to be processed and a degraded distribution; and calculate the distribution to be processed and the degraded distribution based on a preset degraded alignment loss formula to obtain the degraded alignment loss value.

[0057] Optionally, the acquisition unit is specifically used to: acquire the average brightness of the restored image in each chroma channel, and calculate the brightness quality value based on the average brightness and a preset exposure standard value; and acquire the color difference value of the restored image as the chroma quality value.

[0058] Optionally, the device further includes: a first acquisition module, configured to, when the score difference is greater than or equal to the preset score difference threshold and the current diffusion step size is greater than 0, sample according to the preset diffusion step size to obtain the feature corresponding to the next diffusion step size as the estimated feature.

[0059] Optionally, the device further includes: a second acquisition module, configured to, when the score difference is greater than the preset score difference threshold, set the current diffusion step size to 0 and reacquire the Gaussian random noise map.

[0060] The image restoration apparatus provided in this disclosure can execute the image restoration method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0061] This disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the image restoration method provided in any embodiment of this disclosure.

[0062] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement any of the image restoration methods provided in this disclosure.

[0063] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium storing a computer program for performing any of the image restoration methods provided in the present disclosure.

[0064] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0065] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0066] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image restoration method, characterized in that, The method includes: A Gaussian random noise map is obtained, and the Gaussian random noise map is sampled according to a preset diffusion step size based on a pre-trained diffusion model. The estimated features corresponding to the current diffusion step size are obtained, and the estimated features are decoded to obtain the restored image corresponding to the current diffusion step size. The recovered image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features are input into a preset physical coherence degradation model for processing to obtain a degraded observation image; The image loss value is calculated based on the recovered image corresponding to the current diffusion step size, the degraded observation image, and the image to be processed as a conditional input. The network parameters of the preset physical coherence degradation model are then adjusted based on the total image loss value. The restored image corresponding to the current diffusion step size is evaluated based on a preset image evaluation algorithm to obtain an image quality score of the restored image corresponding to the current diffusion step size. When the difference between the image quality score and the reference quality score is less than a preset score difference threshold, the restored image corresponding to the current diffusion step size is used as the target image and output. The reference quality score is the restored image quality score corresponding to the previous diffusion step size relative to the current diffusion step size.

2. The method according to claim 1, characterized in that, The step of inputting the recovered image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features into a preset physical coherence degradation model for processing to obtain a degraded observation image includes: The restored image is processed by a preset first processing module to obtain a first feature; The first feature is processed by a preset second processing module to obtain a second feature; The estimated features are processed by a preset third processing module to obtain a third feature; The first feature and the third feature are multiplied element-wise and then added to the second feature to obtain the fourth feature. The fourth feature is then processed based on the preset fourth processing module to obtain the degraded observation image. The preset first processing module, the preset second processing module, the preset third processing module, and the preset fourth processing module are each composed of a convolutional network, an activation function, and another convolutional network, respectively.

3. The method according to claim 1, characterized in that, The calculation of the total image loss value based on the recovered image corresponding to the current diffusion step size, the degraded observation image, and the image to be processed as conditional input includes: Calculate the degradation alignment loss values ​​for the degraded observed image and the image to be processed; Calculate the mean square error loss values ​​for the degraded observed image and the image to be processed; Calculate the perceptual loss values ​​for the degraded observed image and the image to be processed; Calculate the adversarial loss value among the restored image, the degraded observed image, and the image to be processed; Obtain the luminance quality value and chrominance quality value of the restored image; The degradation alignment loss value, the mean square error loss value, the perceptual loss value, the adversarial loss value, the luminance quality value, and the chrominance quality value are weighted and summed according to multiple preset weights to obtain the total image loss value.

4. The method according to claim 3, characterized in that, The calculation of the degradation alignment loss values ​​for the degraded observed image and the image to be processed includes: The image to be processed and the degraded observation image are encoded respectively to obtain the features to be processed and the degraded observation features. The features to be processed and the degraded observation features are then processed based on a preset generalized Gaussian distribution to obtain the distribution to be processed and the degraded distribution. The degradation alignment loss value is obtained by calculating the distribution to be processed and the degradation distribution based on the preset degradation alignment loss formula.

5. The method according to claim 3, characterized in that, The step of obtaining the luminance quality value and chrominance quality value of the restored image includes: The average brightness of the restored image in each chroma channel is obtained, and the brightness quality value is calculated based on the average brightness and a preset exposure standard value. The color difference value of the restored image is obtained as the color quality value.

6. The method according to claim 1, characterized in that, The method further includes: If the score difference is greater than or equal to the preset score difference threshold, and the current diffusion step size is greater than 0, then sampling is performed according to the preset diffusion step size to obtain the feature corresponding to the next diffusion step size as the estimated feature.

7. The method according to claim 1, characterized in that, The method further includes: If the score difference is greater than the preset score difference threshold, the current diffusion step size is equal to 0, and the Gaussian random noise map is reacquired.

8. An image restoration device, characterized in that, include: The sampling module is used to acquire a Gaussian random noise map and sample the Gaussian random noise map according to a preset diffusion step size based on a pre-trained diffusion model to obtain the estimated features corresponding to the current diffusion step size. The decoding module is used to decode the estimated features to obtain the restored image corresponding to the current diffusion step size; The processing module is used to input the recovered image corresponding to the current diffusion step size and the estimated distribution corresponding to the estimated features into a preset physical coherence degradation model for processing, so as to obtain a degraded observation image; The adjustment module is used to calculate the image loss value based on the recovered image corresponding to the current diffusion step size, the degraded observation image, and the image to be processed as a conditional input, and to adjust the network parameters of the preset physical coherence degradation model based on the total image loss value. The evaluation output module is used to evaluate the restored image corresponding to the current diffusion step size based on a preset image evaluation algorithm, and obtain the image quality score of the restored image corresponding to the current diffusion step size. When the difference between the image quality score and the reference quality score is less than a preset score difference threshold, the restored image corresponding to the current diffusion step size is used as the target image and output; wherein, the reference quality score is the restored image quality score corresponding to the previous diffusion step size relative to the current diffusion step size.

9. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the image restoration method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for performing the image restoration method according to any one of claims 1-7.

Images