A multi-degradation image inpainting method based on adaptive hints

Abstract

The application provides a multi-degradation image restoration method based on adaptive hints. The method contains a two-stage latent space diffusion model, which can flexibly handle various degraded images. Specifically, the model learns the difference between degraded images and clean images, generates degradation hints, and encodes the hint information in the latent space. The first stage obtains the degradation hints as key features through the degradation adaptive hint learning mechanism, which is used to guide the diffusion model in the second stage to perform image restoration. Finally, the hint information is integrated into a Fourier transform restoration network, which can accurately restore different types of degraded images. This method not only improves the accuracy of image restoration, but also significantly reduces the computational complexity, and is suitable for image restoration tasks in various real-world scenarios, including denoising, deblurring, defogging, deraining, desnowing, droplet removal, and low-light enhancement.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to an image restoration method. Specifically, the present application focuses on multi-degradation image restoration technology based on adaptive prompting, aiming to improve the image restoration effect under various degradation conditions. BACKGROUND

[0002] In the field of image restoration, various methods have been proposed in recent years to handle complex restoration tasks, each with its own advantages and disadvantages. The following are several main image restoration methods and their characteristics:

[0003] 1. Image restoration algorithm based on convolutional neural network (CNN)

[0004] CNN method learns local features of images for image restoration, and has achieved significant results in various image restoration tasks such as denoising, deblurring, and deraining. For example, MPRNet restores images progressively through multiple stages, each stage incorporating intermediate information to obtain rich contextual features. MAXIM uses a hierarchical structure based on UNet and a spatial gating multi-layer perceptron to support long-distance interaction, achieving better image restoration results. However, these CNN-based methods mainly rely on local receptive fields, which may cause artifacts when dealing with severely degraded images.

[0005] 2. Image restoration algorithm based on generative adversarial network (GAN)

[0006] Generative adversarial networks generate high-quality images through the adversarial training of generators and discriminators, and have been widely applied to image restoration tasks. For example, HRGAN

[31] uses generative adversarial networks for raindrop removal, and can generate raindrop-free images with realistic details. Although GAN performs well in generating image details and realism, its training process is complex, and may suffer from mode collapse, and requires a large amount of computational resources, affecting the restoration effect.

[0007] 3. Image decoupling algorithm based on self-supervised learning

[0008] The Transformer architecture was originally used for sequence modeling tasks in natural language processing and was introduced to the field of computer vision due to its powerful feature representation capabilities. Transformer-based models can utilize self-attention mechanisms to model long-range dependencies and have achieved significant performance in various image restoration tasks. For example, Restormer [reduces the number of operations by computing scaled dot-product attention in the feature depth domain while maintaining computational efficiency. Uformer utilizes locally enhanced window Transformer blocks to improve the ability to capture local context. Compared to CNN-based methods, these Transformer-based methods can improve the performance of image restoration, but still have limitations in restoring image details.

[0009] 4. Image decoupling algorithm based on unsupervised learning

[0010] Diffusion models are a new type of generative model that simulate the gradual transition from a data distribution to a noise distribution and reconstruct clear images in the reverse process. For example, SR3 uses a denoising diffusion probability model for image super-resolution reconstruction and achieves better performance than methods based on generative adversarial networks. WeatherDiff restores images taken under adverse weather conditions using a guided denoising process on overlapping blocks. However, these methods become time-consuming during inference due to the need for multiple image blocks and a large number of sampling steps.

[0011] In summary, current image restoration algorithms perform well when dealing with the types of degradation seen during training, but their restoration performance often decreases significantly when faced with unknown or rare degradation conditions. In particular, in practical applications, images may encounter multiple complex degradation conditions such as blur, noise, weather effects, etc., which can severely affect image quality and have a negative impact on advanced applications. Although methods based on CNNs, GANs, Transformers, and diffusion models have made some progress in the field of image restoration, they still have limitations in terms of adaptability and flexibility. SUMMARY

[0012] To address the above problems, the present application proposes a multi-degradation image restoration method based on adaptive prompts, which includes the following steps:

[0013] A multi-degradation image restoration method based on adaptive prompts, characterized by the following steps:

[0014] S1, a degraded adaptive prompt learning mechanism is constructed, differences between degraded images and clean images are extracted, degraded prompts are adaptively generated, and prompt information is encoded into a hidden space as a key feature for processing multiple image degradation types, and the step S1 of constructing the degraded adaptive prompt learning mechanism specifically includes the following steps:

[0015] S11, a pair of degraded images and corresponding clean images are received, the degraded images and the clean images are connected along the channel dimension to form an input feature map, which can be represented as:

[0016] F = Concat (I GT ,I LQ ) (1)

[0017] S12, the S11 feature map is input into the pre-trained degraded adaptive prompt learning mechanism;

[0018] S13, the degraded adaptive prompt learning mechanism extracts feature differences between the degraded images and the clean images through a series of convolutional layers and nonlinear activation functions;

[0019] S14, the extracted feature differences are input into the prompt learning module, and the prompt learning module generates a degraded prompt vector through adaptive weighting and normalization operations;

[0020] S15, the generated degraded prompt vector is encoded into the hidden space using the degraded adaptive prompt learning network DAPLM to form key discriminative information, which is used for feature extraction and processing in the subsequent image restoration process;

[0021] p = DAPLM (F) (2) S2, a hidden space diffusion model is constructed, which can adaptively extract key degradation features from degraded images using only degraded images, and directly estimate prompt information similar to the degraded prompts generated by S1, and the step S2 of constructing the hidden space diffusion model specifically includes the following steps:

[0022] S21, first, the hidden space diffusion model, i.e., the degraded prompt estimator, is used, and only degraded images are used as input;

[0023] S22, the degraded prompt estimator adds Gaussian noise to the degraded image step by step through a forward diffusion process, with p = p0 as the starting point of the diffusion process, until it is completely converted into a Gaussian noise distribution, which can be described as:

[0024]

[0025] where t = 1... T, T is the total number of iterations, p t represents the noise feature at the t-th step, N is a Gaussian distribution, and through iterative derivation, the above equation is rewritten as:

[0026]

[0027] S23, in the reverse process, the degenerative prior information is generated from the pure Gaussian distribution, and the purpose of the reverse process of the diffusion model is to estimate the posterior distribution from the following formula:

[0028]

[0029] The degenerative hint estimator takes pure Gaussian noise as the starting point, removes noise step by step using conditional information, and recovers the hint information similar to the degenerative hint obtained in S1 stage as the conditional guidance to optimize the diffusion process, which can be described as:

[0030]

[0031] S25, after T iterations, the degenerative hint estimator outputs the final estimated degenerative hint, which is used for subsequent image inpainting process, and the optimization process is:

[0032]

[0033] S3, build a Fourier transform inpainting network guided by hints, integrate the hints extracted in S2 to enhance the feature representation ability of the model to global frequency features and local spatial information;

[0034] S4, build an image inpainting network based on adaptive hints, dynamically fine-tune the Fourier transform inpainting network using the hints generated in S3, and train the model through a large amount of training data;

[0035] S5, input the degraded image into the built image inpainting network to complete the accurate inpainting of the image. The step S1 of building a degenerative adaptive hint learning mechanism specifically includes the following steps:

[0036] S11, receive a pair of degraded images and corresponding clean images, connect the degraded images and clean images along the channel dimension to form an input feature map, which can be represented as:

[0037] F = Concat (I GT ,I LQ ) (8)

[0038] S12, input the S11 feature map into the pre-trained degenerative adaptive hint learning mechanism;

[0039] S13, the degenerative adaptive hint learning mechanism extracts the feature difference between the degraded image and the clean image through a series of convolution layers and nonlinear activation functions;

[0040] S14, input the extracted feature difference into the hint learning module, and the hint learning module generates a degradation hint vector through adaptive weighting and normalization operation;

[0041] S15, using the degradation adaptive hint learning network DAPLM to encode the generated degradation hint vector into the hidden space to form key discriminative information for subsequent image inpainting process of feature extraction and processing.

[0042] p = DAPLM (F) (9)

[0043] The step S2 is specifically:

[0044] S21, first using the hidden space diffusion model, i.e. the degradation hint estimator, only using the degraded image as input;

[0045] S22, the degradation hint estimator adds Gaussian noise to the degraded image step by step through the forward diffusion process, taking p = p0 as the sampling starting point of the diffusion process, until it is completely transformed into Gaussian noise distribution, which can be described as:

[0046]

[0047] Where t = 1... T, T is the total number of iterations, p t The noise feature at the t-th step is represented by N, which is a Gaussian distribution. Through iterative derivation, the above formula is rewritten as:

[0048]

[0049] S23, in the reverse process, the degradation prior information is generated from the pure Gaussian distribution, and the purpose of the reverse process of the diffusion model is to estimate the posterior distribution from the following formula:

[0050]

[0051] The degradation hint estimator takes pure Gaussian noise as the starting point, uses conditional information to gradually remove noise and restore the hint information similar to the degradation hint obtained in S1 stage as the conditional guidance to optimize the diffusion process, which can be described as:

[0052]

[0053] S25, after T iterations, the degradation hint estimator outputs the final estimated degradation hint, which is used in the subsequent image inpainting process, and the optimization process is:

[0054]

[0055] The step S3 is specifically:

[0056] S31, a prompt-guided Fourier transform restoration network is constructed, which is designed to integrate the degradation cues extracted in step S2, and the prompt-guided Fourier transform restoration network is used to embed the degradation cues into multiple layers of the network to enhance the feature representation capability of the model for global frequency features and local spatial information of the image; specifically, the obtained degradation prior p is embedded into the given input feature F in for different image restoration tasks:

[0057]

[0058] wherein represents element-wise multiplication, Norm(·) represents layer normalization, W l represents a linear layer weight, F n1 ,F in represent input and output feature maps, respectively;

[0059] S32, the input degraded image is converted to the frequency domain by Fourier transform, and the input is projected onto the query Q, the key K and the value V by 1×1 point convolution and 3×3 deconvolution; in addition, the correlation of Q and K in the frequency domain is estimated by performing fast Fourier transform on the obtained features Q and K; further, the total frequency feature is estimated, and then the output feature F out is obtained by using residual operation, which can be expressed as:

[0060]

[0061] wherein F(·) represents FFT, represents conjugate transpose operation, F -1 (·) represents inverse FFT;

[0062] S33, an aggregated local spatial domain information mechanism is constructed to embed the degradation prior into the input; subsequently, 1×1 point convolution and 3×3 deconvolution are used to aggregate information of different channels and spatially adjacent pixels, and the information of spatially adjacent pixels and the information of different channels are aggregated; at the same time, a gating mechanism of aggregated features is used to strengthen information coding; finally, the aggregated local features are estimated, and the output feature F out is obtained by using residual operation, which can be expressed as:

[0063]

[0064] I HQ =W c F ls +F in (21)

[0065] S34, build an image inpainting network based on adaptive cues, which combines the degradation cue information obtained in steps S1 and S2, and trains the image inpainting network using a large amount of training data containing multiple degradation types, including degraded images and their corresponding high-quality clean images; during the training process, dynamically adjust the network parameters to optimize the inpainting effect, ensure that the network can adapt to different types of image degradation; use a loss function to measure the difference between the inpainted image and the clean image, use an optimization algorithm to minimize the loss function, and improve the inpainting quality; after sufficient training, an image inpainting model capable of handling multiple degradation types is obtained, which can quickly and accurately inpaint the input degraded image in the inference stage.

[0066] Beneficial effects: Compared with the prior art, the present application provides a multi-degradation image restoration method based on adaptive cues, which can handle various types of image degradation in an open environment and has the following beneficial effects:

[0067] 1. Strong versatility: The prior art lacks flexibility and adaptability in handling multiple uncertain image degradations. The present application effectively adapts to and handles multiple degradation types including blur, noise, weather effects, etc. by introducing adaptive cues and latent space diffusion models. This method significantly improves the quality of image restoration, especially in handling boundary details and reducing artifacts.

[0068] 2. Computational efficiency: Traditional diffusion models require a large number of iteration steps to restore clean images, consuming a large amount of computational resources. The present application significantly reduces the computational complexity of the model by using low-dimensional latent vector representation cues. In the inference process, this method can restore an SPA+ image in less than 0.09 seconds, showing excellent real-time processing capability.

[0069] 3. Unified restoration framework: Compared with methods that require separate training of models for each degradation type, the present application proposes a unified two-stage restoration framework that can handle multiple degradation tasks simultaneously without the need to retrain the network for each degradation type. This method simplifies the training process, reduces computational and memory burden, and makes the model easier to deploy in practical application scenarios.

[0070] 4. Wide application: The present application has good robustness and universality, and can effectively restore images in various application environments. Whether it is dealing with degradation types seen during training or unknown new degradation, the present application can provide high-quality restoration results, suitable for various computer vision tasks such as autonomous driving, medical imaging, security monitoring, etc. BRIEF DESCRIPTION OF DRAWINGS

[0071] Figure 1This is a flow chart of the image restoration model training process of the present invention.

[0072] Figure 2 A flowchart of a prompt-guided Fourier transform restoration network is constructed for the present invention.

[0073] Figure 3 The high-fidelity restoration results of our method for various image restoration tasks, including (a) snowfall, (b) denoising, (c) raindrop removal, (d) dehazing, (e) denoising, (f) deblurring, and (g) low-light enhancement. The three columns in each image group represent the degraded image, the restored clean image, and the original image.

[0074] Figure 4 and Figure 5 This is a visualization result diagram of the adaptive hint-based multi-degraded image restoration method of the present invention and other methods.

[0075] Figure 6 and Figure 7 This is a comparison chart of the results of the adaptive hint-based multi-degraded image restoration method of the present invention and other methods on the Snow100K-S, Snow100K-K, Outdoor-Rain, RainDrop, Rain100L, SOTS, BSD68, GoPro and LOL datasets. DETAILED DESCRIPTION

[0076] The adaptive hint-based multi-degraded image restoration method of the present invention constructs a two-stage latent space diffusion model and example-driven restoration technology, enabling the model to effectively restore multiple degradation types in the open world according to the degradation features in the image, thereby significantly improving the image restoration effect in actual scenes.

[0077] The invention will be further described below with reference to specific embodiments.

[0078] Example 1:

[0079] like Figure 1 The first stage shows the specific process of training the degradation-adaptive cue prior learning: first, image data pairs under different degradation conditions, including blur and noise, are collected. These image pairs are fed into a pre-trained degradation-adaptive cue learning mechanism, which adaptively generates degradation cues by learning the differences between degraded and clean images. The cue information is encoded into a latent space to obtain key discriminative information. The generated cue is then fed into a cue-guided Fourier transform restoration network, which restores the degraded image based on these cue information. The restored result is compared with the clean image to calculate the loss, and the degradation-adaptive cue learning mechanism is optimized through backpropagation.

[0080] likeFigure 1 The specific process of training the degradation-adaptive prompt-guided latent space diffusion model is shown in the second stage. First, we initialize the weights of the degradation-adaptive prompt learning mechanism pre-trained in the first stage, which are used to generate degradation prompts. Then, we start the diffusion process with the degradation prompts generated by the degradation-adaptive prompt learning mechanism as the starting point, gradually introduce Gaussian noise into it, and simulate the evolution of data from clear state to noisy state.

[0081] In the iterative denoising stage, we gradually recover the clear image features from the noisy data through multiple iterations. In each iteration, the denoising network estimates and removes noise, while using the degradation information of the degradation-adaptive prompt learning mechanism as a conditional guide to optimize the diffusion process. This process optimizes the network parameters by minimizing the difference between the reconstruction error and the noise, thereby improving the quality of image restoration. In addition, we use the generated degradation prompts to dynamically fine-tune the PGFTR to adapt to the specific characteristics of different degraded images, further improving the detail quality and overall naturalness of the restored images. In the loss calculation and optimization stage, we compare the recovered features with the original high-quality images, calculate the reconstruction loss, and optimize the denoising network parameters to minimize the loss function.

[0082] As shown in Figure 2 The specific process of constructing the prompt-guided Fourier transform restoration network is as follows: first, input a degraded image. Then, use the pre-trained degradation-adaptive prompt learning mechanism to extract key degradation prompts from the degraded image, which contain important information about the type and degree of image degradation. These degradation prompts are used to construct the prompt-guided Fourier transform restoration network, which consists of multiple prompt-guided Fourier transformer blocks, each of which focuses on learning and modeling long-range dependencies and local details. The input degradation is first subjected to feature extraction through a series of convolutional layers, and then converted from the spatial domain to the frequency domain using the fast Fourier transform, preparing for further processing.

[0083] In the frequency domain, the extracted degradation prompts are embedded into the prompt-guided Fourier transformer blocks, guiding the network to perform targeted restoration on images of different degradation types. This process involves adjusting and optimizing the frequency features to restore the details and quality of the image. After a series of prompt-guided Fourier transformer block processing, the restored features are converted back to the spatial domain using the inverse fast Fourier transform, obtaining the preliminary restored clean image. By comparing the restored clean image with the clean image, the loss function such as mean square error or structural similarity index is calculated, and the network weights are updated using the backpropagation algorithm, thereby optimizing the performance of the prompt-guided Fourier transform restoration network.

[0084] This invention provides a multi-degraded image restoration method based on adaptive cues. By inputting degraded images and corresponding high-resolution clean images into a two-stage latent space diffusion model, this method enables the model to effectively handle various potential image degradation types based on adaptive cues, including but not limited to denoising, deblurring, rain removal, defogging, raindrop removal, and low-light enhancement. By integrating the adaptive cues into the model, the restoration of images with unknown degradation types in real-world scenarios is significantly improved. In particular, when multiple degradation conditions occur simultaneously, the method of this invention can provide a more accurate and efficient image restoration solution.

[0085] Figure 4 and Figure 5 The figure shows the visualization results of the adaptive hint-based multi-degraded image restoration method of the present invention and other methods. It can be seen that the adaptive hint-based multi-degraded image restoration method has better results than other methods.

[0086] Figure 6 and Figure 7 The following figure compares the results of our adaptive cue-based multi-degraded image restoration method and other methods on the Snow100K-S, Snow100K-K, Outdoor-Rain, RainDrop, Rain100L, SOTS, BSD68, GoPro, and LOL datasets. It can be seen that the adaptive cue-based multi-degraded image restoration method is more accurate than the other methods.

[0087] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

[0088] Although the above describes the specific implementation methods of the present invention, it does not limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A multi-degraded image restoration method based on adaptive hinting, characterized in that: The following steps are involved: S1, constructing a degradation adaptive hint learning mechanism, extracting the difference between the degraded image and the clean image, adaptively generating degradation hints, and encoding the hint information into a latent space as a key feature for processing various image degradation types. The construction of the degradation adaptive hint learning mechanism in step S1 specifically includes the following steps: S11 receives a pair of degraded images and corresponding clean images, concatenates the degraded and clean images along the channel dimension to form an input feature map, which can be expressed as; F=Concat(I GT ,I LQ ) (1) S12, inputs the S11 feature map into the pre-trained degradation adaptive cue learning mechanism; S13, the degradation adaptive hint learning mechanism extracts the feature differences between the degraded image and the clean image through a series of convolutional layers and nonlinear activation functions; S14, inputting the extracted feature differences into a prompt learning module, which generates a degradation prompt vector through adaptive weighting and normalization operations; S15, using the degradation adaptive hint learning network DAPLM to encode the generated degradation hint vector into the latent space to form key discriminant information for feature extraction and processing in the subsequent image restoration process; p=DAPLM(F) (2) S2: Constructing a latent space diffusion model. This model can adaptively extract key degradation features from the degraded image using only the degraded image, and directly estimate the hint information similar to the degradation hint generated in S1. Step S2 of constructing the latent space diffusion model specifically includes the following steps: S21, first uses the latent space diffusion model, i.e., the degradation hint estimator, using only the degraded image as input; S22, the degradation hint estimator uses the forward diffusion process, taking p=p0 as the sampling starting point of the diffusion process, and gradually adds Gaussian noise to the degraded image until it is completely converted into a Gaussian noise distribution. This part can be described as: Where t=1...T, T is the total number of iterations, p t represents the noise characteristics of the t-th step, N is the Gaussian distribution, and through iterative derivation, the previous formula is rewritten as: S23, in the reverse process, the degraded prior information is generated from the pure Gaussian distribution. The purpose of the reverse process of the diffusion model is to estimate the posterior distribution from the following formula: The degradation hint estimator uses pure Gaussian noise as a starting point, uses conditional information to gradually remove the noise and recover hint information similar to the degradation hint obtained in the S1 stage as a conditional guide to optimize the diffusion process. This part can be described as: S25, after T iterations, the degradation hint estimator outputs the final estimated degradation hint, which is used in the subsequent image restoration process. The optimization process is: S3, constructs a cue-guided Fourier transform restoration network to integrate the cues extracted in S2 to enhance the model's feature representation capabilities for global frequency features and local spatial information; S4, builds an adaptive cue-based image restoration network, dynamically fine-tunes the Fourier transform restoration network using the cues generated in S3, and trains the model with a large amount of training data; S5, input the degraded image into the constructed image restoration network to complete the accurate restoration of the image.

2. The method for restoring a multi-degraded image based on adaptive hinting according to claim 1, characterized in that: The step S1 of constructing the degradation adaptive prompt learning mechanism specifically includes the following steps: S11 receives a pair of degraded images and corresponding clean images, concatenates the degraded and clean images along the channel dimension to form an input feature map, which can be expressed as; F=Concat(I GT ,I LQ ) (8) S12, inputs the S11 feature map into the pre-trained degradation adaptive cue learning mechanism; S13, the degradation adaptive hint learning mechanism extracts the feature differences between the degraded image and the clean image through a series of convolutional layers and nonlinear activation functions; S14, inputting the extracted feature differences into a prompt learning module, which generates a degradation prompt vector through adaptive weighting and normalization operations; S15, using the degradation adaptive hint learning network DAPLM to encode the generated degradation hint vector into the latent space to form key discriminant information for feature extraction and processing in the subsequent image restoration process. p=DAPLM(F) (9).

3. The method for restoring a multi-degraded image based on adaptive hinting according to claim 1, characterized in that: The step S2 is specifically as follows: S21, first uses the latent space diffusion model, i.e., the degradation hint estimator, using only the degraded image as input; S22, the degradation hint estimator uses the forward diffusion process, taking p=p0 as the sampling starting point of the diffusion process, and gradually adds Gaussian noise to the degraded image until it is completely converted into a Gaussian noise distribution. This part can be described as: Where t=1...T, T is the total number of iterations, p t represents the noise characteristics of the t-th step, N is the Gaussian distribution, and through iterative derivation, the previous formula is rewritten as: S23, in the reverse process, the degraded prior information is generated from the pure Gaussian distribution. The purpose of the reverse process of the diffusion model is to estimate the posterior distribution from the following formula: The degradation hint estimator uses pure Gaussian noise as a starting point, uses conditional information to gradually remove the noise and recover hint information similar to the degradation hint obtained in the S1 stage as a conditional guide to optimize the diffusion process. This part can be described as: S25, after T iterations, the degradation hint estimator outputs the final estimated degradation hint, which is used in the subsequent image restoration process. The optimization process is: .

4. The method for restoring a multi-degraded image based on adaptive hinting according to claim 1, wherein: The step S3 is specifically as follows: S31, construct a hint-guided Fourier transform restoration network, which is designed to integrate the degradation hints extracted in step S2. The hint-guided Fourier transform restoration network embeds the degradation hints into multiple layers of the network to enhance the model's feature representation ability of the global frequency features and local spatial information of the image; specifically, the obtained degradation prior p is embedded into the given input feature F in Targeted guidance for different image restoration tasks: In the formula represents element-wise multiplication, Norm(·) represents layer normalization, W l represents the linear layer weight, F n1 ,F in Represent the input and output feature maps respectively; S32, converts the input degraded image into the frequency domain through Fourier transform, and converts the input into the frequency domain through 1×1 point-by-point convolution and 3×3 deconvolution. Project it onto the query Q, key K and value V; in addition, perform fast Fourier transform on the obtained features Q and K to estimate the correlation between Q and K in the frequency domain; further estimate the total frequency feature, and then use the residual operation to obtain the output feature F gf , this part can be expressed as: Where F(·) represents FFT, represents the conjugate transpose operation, F -1 (·) represents inverse FFT; S33, constructs a mechanism to aggregate local spatial domain information and embeds the degradation prior into the input; then, uses 1×1 point-by-point convolution and 3×3 deconvolution to aggregate information from different channels and spatially adjacent pixels, and aggregates information from spatially adjacent pixels and different channels; at the same time, uses a gating mechanism for aggregated features to strengthen information encoding; finally, estimates the aggregated local features and uses residual operations to obtain output features, which can be expressed as: I HQ =W c F ls +F in (21) S34, build an image restoration network based on adaptive prompts, which combines the degradation prompt information obtained in steps S1 and S2, and uses a large amount of training data containing multiple degradation types to train the image restoration network, and the training data includes degraded images and their corresponding high-quality clean images; during the training process, dynamically adjust the network parameters to optimize the restoration effect to ensure that the network can adapt to different types of image degradation; use the loss function to measure the difference between the restored image and the clean image, and use the optimization algorithm to minimize the loss function to improve the restoration quality; after sufficient training, obtain an image restoration model that can handle multiple degradation types, which can quickly and accurately repair the input degraded image in the inference stage.

Images