A multi-degraded image restoration method based on degradation correlation utilization

Abstract

The present invention relates to the field of image restoration, and more specifically to a multi-degraded image restoration method based on degradation correlation. The method comprises: constructing an image restoration model using a universal image restoration network of any Transformer architecture as the backbone network; establishing corresponding prior codebooks for each degradation type involved in the training; each prior codebook is responsible for extracting degradation information from the degradation features of the corresponding type, and restoring the image based on the formation principle of the degradation type. Prior codebooks for all degradation types are integrated in the decomposition space to obtain a unified degradation component prior codebook. The degradation component prior codebook is responsible for extracting degradation information from all degradation features and performing dynamic weighted restoration based on the prediction results of the degradation formation principle. A model loss is defined to train the image restoration model, and the trained image restoration network is used to restore the degraded image. The present invention has strong scalability when the number of degradation types increases; and the model can also show good generalization for unseen degradation.

Description

Technical Field

[0001] The present invention relates to the field of image restoration, and in particular to a method for restoring multiple degraded images based on degradation correlation utilization. Background Art

[0002] Image restoration aims to recover potentially high-quality images from degraded observations. Common image restoration tasks include image deraining, image dehazing, image denoising, image deblurring, and low-light image enhancement. Beyond classic photo restoration, image restoration has also been widely applied to other real-world scenarios, such as autonomous driving and road monitoring. Currently, most image restoration methods address only one type of degradation, making them difficult to adapt to the variability and unpredictability of degradation types in complex scenes. Consequently, deployments often require switching between multiple degradation restoration models, resulting in significant consumption of storage and computing resources.

[0003] To address this issue, work in the era of deep learning has begun to focus on the problem of multi-degraded image restoration. These methods can be categorized into general multi-degraded image restoration methods and joint multi-degraded image restoration methods. The former uses the same network structure but employs different network parameters for different image restoration tasks. The latter uses a set of network parameters to handle different types of degradation. It should be noted that the present invention belongs to the latter, joint multi-degraded image restoration method.

[0004] Currently, joint multi-degraded image restoration methods typically focus on restoring two or three degradation types. While achieving promising performance, practical applications often require manual specification of the specific degradation type for the input. For example, in addition to a shared backbone network, different head and tail networks are used for different degradation types, or different queries are set for different degradation types in the Transformer architecture for discriminative querying. This pre-specified approach significantly limits the model's further application scenarios. A few methods, such as AirNet, which utilizes contrastive learning, have overcome this limitation. However, it is important to note that current joint multi-degraded image restoration methods all suffer from the common problem of mutually exclusive learning between degradations (see the aforementioned multi-head and multi-tail networks and contrastive learning). This underutilizes the correlations between various degradations, resulting in poor scalability (i.e., network performance significantly degrades as the number of degradation types increases) and poor generalization (i.e., poor restoration performance for unseen degradations). Summary of the Invention

[0005] To solve the above problems, the present invention provides a method for restoring multiple degraded images based on degradation correlation.

[0006] The method includes:

[0007] Step 1: Build an image restoration model using a general image restoration network of any Transformer architecture as the backbone network;

[0008] Step 2: For each of the K degradation types involved in the training process, establish the corresponding prior codebooks. It is a degraded type;

[0009] Step 3: Prepare K types of degraded images, where images of the same degradation type are grouped as a batch. Each degraded image D corresponds to a clean image Y and a degradation type label y. The preprocessed images are used as training data for degradation type-guided knowledge collection.

[0010] Step 4: Define the image features output at each scale of the backbone network as input features x i , i represents the serial number of each scale of the backbone network, and calculates the input feature x i The degradation and enhancement feature SDAM(x i ); Assume that the degradation type of the batch of images is k, according to the prior codebook and degradation enhancement feature SDAM(x i ) for the input feature x i According to the formation principle of this type of degradation, restore it and get the pseudo clean feature x′ i , the pseudo clean feature x′ i As the input feature of the next scale of the backbone network;

[0011] Step 5: Input the degraded image D into the image restoration model to generate the restored image I;

[0012] Step 6: Define the reconstruction loss for generating the restored image Defining degradation classification loss Then the overall loss function of the degradation-oriented knowledge collection is λ cls To balance the reconstruction loss With classification loss Parameters;

[0013] Step 7: Overall loss function based on knowledge collection guided by degradation type Train the image restoration model until the preset training completion conditions are met;

[0014] Step 8: Integrate the prior codebooks of all degradation types Get a unified degradation component prior

[0015] Step 9: prepare a batch of images containing different degradation types, and use the pre-processed images as degradation component-oriented knowledge organization training data;

[0016] Step 10: Calculate input feature x i The degradation and enhancement feature SDAM(x i ), with a unified degradation component prior For the degradation enhancement feature SDAM(x i ) to obtain the degradation representation aggregated by the degradation components prior

[0017] Step 11: Degraded representation based on prior aggregation of degraded components For the input feature x i According to the predicted degradation type, dynamic recovery based on the degradation formation principle is performed to obtain the pseudo-clean feature x′ i , the pseudo clean feature x′ i As the input feature of the next scale of the backbone network;

[0018] Step 12, regenerating the restored image according to step 5;

[0019] Step 13: Reconstruction loss The image restoration model is trained with the overall loss as the knowledge sorting guided by the degradation components until the preset training completion condition is reached;

[0020] Step 14: Use the trained image restoration model to restore the degraded image.

[0021] Furthermore, the step 4 specifically includes:

[0022] Step 4A: Define the image features output at each scale of the backbone network as input features x i , i represents the serial number of each scale of the backbone network, and the same scale D as the input degraded image D is used to scale D i As a supervisory signal, calculate the input feature x i The degradation and enhancement feature SDAM(x i ), and define the degradation supervision loss

[0023] Step 4B: Assume that the degradation type of the batch of images is k, and perform the degradation enhancement feature SDAM(x i ) to extract the degradation type information and update the prior codebook of the type of degradation Computational prior-guided degraded representations

[0024] Step 4C, based on the degradation For the input feature x i According to the formation principle of this type of degradation, restore it and get the pseudo clean feature x′ i , the pseudo clean feature x′ iAs the input feature of the next scale of the backbone network.

[0025] Furthermore, the degradation supervision loss is defined as described in step 4A Specifically refers to:

[0026] By comparing the supervision signal with the input feature x i The projection conv(x in image space i ), calculate the degradation supervision loss

[0027]

[0028] Among them, conv(.) represents the convolution operator, and ||.|||1 represents the L1 norm.

[0029] Furthermore, the input feature x is calculated as described in step 4A i The degradation and enhancement feature SDAM(x i ), specifically including:

[0030] Project the image space conv(x i ) is restored to the original channel dimension through 1×1 convolution as the degraded attention map A i , then the degradation enhancement feature SDAM(x i )for:

[0031] SDAM(x i )=conv(x i )*A i +x i .

[0032] Furthermore, step 4B specifically includes:

[0033]

[0034] Among them, x i Represents the input feature, and assumes that the image degradation type of this batch is k, represents the prior codebook with degradation type k, C i Represents the input feature x i Dimension, T represents matrix transpose, Softmax(.) represents the normalized exponential function, SDAM(x i ) represents x i The degradation enhancement features of , LN represents layer normalization.

[0035] Furthermore, step 4C specifically includes:

[0036] Will with x iAfter passing through a fully connected layer, each channel is cascaded as a unified degraded representation, and the unified degraded representation is passed through x i Parameter network θ corresponding to degradation attributes j Then generate the degradation kernel, degradation kernel and degradation feature x i According to the degradation formation principle, the pseudo-clean feature x′ is obtained by interaction i :

[0037]

[0038] Among them, j∈{additive, multiplicative, curvature}, j represents x i The degradation property of W h , W x represents the fully connected layer, [·] represents channel-wise cascading, θ j Represents the parameter network used to generate the degradation kernel of the corresponding attribute, φ j Represents the degraded kernel and degraded features x generated by the parameter network i Interact according to the principle of degradation formation.

[0039] Furthermore, the reconstruction loss of the step of generating the restored image is defined as described in step 6 Defining degradation classification loss Specifically refers to:

[0040] By comparing the restored image I with the clean image Y, and combining the degradation supervision loss Define the reconstruction loss for the step of generating restored images By comparing the degradation-enhanced features SDAM(x i ) The predicted degradation type and the degradation type label y of the input degradation image D, define the degradation classification loss

[0041] Furthermore, step eight specifically includes:

[0042] For each Perform singular value decomposition and get Among them, U k , V k Represents the prior codebook of degradation type k The singular vector of Sk represents the prior codebook The singular values ​​of , T represents the matrix transpose;

[0043] For K groups of singular vectors with different degradation types Perform the fusion:

[0044]

[0045] in, represents a shallow fusion network, represents a set of singular vectors based on degradation components after the fusion of K groups of singular vectors based on degradation types;

[0046] Fixed K groups of degradation type prior codebooks Singular values ​​after decomposition Through the prediction head For singular values Weighted

[0047]

[0048] The unified degradation component prior

[0049] Furthermore, step 11 specifically includes:

[0050] Degraded representation based on prior aggregation of degraded components For the input feature x i Perform dynamic recovery to obtain pseudo-clean features x′ i :

[0051]

[0052] Among them, β j is the weight coefficient of each parameter network, which is determined by the prediction head Provide; j∈{additive, multiplicative, convolutional}, j represents x i The degradation property of W h , W x represents the fully connected layer, [·] represents channel-wise cascading, θ j Represents the parameter network used to generate the degradation kernel of the corresponding attribute, φ j Represents the degraded kernel and degraded features x generated by the parameter network i Interact according to the principle of degradation formation.

[0053] Furthermore, the universal image restoration network of the Transformer architecture described in step 1 is a Restormer network.

[0054] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0055] In response to the problem of mutually exclusive learning between degradations that is prevalent in current multi-degraded image restoration methods, the present invention designs a new multi-degraded image restoration optimization framework. The framework consists of two stages, namely, degradation type-oriented knowledge collection and degradation component-oriented knowledge organization. By obtaining a unified degradation representation based on degradation components, the correlation between various degradation types can be fully utilized. When the number of degradation types increases, the model still maintains good performance, that is, strong scalability; and for unseen degradation, the model can also show good generalization. In addition, the present invention also inherits the advantages of recent multi-degraded image restoration methods, that is, there is no need to pre-specify degradation types. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram illustrating the beneficial effects of the multi-degraded image restoration method provided by an embodiment of the present invention;

[0057] Figure 2 Schematic diagram of a multi-degraded image restoration optimization framework provided by an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of an embedded backbone image restoration network method provided by an embodiment of the present invention;

[0059] Figure 4 A schematic diagram of a meta-priori learning module provided in an embodiment of the present invention;

[0060] Figure 5 Schematic diagram of the degradation supervision module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. Before describing in detail the technical solutions of each embodiment of the present invention, the nouns and terms involved are explained. In this specification, components with the same name or the same number represent similar or identical structures and are for illustrative purposes only.

[0062] This paper addresses the problems and defects in the current joint multi-degraded image restoration methods and constructs an image restoration model, aiming to use a single model to remove multiple image degradations and fully utilize the correlation between multiple degradations to obtain a more universal degradation representation. Figure 1 As shown, compared with the previous paradigm of mutually exclusive learning between degradations, the present invention obtains a more universal degradation representation based on degradation components through degradation correlation learning, thereby achieving better generalization to unseen degradations.

[0063] This paper uses the correlation between degradation and proposes a two-stage optimization strategy to construct an image restoration model. Figure 2As shown, the present invention adopts a two-stage optimization framework: the first stage is degradation type-oriented knowledge collection, and the second stage is degradation component-oriented knowledge organization. The two stages are bridged by learnable principal component analysis. The image restoration model can be integrated into any Transformer architecture image restoration backbone network.

[0064] 1. Build an image restoration model

[0065] The image restoration model is constructed using a universal image restoration network of any Transformer architecture as the backbone network. Here, the image restoration model is constructed using a Restormer network as the backbone network to explain the present invention.

[0066] The image restoration model builds a meta-prior learning module based on the backbone network to support the two-stage multi-degraded image restoration optimization framework described in this invention. The architecture of the meta-prior learning module is shown in the figure below. Figure 4 As shown in , it consists of two parts: prior-guided degradation representation and principle-guided degradation operation, and can be instantiated as type prior learning module and component prior learning module at different optimization stages. The meta-prior learning module is embedded in the backbone network of the image restoration model, as shown in Figure 3 As shown in Figure 3, for multi-scale backbone networks, the meta-prior learning module is embedded at the end of each scale of the backbone network, that is, where the feature map size changes. For single-scale backbone networks, the meta-prior learning module embedding position can be set arbitrarily.

[0067] 2. Construct a priori codebook of image degradation types

[0068] For the K types of degradation involved in the training process, corresponding prior codebooks are established N is the codebook capacity for each degradation type, C d is the codebook feature dimension, k∈[1, K] is the degradation type number, is a priori codebook with degradation type k, where each degradation type priori codebook is randomly initialized. Here, we take K=5 as an example to explain the present invention, and the five degradation types are rain removal, fog removal, noise removal, blur removal, and low light. Therefore, the overall embedding framework of the present invention is as follows Figure 3 As shown, Figure 3 The top part shows the general image restoration backbone network diagram, and the bottom part shows the embedded diagram of the present invention, which includes a unified degradation prior codebook and a meta-prior learning module for embedding at each scale.

[0069] 3. Degradation-oriented knowledge collection

[0070] 3.1 Data Preparation

[0071] Prepare K types of degraded images, grouping them into a batch. Each degraded image is assigned a clean image and a degradation type label. After data augmentation, including random horizontal and vertical flipping and rotation, these images are cropped into 128×128 image patches to serve as training data for degradation-based knowledge collection.

[0072] 3.2 Degradation Information of Enhanced Image Features

[0073] Strengthen the degradation information of features in the backbone network of the image restoration model to facilitate the subsequent extraction of degradation type information.

[0074] The architecture of the degradation supervision module is as follows Figure 5 As shown in the figure, by introducing supervision of degraded images into the intermediate features of the network, the degradation information in the features is strengthened and the background content information is diluted, which facilitates the subsequent content-independent degradation prior extraction.

[0075] Taking the i-th meta-prior learning module embedded in the image restoration model as an example, the image features output by the backbone network module of the corresponding scale of the i-th meta-prior learning module are used as the input features x i , i also represents the serial number of each scale of the backbone network, H i and W i Represents the input features x i The height and width of C i Represents the input feature x i The dimension of . For the input feature x i Apply the degraded supervision module SDAM in the meta-prior learning module to enhance the input feature x i The supervision signal is the same scale scaling of the input degraded image. supply.

[0076] By comparing the supervision signal with the input feature x i The projection conv(x in image space i ), we can get the degradation supervision loss of the i-th meta-prior learning module

[0077]

[0078] Among them, conv(.) represents the convolution operator, and ||.|||1 represents the L1 norm.

[0079] The image space under degradation supervision is projected conv(x i ) is restored to the original channel dimension through 1×1 convolution as the degraded attention map A i , Then the final degradation enhancement feature SDAM(x i ) can be expressed as:

[0080] SDAM(x i )=conv(x i )*A i +x i ;

[0081] The output of the degradation supervision module can be regarded as the input feature x i The degraded residual enhanced by the degraded attention map is attached. In the reverse optimization, For the input feature x i The gradient of is stopped to prevent the perturbation of the backbone recovery features, such as Figure 4 shown.

[0082] 3.3 Input feature x i Predicting degradation type

[0083] is the prediction head in the i-th element prior learning module, which consists of two layers of convolution. i ) Input prediction head Get the predicted degradation type of the input image

[0084] 3.4 Extracting degradation type information from degradation enhancement features using prior codebook

[0085] Extract the degradation type information from the degradation enhancement feature to update the prior codebook of different degradation types And get the prior-guided degraded representation Taking the i-th meta-prior learning module embedded in the image restoration model as an example, the input feature is represented as Assume that the degradation type of the batch of images is k, then the prior codebook corresponding to the degradation type k is Responsible for extracting input features x i The degradation information in the , and the degradation representation aggregated by the degradation type prior is obtained

[0086]

[0087] Among them, T represents matrix transpose, Softmax(.) represents the normalized exponential function, SDAM(x i ) represents the output of x by the degradation supervision module i The degradation enhancement feature of LN represents layer normalization. Only interact with features of degradation type k, avoiding leakage of prior information between degradation types. In the reverse gradient update, Continuous iteration becomes the k-th type of degradation-specific prior representation, realizing degradation type-guided knowledge collection.

[0088] 3.5 Modeling degradation attributes for different degradation types based on degradation formation principles

[0089] According to the degradation formation principle, the degradation attributes of different degradation types are modeled. Among them, rain and noise are modeled as additive degradation, fog and low light are modeled as multiplicative degradation, and blur is modeled as roll degradation. Figure 4 shown.

[0090] 3.6 Principle-guided Restoration of Different Degraded Features Based on Prior-guided Degraded Representations

[0091] According to the degradation representation aggregated by degradation type prior The input feature x of the i-th meta-prior learning module i Restore based on the degradation formation principle of this batch type. The degradation represented by the degradation prior aggregation With input feature x i Each is cascaded into a unified degraded representation after passing through a fully connected layer. The unified degraded representation is obtained by x i Parameter network θ corresponding to degradation attributes j Then generate the degradation kernel, degradation kernel and degradation feature x i According to the degradation formation principle, the pseudo-clean feature x′ is obtained by interaction i :

[0092]

[0093] Where j represents x i The degradation property of W h , W x represents the fully connected layer, [·] represents channel-wise cascading, θ j Represents the parameter network used to generate the degradation kernel of the corresponding attribute, φ j Represents the degraded kernel and degraded features x generated by the parameter network i They interact according to the principle of degradation formation (i.e., addition, multiplication, and convolution). The additive and multiplicative parametric networks each consist of a convolutional layer and two residual blocks, while the convolutional parametric network has an additional 1×1 convolution layer to extend the channel dimension and generate a dynamic convolution kernel.

[0094] With pseudo-clean feature x′ i As the output of the i-th meta-prior learning module, it is input into the next-scale module of the backbone network.

[0095] 3.7 Generating restored images

[0096] Degrade the image After inputting the image restoration model, the restored image is generated like Figure 3 Shown at the bottom.

[0097] 3.8 Defining Loss

[0098] By comparing the restored image I with the clean image Y, and combining the degradation supervision loss Define the reconstruction loss for the step of generating restored images

[0099]

[0100] Where Y is the clean image paired with the input degraded image D.

[0101] By comparing the degradation-enhanced features SDAM(x i ) The predicted degradation type and the degradation type label y of the input degradation image D, define the degradation classification loss

[0102]

[0103] in, is the cross entropy loss function. Degraded classification loss Only used for the first stage optimization of the present invention, that is, the knowledge collection guided by the degradation type, the prediction head The prediction results are used as the dynamic weights for the second stage of optimization of the unobserved degradation. See step 5.3 for details.

[0104] The overall loss function of the first stage degradation-oriented knowledge collection is for:

[0105]

[0106] Among them, λ cls To balance the reconstruction loss With classification loss Parameters of λ cls Take 0.01 and Label smoothing is used in the dataset, and the smoothing coefficient is 0.1.

[0107] 3.9 Training the Image Restoration Model Based on the Overall Loss Function

[0108] According to the overall loss function Train the image restoration model, update the image restoration model network parameters and the degradation prior codebook Until the preset training completion conditions are met.

[0109] Here, 500 training times are used as the preset training completion condition.

[0110] 4. Integrate the prior codebooks of all degradation types to obtain a unified prior for degradation components

[0111] Degraded type prior codebook Arrange and obtain a unified degradation component prior To fully explore the correlation between different degradation types.

[0112] For each Perform singular value decomposition and get Among them, U k , Represents the prior codebook of degradation type k The singular vector of Sk represents the prior codebook The singular values ​​of , T represents the matrix transpose.

[0113] For K groups of singular vectors with different degradation types Perform the fusion:

[0114]

[0115] in, represents a shallow fusion network, Represents a set of singular vectors based on degradation components after the fusion of K groups of singular vectors based on degradation types. Different from directly selecting China and the United States The principal components are used for dimensionality reduction, here through the network The fusion dimension reduction is gradually performed in a learnable way, and the fusion coefficient is adaptively adjusted during the optimization process to realize the prior correlation mining of different degradation types.

[0116] Fixed K groups of degradation type prior codebooks Singular values ​​after decomposition The prediction head in the i-th meta-prior learning module obtained by the first stage optimization For singular values Weighted

[0117]

[0118] in The same in each embedding meta-prior module, The degradation representation constructed by dynamically adjusting the degradation type in each embedded meta-prior module is the trend of a certain degradation type, and the final unified degradation component prior is

[0119] 5. Knowledge organization based on degraded ingredients

[0120] 5.1 Data Preparation

[0121] Prepare a batch of images containing different types of degradation. After data enhancement including random horizontal / vertical flipping and random rotation, these images are cropped into 128×128 image blocks and sent to the image restoration network as degradation component-oriented knowledge organization training data.

[0122] 5.2 Extracting Degradation Information from Degradation Enhancement Features Using a Unified Degradation Component Prior

[0123] With a unified degradation component prior For the degradation enhancement feature SDAM(x i ) to obtain the degradation representation aggregated by the degradation components prior Compared with the prior based on degradation type, the prior based on degradation component that completes knowledge organization can achieve better generalization to unseen degradation and better scalability to more degradation. for:

[0124]

[0125] Different from the type-specific interactions in the first stage optimization, the unified degradation component prior Interacts with all types of degraded features.

[0126] 5.3 Principle-guided Restoration of Different Degraded Features Based on Prior-guided Degraded Representations

[0127] Degraded representation based on prior aggregation of degraded components For the input feature x i Perform dynamic recovery, that is, weighted recovery of different degradation attribute parameter networks, while exempting the restrictions specified by the degradation type, and obtain the pseudo-clean feature x′ i :

[0128]

[0129] Among them, β j is the weight coefficient of each parameter network, which is determined by the prediction head supply.

[0130] 5.4 Training the image restoration model based on the overall loss function

[0131] Reconstruction loss The overall loss as the knowledge organization guided by the degradation component is used to train the image restoration model, and the network parameters of the image restoration model and the shallow prior fusion network parameters are updated until the preset training completion conditions are met.

[0132] Here, 700 training times are used as the preset training completion condition.

[0133] The Adam optimizer is used to optimize the image restoration model. The learning strategy adopts cosine annealing, the initial learning rate is 1e-4, and the batch size is set to 8.

[0134] After the training is completed, a trained image restoration model is obtained, and the image restoration model is used to restore the input degraded image.

[0135] The above-described embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A method for restoring multiple degraded images based on degradation correlation, characterized in that: The following steps are involved: Step 1: Build an image restoration model using a general image restoration network of any Transformer architecture as the backbone network; Step 2: For each of the K degradation types involved in the training process, establish the corresponding prior codebooks. k∈[1, K] is the degradation type; Step 3: Prepare K types of degraded images, where images of the same degradation type are grouped as a batch. Each degraded image D corresponds to a clean image Y and a degradation type label y. The preprocessed images are used as training data for degradation type-guided knowledge collection. Step 4: Define the image features output at each scale of the backbone network as input features x i , i represents the serial number of each scale of the backbone network, and calculates the input feature x i The degradation enhancement feature SDAM(x i ); Assume that the degradation type of the batch of images is k, according to the prior codebook and degradation enhancement feature SDAM(x i ) for the input feature x i According to the formation principle of this type of degradation, restore it and get the pseudo clean feature x′ i , the pseudo clean feature x′ i As the input feature of the next scale of the backbone network; Step 5: Input the degraded image D into the image restoration model to generate the restored image I; Step 6: Define the reconstruction loss for generating the restored image Defining degradation classification loss Then the overall loss function of the degradation-oriented knowledge collection is λ cls To balance the reconstruction loss With classification loss Parameters; Step 7: Overall loss function based on knowledge collection guided by degradation type Train the image restoration model until the preset training completion conditions are met; Step 8: Integrate the prior codebooks of all degradation types Get a unified degradation component prior Step 9: prepare a batch of images containing different degradation types, and use the pre-processed images as degradation component-oriented knowledge organization training data; Step 10: Calculate input feature x i The degradation enhancement feature SDAM(x i ), with a unified degradation component prior For the degradation enhancement feature SDAM(x i ) to extract the degradation information from the degraded components and obtain the degradation representation a priori aggregated by the degradation components Step 11: Degraded representation based on prior aggregation of degraded components For the input feature x i According to the predicted degradation type, dynamic recovery based on the degradation formation principle is performed to obtain the pseudo-clean feature x′ i , the pseudo clean feature x′ i As the input feature of the next scale of the backbone network; Step 12, regenerating the restored image according to step 5; Step 13: Reconstruction loss The image restoration model is trained with the overall loss as the knowledge sorting guided by the degradation components until the preset training completion condition is reached; Step 14: Use the trained image restoration model to restore the degraded image.

2. The method for restoring multiple degraded images based on degradation correlation utilization according to claim 1, characterized in that: The step 4 specifically includes: Step 4A: Define the image features output at each scale of the backbone network as input features x i , i represents the serial number of each scale of the backbone network, and the same scale D as the input degraded image D is used to scale D i As a supervisory signal, calculate the input feature x i The degradation and enhancement feature SDAM(x i ), and define the degradation supervision loss Step 4B: Assume that the degradation type of the batch of images is k, and perform the degradation enhancement feature SDAM(x i ) to extract the degradation type information and update the prior codebook of the type of degradation Computational prior-guided degraded representations Step 4C, based on the degradation For the input feature x i According to the formation principle of this type of degradation, restore it and get the pseudo clean feature x′ i , the pseudo clean feature x′ i As the input feature of the next scale of the backbone network.

3. The method for restoring multiple degraded images based on degradation correlation utilization according to claim 2, characterized in that: Define the degradation supervision loss as described in step 4A Specifically refers to: By comparing the supervision signal with the input feature x i The projection conv(x in image space i ), calculate the degradation supervision loss Among them, conv(.) represents the convolution operator, and ||.|||1 represents the L1 norm.

4. The method for restoring multiple degraded images based on degradation correlation utilization according to claim 3, characterized in that: Calculate the input feature x as described in step 4A i The degradation enhancement feature SDAM(x i ), specifically including: Project the image space conv(x i ) is restored to the original channel dimension through 1×1 convolution as the degraded attention map A i , then the degradation enhancement feature SDAM(x i )for: SDAM(x i )=conv(x i )*A i +x i 。 5. The method for restoring multiple degraded images based on degradation correlation utilization according to claim 2, characterized in that: Step 4B specifically includes: Among them, x i Represents the input feature, and the degradation type of the batch of images is set to k. represents the prior codebook with degradation type k, C i Represents the input feature x i Dimension, T represents matrix transpose, Softmax(.) represents the normalized exponential function, SDAM(x i ) represents x i The degradation enhancement features of , LN represents layer normalization.

6. The method for restoring multiple degraded images based on degradation correlation utilization according to claim 2, characterized in that: Step 4C specifically includes: Will with x i After passing through a fully connected layer, the channels are cascaded as a unified degraded representation, and the unified degraded representation is passed through x i Parameter network θ corresponding to degradation attributes j Then generate the degradation kernel, degradation kernel and degradation feature x i According to the degradation formation principle, the pseudo-clean feature x′ is obtained by interaction i : Among them, j∈{additive, multiplicative, curvature}, j represents x i The degradation property of W h , W x represents the fully connected layer, [·] represents channel-wise cascading, θ j represents the parameter network used to generate the degradation kernel of the corresponding attribute, φ j Represents the degraded kernel and degraded features x generated by the parameter network i Interact according to the principle of degradation formation.

7. The method for restoring multiple degraded images based on degradation correlation utilization according to claim 2, characterized in that: Step 6 defines the reconstruction loss for generating the restored image Defining degradation classification loss Specifically refers to: By comparing the restored image I with the clean image Y, and combining the degradation supervision loss Define the reconstruction loss for the step of generating restored images By comparing the degradation-enhanced features SDAM(x i ) The predicted degradation type and the degradation type label y of the input degradation image D, define the degradation classification loss 8. The method for restoring multiple degraded images based on degradation correlation utilization according to claim 2, characterized in that: Step 8 specifically includes: For each Perform singular value decomposition and get Among them, U k , V k Represents the prior codebook of degradation type k The singular vectors, S k Represents the prior codebook The singular values ​​of , T represents the matrix transpose; For K groups of singular vectors with different degradation types Perform the fusion: in, represents a shallow fusion network, represents a set of singular vectors based on degradation components after the fusion of K groups of singular vectors based on degradation types; Fixed K groups of degradation type prior codebooks Singular values ​​after decomposition Through the prediction head For singular values Weighted The unified degradation component prior 9. The method for restoring multiple degraded images based on degradation correlation utilization according to claim 1, characterized in that: Step 11 specifically includes: Degraded representation based on prior aggregation of degraded components For the input feature x i Perform dynamic recovery to obtain pseudo-clean features x′ i : Among them, β j is the weight coefficient of each parameter network, which is determined by the prediction head Provide; j∈{additive, multiplicative, convolutional}, j represents x i The degradation property of W h , W x represents the fully connected layer, [·] represents channel-wise cascading, θ j represents the parameter network used to generate the degradation kernel of the corresponding attribute, φ j Represents the degraded kernel and degraded features x generated by the parameter network i Interact according to the principle of degradation formation.

10. The method for restoring multiple degraded images based on degradation correlation utilization according to claim 1, characterized in that: The general image restoration network of the Transformer architecture described in step 1 is the Restormer network.

Images