A visual task-driven degraded image restoration method

Abstract

The present application relates to the field of image restoration, and particularly relates to a visual task driven degraded image restoration method, which comprises the following steps: extracting initial features from training data input, and calculating features after normalization of the training data; adaptively combining the initial features and the features after normalization to obtain fused features; converting the initial features to frequency domain space by Fourier transform to obtain statistical information thereof, dynamically repairing input degraded image features by the statistical information to obtain restored features, reconstructing an image according to the restored features, and separately training a semantic perception decoder SAD and an intrinsic semantic enhancement module ISE to obtain a trained image restoration model; and using the trained image restoration model to restore input degraded images. The present application can more directly enhance semantic information of an image, and can better adapt to a downstream high-level semantic recognition task; and the present application can simultaneously restore multiple degraded images.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image restoration, in particular to a visual task driven degraded image restoration method. BACKGROUND

[0002] Image restoration task aims to restore a low-quality image suffering from visual degradation into a high-quality image with better visual effect through reasonable algorithm. Common image restoration tasks include image dehazing, image denoising, image deraining, image super-resolution, image deblurring, etc. Due to various degradation environments in the real world, image restoration task is particularly important. Image restoration algorithm has a wide range of applications in real-world scenarios such as autonomous driving and monitoring systems.

[0003] With the rise of deep learning in recent years, deep learning based image restoration methods have gradually surpassed traditional manual restoration methods and achieved better matching performance. People input the degraded image into the designed convolutional neural network, output the degraded image as the restored image, and use the corresponding clean image as the label for supervision.

[0004] However, the current mainstream image restoration methods are supervised by L1 or L2 loss function. Due to the nature of L1 and L2 loss function, the image restoration model trained by the image restoration model often has good visual effect, but the performance on downstream high-level semantic tasks (such as image classification, object detection, etc.) is not good. Although there have been specific visual recognition methods under degraded environment, they mostly assume that the target data set and model are available during training, which limits their practicality in the real world. At the same time, the current image restoration methods mostly only target a specific degradation, which affects their practicality in the complex real world. SUMMARY

[0005] To solve the above problems, the present application provides a visual task driven degraded image restoration method.

[0006] The method constructs an image restoration model, inputs the degraded image into the image restoration model for restoration, and the restored image retains the semantics of the degraded image, comprising the following steps:

[0007] Step one, prepare a batch of degraded images and corresponding label clean images as training data;

[0008] Step two, input the training data into the existing high-level semantic feature extractor to obtain the corresponding initial feature F i , calculate the normalized feature of the training data through the degradation normalization and compensation module DNC in the intrinsic semantic enhancement module ISE 1≤i≤N, N is the number of the degradation normalization and compensation modules DNC in the intrinsic semantic enhancement module ISE;

[0009] Step three, the initial feature F i is combined with the normalized feature using the attention mechanism to obtain the fusion feature

[0010] Step four, the initial feature F i is combined with the fusion feature and input into the Fourier guide module FGM in the intrinsic semantic enhancement module ISE, and the initial feature F i is converted to the frequency domain space by Fourier transform to obtain its statistical information, and the input degraded image feature is dynamically repaired by the statistical information to obtain the guide map M;

[0011] Step five, using the guide map M to perform feature weighting on all fusion features to obtain the restored feature F';

[0012] Step six, inputting the restored feature F' into the semantic perception decoder SAD, if the training data corresponding to F' is a label clean image, the semantic perception decoder SAD generates a predicted clean image; if the training data corresponding to F' is a degraded image, the semantic perception decoder SAD generates an abnormal image;

[0013] Step seven, defining the total loss function L SAS of the semantic perception decoder SAD training;

[0014] Step eight, fixing the parameters of the high-level semantic feature extractor, training the semantic perception decoder SAD according to the total loss function L SAD ;

[0015] Step nine, defining the semantic loss function L smc and the reconstruction loss function L rec of the intrinsic semantic enhancement module ISE.

[0016] Step ten, fixing the parameters of the semantic perception decoder SAD, training the intrinsic semantic enhancement module ISE according to the semantic loss function L smc and the reconstruction loss function L rec , to obtain the trained image restoration model;

[0017] Step eleven, using the trained image restoration model to restore the input degraded image.

[0018] Further, the feature of the training data after normalization is calculated by the degradation normalization and compensation module DNC in the intrinsic semantic enhancement module ISE in step two Specifically comprising:

[0019]

[0020] where i represents the i-th degradation type, IN(F i ) represents normalization, μ(·) represents mean calculation, σ(·) represents standard deviation calculation, and γ and β are parameters of normalized IN.

[0021] Further, step three specifically comprises:

[0022]

[0023]

[0024] where W2 and W1 represent the weights of the two full connection layers, relu represents the ReLU activation function, pooling represents global maximum pooling, sigmoid represents the Sigmoid activation function, A F represents the attention map obtained in the attention mechanism.

[0025] Further, step four specifically comprises:

[0026] By Fourier transform, the initial feature F i is converted to the frequency domain space, and the corresponding space before conversion is the spatial domain, to obtain the initial feature F i corresponding to the frequency domain information:

[0027] F(F i )=R(F i )+jI(F i );

[0028]

[0029] where e represents the natural constant, j represents the imaginary unit, h and w represent the image height and width dimensions index respectively, H represents the image height, W represents the image width, u and v represent two-dimensional indexes of traversing the frequency domain information, F(F i ) represents the frequency domain information corresponding to the initial feature, R(F i ) represents the real part of F(F i ), I(F i ) represents the imaginary part of F(F i ), and F(F i )(u,v) represents the value of the frequency domain information F(F i ) at index (u,v).

[0030] According to the Fourier transform characteristics, the initial feature F iStatistical characteristics A(F i )(u,v) on the amplitude spectrum

[0031]

[0032] Feature extraction is performed on the amplitude spectrum A(F i )(u,v) using a 1x1 convolution kernel:

[0033] A'(F i )(u,v) = A(F i )(u,v) * kernel1;

[0034] where * represents convolution operation, kernel1 represents a 1x1 convolution kernel, and A'(F i )(u,v) represents the result of feature extraction.

[0035] The frequency domain information F'(F i ) corresponding to A'(F i )(u,v) is converted back to the spatial domain through inverse Fourier transform to obtain the spatial domain feature M' corresponding to F'(F i ).

[0036] M' is passed through a convolution network and a Softmax activation function in the spatial domain to obtain the guidance map M.

[0037] Further, step five specifically includes:

[0038]

[0039] where, represents the multiplication operation.

[0040] Further, step seven specifically includes:

[0041] According to the difference between the predicted clean image and the labeled clean image and the difference between the model predicted abnormal image and the input degraded image, the reconstruction loss function L

[0042] The relative similarity of the labeled clean image and the predicted clean image, the degraded image I deg and the abnormal image is calculated, and the similarity ranking loss L sr is calculated according to the two similarities.

[0043] The total loss of the semantic perception decoder SAD training is

[0044] Further, step nine specifically includes:

[0045] Input the label clean image corresponding to the input picture into the high-level semantic feature extractor to obtain the semantic space feature F of the label clean image cle ;

[0046] According to the restored feature F' and the semantic space feature F of the label clean image, a semantic loss function L cle is calculated smc :

[0047]

[0048] Wherein, ‖·‖2 represents the L2 loss function normalization of the existing image restoration method, f' and f' represent the restored feature F' and the semantic space feature F of the label clean image respectively cle after maximum pooling; cle .

[0049] According to the predicted clean image and the label clean image, a reconstruction loss function L rec. is defined

[0050] L rec. =||f SAD (f ISE (F′))-y||1;

[0051] Wherein, ‖·‖1 represents the L1 loss function regularization of the existing image restoration method, f SAD (·) represents the semantic perception decoder SAD, f ISE (·) represents the intrinsic semantic enhancement module ISE, and y represents the label clean image.

[0052] One or more technical solutions provided in the embodiments of the application have at least the following technical effects or advantages:

[0053] 1. The application selects to repair the feature corresponding to the degraded image in the semantic space. Compared with the previous image restoration algorithm, the application can more directly enhance the semantic information of the image, and then make the intrinsic semantics of the restored image more accurately recovered. This also makes the image restored by the application better adapt to the downstream high-level semantic recognition task.

[0054] 2. The application can simultaneously restore multiple degraded images, while most of the current image restoration algorithms can only restore a specific degradation. Compared with these methods, the application has higher flexibility. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The module schematic diagram of the intrinsic semantic enhancement module ISE provided in the embodiments of the application is shown in the figure;

[0056] Figure 2 A schematic diagram of the quality degradation normalization and compensation module DNC provided for the embodiments of the present application is shown in FIG. 2.

[0057] Figure 3 A schematic diagram of the Fourier guide module FGM provided for the embodiments of the present application is shown in FIG. 3.

[0058] Figure 4 A schematic diagram of the priori endowment optimization strategy provided for the embodiments of the present application is shown in FIG. 4.

[0059] Figure 5 A schematic diagram of the semantic-aware decoder SAD module provided for the embodiments of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0060] Before the technical solutions of the embodiments of the present application are described in detail, the terms and names involved are explained. In the present specification, components with the same name or the same reference numeral represent similar or identical structures, and are only for illustrative purposes.

[0061] The present application designs a visual task-driven degradation restoration method aiming at the problems and defects in the current mainstream image restoration method. The method can simultaneously process multiple degradations, and can make the restored image better match the downstream task. Compared with the current mainstream image restoration method, the present application can simultaneously restore multiple degradations, and effectively improve the performance of the restored image in the downstream visual recognition task. The present application can be inserted as a preprocessing tool before the existing high-level semantic model to enhance the degraded data and improve the performance of the downstream task. The downstream task refers to subsequent image processing of the high-quality image after the degraded image is restored to a high-quality image, such as image classification and object detection.

[0062] The present application constructs an image restoration model to convert the input degraded image into a high-quality image that preserves the semantics of the degraded image. Degraded images are images whose quality has been reduced due to various factors such as limitations of image acquisition devices, noise interference during image transmission, etc., and usually include problems such as blurring, noise, distortion, etc.

[0063] The image restoration model includes an intrinsic semantic enhancement module ISE and a semantic-aware decoder SAD. The intrinsic semantic enhancement module ISE uses instance normalization and frequency domain information to repair different degradation semantic features. Then, the present application proposes a priori endowment optimization strategy. Specifically, a two-stage training mode is used, such as Figure 4As shown in the figure, to improve the ability of the image restoration model to perceive different semantics and repair semantic features, and in the first stage of training, a reasonable loss function is proposed to improve the ability of the semantic-aware decoder SAD to perceive different semantics by relatively ranking clean and degraded images, without increasing the parameter amount of the semantic-aware decoder SAD.

[0064] 1. Constructing the intrinsic semantic enhancement module ISE to repair different degraded semantic features

[0065] The specific structure of the intrinsic semantic enhancement module ISE of the image restoration model is as shown in the figure Figure 1 As shown in the figure, the intrinsic semantic enhancement module ISE includes a multi-branch structure composed of N degraded normalization and compensation modules DNC, which processes the input degraded image features in a dynamic manner; the intrinsic semantic enhancement module ISE also includes a Fourier guide module FGM, which guides the recovery of image semantic features by using the statistical information of degraded image features; wherein the DNC module is composed of an instance normalization IN and an attention mechanism module, and each DNC module initially corresponds to a degradation type and uses corresponding data for preprocessing; and the FGM module is composed of a Fourier transform, a 1x1 convolution, and an inverse Fourier transform module.

[0066] 1.1 Feature extraction and fusion

[0067] Prepare a batch of degraded images and the corresponding label clean images of the degraded images as training data, where the label clean image refers to the high-quality image corresponding to the degraded image. Input the training data into the existing high-level semantic feature extractor to obtain the corresponding initial feature F i Input the initial feature F i into the degraded normalization and compensation module DNC, as shown in the figure Figure 2 , obtain the feature of the training data after normalization through instance normalization IN The specific calculation process of the instance normalization is as follows:

[0068]

[0069] Wherein, i represents the i-th degradation type and the i-th DNC module, represents the feature of the training data after normalization, IN(F i ) represents normalization, μ(·) represents mean calculation, σ(·) represents standard deviation calculation, and γ and β are parameters learned from the training data.

[0070] The instance normalization IN inevitably affects the initial feature F iThe information with the discriminant function in the loss causes loss, and further causes adverse effects on semantic recovery and image restoration. i The initial feature F After the IN normalization, the normalized feature is combined to perform information compensation, specifically, the adaptive combination is performed by using an attention mechanism, which is composed of two fully connected layers, two activation functions, and a global maximum pooling layer, and the calculation process is as follows:

[0071]

[0072]

[0073] Wherein, W2 and W1 represent the weights of the two fully connected layers, relu represents the ReLU activation function, pooling represents the global maximum pooling, sigmoid represents the Sigmoid activation function, A F represents the attention map obtained in the attention mechanism, is the fusion feature output by the final DNC.

[0074] 1.2 The initial feature F i is combined with the fusion feature to obtain the guidance map M

[0075] The initial feature F i is further input into the Fourier guidance module FGM, and the statistical information of the initial feature F i is used to dynamically repair the input degraded image feature, as shown in . Figure 3

[0076] Firstly, the input initial feature F i is converted into the frequency domain space by Fourier transform to obtain its statistical information, and the space before conversion corresponds to the spatial domain, and the specific calculation formula is as follows:

[0077] F(F i )=R(F i )+jI(F i );

[0078]

[0079] Wherein, e represents the natural constant, j represents the imaginary unit, h and w represent the index of image height and width dimension respectively, H represents the image height, W represents the image width, u and v represent the two-dimensional index of traversing the frequency domain information respectively, and F(F i ) represents the frequency domain information corresponding to the initial feature. R(Fi ) represents the real part of F(F i ) represents the imaginary part of F(F i ) represents the real part of F(F i ) represents the imaginary part of F(F i )(u, v) represents the value of the frequency domain information F(F i ) at index (u, v).

[0080] According to the Fourier transform characteristics, the initial feature F i The statistical characteristics A(F i )(u, v) on the amplitude spectrum are calculated as follows:

[0081]

[0082] Then, a 1x1 convolution kernel is used to extract features from the amplitude spectrum A(F i )(u, v), and the extracted results are transformed back to the spatial domain by inverse Fourier transform. The calculation process of feature extraction from the amplitude spectrum A(F i )(u, v) is as follows:

[0083] A'(F i )(u, v) = A(F i )(u, v) * kernel1;

[0084] Where * represents convolution operation, kernel1 represents a 1x1 convolution kernel, A'(F i )(u, v) represents the result of feature extraction, and then the frequency domain information F'(F i ) corresponding to A'(F i )(u, v) is converted back to the spatial domain by inverse Fourier transform, obtaining the spatial domain feature M' corresponding to F'(F i ).

[0085] In the spatial domain, M' is obtained by convolution network and Softmax activation function to obtain the guide map M to guide the network to recover the degraded feature.

[0086] 1.3 Obtain the restored feature

[0087] In the intrinsic semantic enhancement module ISE, a multi-branch structure composed of N DNC modules in parallel is constructed, and each DNC module performs feature extraction and fusion operation on the training data to obtain a series of fusion features The guide map M is used to weight the N features to obtain the restored feature F':

[0088]

[0089] Wherein, Representative evidence multiplication operation.

[0090] 2. Constructing semantic-aware decoder SAD

[0091] The image restoration model further comprises a semantic-aware decoder SAD composed of two attention mechanism modules and a convolutional layer, as shown in the structure of Figure 5 The semantic-aware decoder SAD maps the input restored feature F' to the corresponding image to generate a predicted image, and the input image is a label clean image, then the semantic-aware decoder SAD generates a predicted clean image; the input image is a degraded image, then the semantic-aware decoder SAD generates a non-normal image. The non-normal image refers to an image that does not conform to the normal image distribution in the image space.

[0092] 3. Training the semantic-aware decoder SAD

[0093] The high-level semantic feature extractor and the semantic-aware decoder SAD are connected in series, the degraded image and the label clean image are input into the image restoration model, the parameters of the high-level semantic feature extractor are fixed, only the parameters of the semantic-aware decoder SAD are trained, and the high-quality image is reconstructed according to the semantic features of the label clean image.

[0094] 3.1 Loss of reconstructed image

[0095] A given label clean image I cle and a degraded image I deg in the training data are input into the high-level semantic feature extractor to obtain the semantic space feature F cle of the label clean image and the semantic space feature F deg of the degraded image:

[0096] F cle = f ext (I cle );

[0097] F deg = f ext (I deg );

[0098] Wherein, f ext (·) represents the high-level semantic feature extractor.

[0099] Then, the semantic space feature F cle of the label clean image and the semantic space feature F deg of the degraded image are input into the semantic-aware decoder SAD for reconstruction to obtain the predicted clean image and the non-normal image The specific calculation is as follows:

[0100]

[0101]

[0102] wherein, f SAD denotes the semantic-aware decoder SAD.

[0103] According to the difference between the predicted clean image and the labeled clean image, the model predicts the difference between the abnormal image and the input degraded image, and calculates the reconstruction loss function

[0104]

[0105] wherein, λ denotes a weight parameter measuring the reconstruction effect of the predicted clean image and the reconstruction effect of the abnormal image,

[0106] ‖·‖1 denotes the L1 loss function regularization of the current mainstream image restoration method.

[0107] 3.2 Similarity ranking loss

[0108] The relative similarity cle between the labeled clean image I and the predicted clean image is calculated in the image space.

[0109]

[0110] wherein, A(·) represents the relative similarity calculation function, and ‖·‖2 denotes the L2 loss function normalization of the current mainstream image restoration method.

[0111] The relative similarity deg between the degraded image I and the abnormal image is calculated in the image space.

[0112]

[0113] The similarity ranking loss L sr is:

[0114]

[0115] wherein, m represents a preset threshold of the difference between the two relative similarities and .

[0116] The total loss function L SAD for training the semantic-aware decoder SAD is obtained by adding the reconstruction loss and the similarity ranking loss:

[0117]

[0118] Fixing the parameters of the high-level semantic feature extractor according to the total loss function L SAD Training the semantic-aware decoder SAD.

[0119] 4. Training of the intrinsic semantic enhancement module ISE

[0120] Cascading the trained semantic-aware decoder SAD and the intrinsic semantic enhancement module ISE, fixing the parameters of the semantic-aware decoder SAD, only training and updating the parameters of the intrinsic semantic enhancement module ISE, and repairing the degraded image and its semantic features.

[0121] According to the recovered feature F' and the semantic space feature F of the labeled clean image cle , a semantic loss function L smc is calculated:

[0122]

[0123] where ||·||2 represents the L2 loss function normalization of the current mainstream image restoration method, and f' and f c ' le represent the recovered feature F' and the semantic space feature F of the labeled clean image cle after maximum pooling, respectively.

[0124] At the same time, according to the predicted clean image and the labeled clean image, a reconstruction loss function L rec. is calculated at the image level to constrain the process of reconstructing a high-quality image. The specific calculation process is:

[0125] L rec. = ||f SAD (f ISE (F'))-y||1;

[0126] where ||·||1 represents the L1 loss function regularization of the current mainstream image restoration method, f SAD (·) represents the semantic-aware decoder SAD, f ISE (·) represents the intrinsic semantic enhancement module ISE, and y represents the labeled clean image.

[0127] Fixing the parameters of the semantic-aware decoder SAD, training the intrinsic semantic enhancement module ISE according to the semantic loss function L smc and the reconstruction loss function L rec .

[0128] After the inner semantic enhancement module ISE and the semantic-aware decoder are trained, a trained image restoration model is obtained, and a high-quality image with reserved semantics can be obtained by inputting a degraded image into the trained image restoration model.

[0129] To sum up, the application proposes a high-level semantic visual task driven degraded image restoration method. First, an inner semantic enhancement module ISE is designed by using individual case normalization and frequency domain information to repair the input degraded semantic features in a dynamic manner. Then, a priori assignment optimization strategy is designed, and a two-stage training process is used to effectively perceive different semantic features and repair degraded semantic features. Finally, a reasonable loss function is designed in the first stage of training to ensure the perception ability of different semantic features while reducing the parameter quantity requirement. The experimental results on the public data sets of image classification and target detection show that the application can effectively combine with the downstream high-level semantic visual task.

[0130] The above-described embodiments are merely preferred embodiments of the application and are not intended to limit the scope of the application. Various modifications and improvements to the technical solutions of the application made by those of ordinary skill in the art without departing from the design spirit of the application shall fall within the protection scope of the claims of the application.

Claims

1. A visual task driven degraded image restoration method, characterized in that, A image restoration model is constructed, and a degraded image is input into the image restoration model for restoration, and the restored image retains the semantics of the degraded image. The method comprises the following steps: Step one, prepare a batch of degraded images and corresponding label clean images as training data; Step two, input the training data into the existing high-level semantic feature extractor to obtain the corresponding initial features , calculate the features of the training data after normalization by the degradation normalization and compensation module DNC in the intrinsic semantic enhancement module ISE , , is the number of degradation normalization and compensation modules DNC in the intrinsic semantic enhancement module ISE. Step three, combine the initial features with the normalized features using an attention mechanism to obtain the fused features ; Step 4, initial features With fusion features The input is fed into the Fourier Guided Module (FGM) within the Intrinsic Semantic Enhancement (ISE) module, where the initial features are transformed using Fourier transform. The statistical information of the input degraded image is obtained by converting it to the frequency domain. The statistical information is then used to dynamically repair the features of the input image to obtain the guide image M. Step five, using the guide map M to all fusion features Perform feature weighting to get restored features ; Step six, restoring the features inputting the semantic-aware decoder SAD, if the corresponding training data is a label clean image, the semantic-aware decoder SAD generates a predicted clean image; if the corresponding training data is a degraded image, the semantic-aware decoder SAD generates an abnormal image; Step seven, define the total loss function for semantic-aware decoder SAD training ; Step eight, fix the parameters of the high-level semantic feature extractor according to the total loss function training the semantic-aware decoder SAD; Step nine, defining the semantic loss function of the intrinsic semantic enhancement module ISE and the reconstruction loss function ; Step ten, fix the parameters of the semantic-aware decoder SAD according to the semantic loss function and the reconstruction loss function Train the intrinsic semantic enhancement module ISE to obtain the trained image restoration model; Step eleven, using the trained image restoration model to restore the input degraded image.

2. The visual task-driven degraded image restoration method according to claim 1, wherein, the feature of the training data after normalization calculated by the degradation normalization and compensation module DNC in the intrinsic semantic enhancement module ISE in step two , specifically comprising: ； wherein, represents a first degradation type, represents normalization, denotes mean calculation, denotes standard deviation calculation, and is a parameter of the normalized IN.

3. The visual task-driven degraded image restoration method of claim 1, wherein, Step three specifically comprises: ； ； where, and represent weights of two fully connected layers, denotes a ReLU activation function, denotes a global max pooling, denotes a Sigmoid activation function, denotes an attention map obtained in the attention mechanism.

4. The visual task-driven degraded image restoration method of claim 1, wherein, Step four specifically comprises: By Fourier transform, the initial feature is converted to a frequency domain space, and the corresponding conversion before the space is a spatial domain, to obtain the initial feature corresponding to the frequency domain information: ； ； wherein, represents the natural constant, represents the imaginary unit, and represent the indices of the image height and width dimension, respectively, represents the image height, represents the image width, and represent the two-dimensional indices of the traversed frequency domain information, respectively, represents the frequency domain information corresponding to the initial feature, represents the real part of the imaginary part of , and represents the value of the frequency domain information at index . According to the Fourier transform characteristics, the initial features are obtained Statistical characteristics on the amplitude spectrum : ； Using a 1x1 convolutional kernel on the magnitude spectrum Perform feature extraction: ； wherein, represents a convolution operation, represents a 1 x 1 convolution kernel, represents the result of feature extraction; Will corresponding frequency domain information converted back into the spatial domain by inverse Fourier transform, to obtain corresponding spatial domain features ; In the spatial domain, we have By the convolution network and the Softmax activation function, we get the guidance map M.

5. The visual task-driven degraded image restoration method of claim 1, wherein, Step five specifically comprises: ； wherein represents a matrix multiplication operation.

6. The visual task-driven degraded image restoration method of claim 1, wherein, Step seven specifically comprises: According to the difference between the predicted clean image and the labeled clean image and the difference between the model predicted abnormal image and the input degraded image, a reconstruction loss function is calculated ; computing a relative similarity of the labeled clean image and the predicted clean image, the degraded image and the abnormal image , computing a similarity ranking loss from the two similarities ; The total loss of semantic-aware decoder SAD training is .

7. The visual task-driven degraded image restoration method of claim 1, wherein, Step nine specifically comprises: The label clean image corresponding to an input picture is input into a high-level semantic feature extractor to obtain semantic space features of the label clean image ; According to the restored feature and semantic space features of the label clean image , a semantic loss function is calculated : ； wherein, L2 loss function normalization of the existing image restoration method, and respectively represent the restored feature and the semantic space feature of the label clean image the result after max-pooling; According to the predicted clean image and the labeled clean image, a reconstruction loss function is defined : ； wherein, denotes an L1 loss function regularization of the existing image restoration method, denotes a semantic-aware decoder SAD, denotes an intrinsic semantic enhancement module ISE, denotes a label clean image.

Images