Super-resolution image reconstruction method based on regional degradation perception

Abstract

The invention discloses a super-resolution image reconstruction method based on regional degradation perception. The method comprises the following steps: respectively constructing a Dse training data set for a degradation region estimator and an SR training data set for a super-resolution model; constructing a degradation area estimator and a mask image encoder, and generating N random masks by using a random mask generator; training a degradation region estimator and a mask image encoder by using a random mask and Dse training data set; then, constructing an EDSR model combined with degraded region features as a super-division model, and generating N random masks' by using a random mask generator; and training the super-division model by using the random mask'and the SR training data set, and outputting a reconstructed image by the super-division model after the training is completed. According to the method, spatial domain difference degradation with diversified degradation shapes is designed to serve as data enhancement, perception and area adaptability optimization are carried out on the degradation area, the robustness and adaptability of the model are enhanced, and the super-resolution effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of single image super-resolution, and in particular to a super-resolution image reconstruction method based on regional degradation perception. Background Art

[0002] As an important research direction in the field of computer vision and image processing, super-resolution image reconstruction aims to reconstruct high-quality images with higher resolution from low-resolution images. It has a wide range of applications, including medical imaging, satellite remote sensing, video surveillance, image enhancement, etc. In recent years, with the rapid development of deep learning, super-resolution methods based on convolutional neural networks have made significant progress. By training end-to-end deep models, it is possible to reconstruct clearer and more detailed high-resolution images from low-resolution images.

[0003] Traditional super-resolution models are usually trained using the full-region degradation assumption, which assumes that the degradation process of all pixels in the image is uniform. In the preparation stage of the training dataset, high-resolution images are usually uniformly degraded (such as downsampling, blurring, or adding noise) to generate low-resolution images. This training method simplifies the complexity of degradation modeling and makes model training and optimization more convenient. However, in practical applications, the degradation process of low-resolution images is often not uniform, but has regional differences. For example, some areas may be severely degraded due to motion blur, changes in lighting conditions, or distortion of the compression algorithm, while other areas may be less degraded or even close to the original high-resolution quality.

[0004] The differences in these degraded areas pose a challenge to the performance of traditional super-resolution models. In actual scenarios, when the model faces low-resolution images with spatially differentiated degradation, it is often difficult to accurately restore the true information of each area. On the one hand, due to the tendency of global optimization, the model may over-super-resolve areas with less degradation, thereby introducing unnatural phenomena such as artifacts or over-sharp textures; on the other hand, for areas with more severe degradation, the model may not be able to fully reconstruct details, resulting in the effect of this area being far below expectations. This phenomenon not only reduces the robustness of the super-resolution model, but also affects the user experience and application effect to a certain extent. Summary of the invention

[0005] In view of the deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a super-resolution image reconstruction method based on regional degradation perception.

[0006] The technical solution of the present invention to solve the technical problem is to provide a super-resolution image reconstruction method based on regional degradation perception, characterized in that the method comprises the following steps:

[0007] Step 1: construct a Dse training dataset for the degradation region estimator and a SR training dataset for the super-resolution model respectively;

[0008] Step 2: construct a degraded region estimator and a mask image encoder, and use a random mask generator to generate N random masks; then use the random masks and the Dse training data set obtained in step 1 to train the degraded region estimator and the mask image encoder;

[0009] Step 3: Build a super-resolution model and use a random mask generator to generate N random masks mask′; then use the random masks mask′ and the SR training data set obtained in step 1 to train the super-resolution model. After the training is completed, the super-resolution model outputs a reconstructed image.

[0010] Among them, the EDSR model combined with the characteristics of the degraded area is used as the super-resolution model.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] (1) The present invention discloses a super-resolution image reconstruction method that can perceive the differences in degraded regions. It not only designs spatial difference degradation of diverse degraded shapes as data enhancement, but also perceives the degraded regions and optimizes regional adaptability, thereby improving the performance of the super-resolution model in actual scenes, thereby enhancing the robustness and adaptability of the model and improving the super-resolution effect.

[0013] (2) The present invention introduces a random mask generator to generate diverse and complex mask images to simulate different degraded regions in the real world. In the process of training the super-resolution model, the degraded low-resolution image is synthesized with the clean low-resolution image according to the mask to generate a low-quality image for input into the model. This method can effectively increase the diversity of training data and improve the robustness of the model to different degradation situations by strengthening the network's perception of degraded regions.

[0014] (3) The present invention designs a degradation region estimator, which can predict the distribution of degradation regions in an image based on a low-quality input image. By using the degradation region features extracted by the degradation region estimator, the model can process different degradation regions more accurately, avoiding the problem of over-processing of certain degradation regions in traditional methods.

[0015] (4) The present invention proposes a gating mechanism based on the idea of ​​a dynamic neural network to enhance the model's adaptability to complex data. The mechanism dynamically controls the network path through which data flows through the degraded region information predicted by the degraded region estimator. Specifically, based on the distribution of the degraded region, the gating mechanism determines the degree of image degradation and selectively activates a dynamic network branch. The dynamic network branch processes the shallow features and then combines them with the output of the deep feature extraction module of the EDSR model, thereby improving the flexibility and reconstruction quality of the super-resolution model when reconstructing low-resolution images. The gating mechanism and the dynamic network branch adopt a modular design with plug-and-play characteristics, and can be easily integrated into existing super-resolution models.

[0016] (5) The dynamic network branch of the present invention enhances the ability to perceive degraded regions. Combining the three improvements of data enhancement strategy, degraded region estimation and gated dynamic network branch, the present invention can more efficiently and accurately reconstruct high-quality images under complex degradation conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A partial image of a random mask of the present invention;

[0018] Figure 2 is a schematic diagram of the structure of the degradation region estimator of the present invention;

[0019] Figure 3 A training diagram of a degradation region estimator of the present invention;

[0020] Figure 4 It is a structural schematic diagram of the super-resolution model of the present invention;

[0021] Figure 5 It is a schematic diagram of the center circle mask and the lower right corner rectangle mask processing of the present invention;

[0022] Figure 6 This is a diagram showing the effects of the present invention and the original EDSR when facing inconsistent degradation areas. DETAILED DESCRIPTION

[0023] The specific embodiments of the present invention are given below. The specific embodiments are only used to further illustrate the present invention in detail and do not limit the protection scope of the present invention.

[0024] The present invention provides a super-resolution image reconstruction method based on regional degradation perception (hereinafter referred to as method), characterized in that the method comprises the following steps:

[0025] Step 1: construct a Dse training dataset for the degradation region estimator and a SR training dataset for the super-resolution model (i.e., super-resolution model) respectively;

[0026] Preferably, in step 1, the construction of the training data set for the degraded region estimator is: a clean low-resolution image Clean_LR of uniform size is cropped from the clean low-resolution data set, and then a degraded low-resolution image Degraded_LR is obtained through a degradation operation, and then each clean low-resolution image Clean_LR is paired with its corresponding degraded low-resolution image Degraded_LR one by one to form a number of data pairs, and the data set composed of all the data pairs is used as the training data set for the degraded region estimator, which is called the Dse training data set.

[0027] Preferably, in step 1, the construction of the training data set for the super-resolution model is: the high-resolution images HR and low-resolution images LR of the real-world super-resolution data set are cropped to generate several HR image blocks of uniform size and several LR image blocks of uniform size, and it is ensured that each HR image block and its corresponding LR image block come from the same position and keep the region aligned; then each HR image block is bicubic downsampled to obtain several clean low-resolution image blocks Clean_LR′; then the HR image blocks, the corresponding LR image blocks and the corresponding clean low-resolution image blocks Clean_LR′ are paired one by one to form several data pairs, and the data set consisting of all the data pairs is used as the training data set for the super-resolution model, which is called the SR training data set.

[0028] Preferably, in step 1, the size of the HR image block is four times the size of the LR image block.

[0029] Preferably, in step 1, the clean low-resolution dataset adopts the DIV2K low-resolution dataset; and the real-world super-resolution dataset adopts the RealSR_V3 dataset.

[0030] Preferably, in step 1, the degradation operation includes Gaussian blurring, adding noise and JPEG compression.

[0031] Step 2: Construct a degraded region estimator and a mask image encoder, and use a random mask generator to generate N random masks; then use the random masks and the Dse training data set obtained in step 1 to train the degraded region estimator (e.g. Figure 3 As shown) and a mask image encoder;

[0032] Preferably, in step 2, the degradation region estimator (such as Figure 2 The proposed method combines the advantages of convolutional layer CNN and visual transformer ViT. CNN is used to extract local features in the front and ViT is used to extract global features in the back. It consists of a shallow feature extraction module, a deep feature extraction module and a global feature extraction module connected in sequence.

[0033] Preferably, in step 2, the shallow feature extraction module is composed of a convolutional layer, through which the shallow feature F is extracted. SF ; In this embodiment, the size of the convolution layer is 3×3; Using a 3×3 convolution layer can balance the performance and efficiency of shallow feature extraction, reduce the number of model parameters, and reduce the risk of overfitting.

[0034] The deep feature extraction module consists of five residual groups connected sequentially, a residual connection, and a convolutional layer; each residual group consists of four convolutional layers;

[0035] The global feature extraction module consists of five ViTs connected sequentially.

[0036] Preferably, in step 2, the mask image encoder is composed of five ViTs connected in sequence.

[0037] Preferably, in step 2, the specific process of using a random mask generator to generate a random mask is: first initialize an all-zero mask matrix for storing the mask; then randomly generate N curve starting points on the mask matrix, and then generate N curves of unequal lengths through a curve generation function, and then draw a circle with this point as the center and a radius of 3 to the maximum circular radius for each point on the curve; finally, use a Gaussian filter to smooth the edge of the mask, and the generated smooth mask matrix is ​​the random mask, ensuring that the generated mask has a natural transition rather than a hard boundary; the random mask is used to simulate the degradation mode of different regions; a mask value of 0 represents a non-degraded region, and a mask value of 1 represents a degraded region.

[0038] Preferably, in step 2, the specific operation of the curve generation function is: given a curve starting point, then set the current position point as the starting point, randomly select a moving direction from the left, lower left, lower, lower right and right directions, move one step in the moving direction, that is, set to 1, and then set the current position point as the new moving position; at each move, determine whether it exceeds the boundary of the mask matrix; if it exceeds, stay where it is and enter the next loop directly; if it does not exceed, continue to move; repeat the above operations until the number of cycles is equal to the maximum length of the curve, and then stop moving. In this embodiment, the size of the random mask is 224×224, the number of starting points N is 12, the maximum length of the curve is 120, and the maximum circle radius is 15.

[0039] Preferably, in step 2, the specific steps of training the degraded region estimator and the mask image encoder are: using a random mask mask to generate a low-quality image LQ; then inputting the low-quality image LQ into the degraded region estimator to extract the degraded region features; at the same time, inputting the random mask mask into the mask image encoder to obtain the mask image features; then performing a matrix multiplication operation on the degraded region features and the mask image features; finally, using a loss function to shorten the distance between matching features (matching features, i.e., mask image features and degraded region features containing the mask information) and lengthen the distance between unmatched features, thereby optimizing the parameters of the degraded region estimator and the mask image encoder so that the degraded region features are closer to the corresponding mask image features in the feature space.

[0040] Preferably, in step 2, the low-quality image LQ is generated as follows:

[0041] LQ=mask*degraded_lr+(1-mask)*clean_lr (1)

[0043] In formula (1), Degraded_LR is the low-resolution image after degradation processing (i.e., degraded low-resolution image), Clean LR is the clean low-resolution image; mask marks the degraded area;

[0044] Preferably, in step 2, the degraded region features are obtained as follows: the low-quality image LQ is subjected to feature extraction by a shallow feature extraction module, and the shallow feature F is output. SF ; Then the shallow feature F SF The first residual group transmitted to the deep feature extraction module and the residual connection of the deep feature extraction module; then the shallow feature F SF After the residual groups are connected sequentially, the sum of the input and output of the previous residual group is used as the input of the next residual group; then the output of the last residual group is combined with the shallow feature F at the residual connection. SF Add element by element; then enhance the features through the convolution layer to obtain the deep features F DF ; Finally, the deep feature F DF It is input into the global feature extraction module and processed by five layers of ViT to obtain the degraded area features of the image.

[0045] The calculation formula of the deep feature extraction module is:

[0046]

[0047] In formula (2), F SF Indicates shallow features; H i represents the i-th residual group, i = 1 to 5; H ConvRepresents the weight parameter of a convolutional layer; F DF Indicates deep features; F i represents the result of adding the input and output of the i-th residual group;

[0048] The calculation formula for each residual group is:

[0049]

[0050] In formula (3), Conv h represents a convolutional layer with a kernel size of 3×3; F i and F i+1 represents the input and output of the i+1th residual group; x head Conv h The output result of ; ReLU6 represents the ReLU6 activation function; f1 represents the convolution layer with a kernel size of 7×7; f2 represents the convolution layer with a kernel size of 5×5; Conv g Represents a convolutional layer with a kernel size of 1×1.

[0051] Step 3: Build a super-resolution model and use a random mask generator to generate N random masks mask′; then use the random masks mask′ and the SR training data set obtained in step 1 to train the super-resolution model. After the training is completed, the super-resolution model outputs a reconstructed image.

[0052] Among them, the EDSR (Enhanced Deep Super-Resolution) model combined with the degraded region features is used as the super-resolution model;

[0053] Preferably, in step 3, the super-resolution model is composed of the trained degradation region estimator and the gating module Gat obtained in step 2. e , linear layer Fc, dynamic network branch DynamicNet and EDSR model; the EDSR model is composed of a shallow feature extraction module, a deep feature extraction module and an upsampling module connected in sequence;

[0054] The EDSR model combined with the characteristics of the degraded region (such as Figure 4 The calculation formula is as follows:

[0055]

[0056] In formula (4), Conv_first represents the shallow feature extraction module of EDSR; x first represents the output of Conv_first; Dse represents the degraded region estimator; Fc represents the linear layer; Repeat represents the operation of replicating the vector of shape [batchsize, 64, 1, 1] in both height and width dimensions; xdse represents the processing result of the low-quality image LQ′ after Dse, Fc, and Repeat in sequence; x fd Represents x first and x dse The addition result of ; Gate represents the gate control module; gate represents the output result of Gate; DynamicNet represents the dynamic network branch; x dyn Represents the output result of DynamicNet; Conv_body represents the deep feature extraction module of EDSR; x body Represents deep features; res represents x body 、gate[0]*x dyn and x fd The sum of the three; Upsample is the upsampling module of EDSR; hr represents the output result of Upsample, that is, the output image of the entire model. first The channels of and res are both 64-dimensional.

[0057] Preferably, in step 3, the specific process of using the random mask generator to generate the random mask is the same as that in step 2.

[0058] Preferably, in step 3, the process of training the super-resolution model is: using mask mask′ to generate a low-quality image LQ′, and then inputting the low-quality image LQ′ into the super-resolution model; in the super-resolution model, the low-quality image LQ′ is passed through a shallow feature extraction module to obtain shallow features x first ; At the same time, the trained degraded region estimator extracts the low-quality image LQ′ into a dimension N dim The degraded region features; then the dimension is N dim The degraded region features are input into the gate control module Gate to obtain gate (gate is a vector containing two numbers, and the gate value is 0 or 1); at the same time, in order to integrate the degraded region features into the EDSR model, a learnable linear layer is used to obtain the dimension N dim The degraded region feature is transformed into 64 dimensions, and then replicated in the height and width dimensions to obtain the new degraded region feature x dse ; Then the shallow feature x first With the new degenerate region feature x dse The added results are input into the deep feature extraction module of EDSR and the dynamic network branch DynamicNet to obtain the deep features x body and branch feature x dyn ; Then branch feature x dyn Multiply with gate[0] (gate[0] is the first element of gate) followed by shallow feature x first and deep features x bodyAdd them together to get the addition result res; finally, the addition result res is input to the upsampling module to obtain the reconstructed image.

[0059] Preferably, in step 3, the low-quality image LQ′ is generated as follows:

[0060] LQ′=mask′*LR+(1-mask′)*clean_lr′(5)

[0062] In formula (5), LR represents the low-resolution image; Clean_LR′ represents the clean low-resolution image block.

[0063] Preferably, in step 3, the calculation formula of the EDSR model is as follows:

[0064]

[0065] Preferably, in step 3, the shallow feature extraction module is a 3×3 convolutional layer that converts a 3-channel image into a 64-channel feature map.

[0066] Preferably, in step 3, the gate control module Gate comprises a three-layer fully connected network for converting the input dimension N dim The degraded region features are gradually reduced in dimension: In this embodiment, the LeakyReLU activation function is first used to reduce the dimension of the input feature from 512 to 128, then to 32, and finally outputs a 2D feature; then the Gumbel-Softmax activation function is used to convert the output into a discrete one-hot vector to represent the category selection. This operation is usually used to handle hard selection in classification tasks to ensure that the gradient flow can be maintained during training.

[0067] Preferably, in step 3, the dimension conversion of the degraded region feature is as follows:

[0068] x dse =Repeat(Fc(Dse(LQ′)))(7)

[0070] In formula (7), in this embodiment, LQ′[batchsize,3,h,w] is obtained by the degradation region estimator with a dimension of N dim The degradation region features [batchsize,N dim ], and then pass through the linear layer Fc to obtain the degraded region feature [batchsize, 64], and then copy it on H*W to obtain the new degraded region feature x dse [batchsize,64,h,w].

[0071] Preferably, in step 3, the dynamic network branch DynamicNet is composed of a convolutional layer with a kernel size of 3×3, a convolutional layer with a kernel size of 1×1, a convolutional layer with a kernel size of 3×3, a convolutional layer with a kernel size of 1×1, and a convolutional layer with a kernel size of 3×3, which are connected in sequence; each convolutional layer uses an adaptive activation function. At present, many networks improve performance by repeatedly stacking conventional convolutional units. However, although the adaptive activation function has a larger computational complexity than an ordinary convolutional unit, it can achieve an effect similar to that of multiple ordinary convolutional units by stacking fewer units. The key to the adaptive activation function is that learnable parameters are introduced in the activation function of each convolutional layer. Compared with the traditional activation function that only uses parameter-free ReLU and other activation functions, the adaptive activation function can provide stronger expression capabilities.

[0072] The formula for the adaptive activation function is as follows:

[0073] x act =x*Sigmoid(Conv(x))(8)

[0075] In formula (8), Conv represents a 3×3 convolutional layer; Sigmoid is an activation function; * represents element-by-element multiplication; x act Represents the output result after the input x uses the adaptive activation function.

[0076] Embodiment 1:

[0077] Step 1: Construct training datasets for the degraded region estimator and the super-resolution model (i.e., super-resolution model).

[0078] The training data set for the degraded region estimator is: 224×224 clean low-resolution images Clean_LR are obtained by cropping from the DIV2K low-resolution data set. These clean low-resolution images Clean_LR are first Gaussian blurred, noisy, and compressed with JPEG to obtain degraded low-resolution images Degraded_LR. Then, the clean low-resolution images Clean_LR and the corresponding degraded low-resolution images Degraded_LR are paired one by one to form several data pairs, and the data set composed of all the data pairs is used as the training data set for the degraded region estimator, called the Dse training data set.

[0079] The training data set for the super-resolution model is: the original high-resolution HR image and low-resolution LR image of the RealSR_V3 training set are cropped to generate several 192×192 HR image blocks and several 48×48 LR image blocks, where the size of the HR image block is four times the size of the LR image block, and it is ensured that each HR image block and its corresponding LR image block come from the same position to keep the region aligned; then each HR image block is bicubicly downsampled to obtain several clean low-resolution image blocks Clean_LR′; then the HR image block, the corresponding LR image block and the corresponding clean low-resolution image block Clean_LR′ are paired one by one to form several data pairs, and the data set consisting of all the data pairs is used as the training data set for the super-resolution model, called the SR training data set;

[0080] Step 2: Training of the degraded region estimator. First, use the random mask generator to generate N random masks, and then use the random mask mask to fuse the degraded low-resolution image Degraded_LR and the clean low-resolution image Clean_LR in the Dse training data set to generate a low-quality image LQ containing different degraded regions; then the low-quality image LQ is input into the degraded region estimator, and the corresponding random mask mask is input into the mask image encoder. Since the batchsize of this experiment is set to 32, the output data of the degraded region estimator is imgs_f[32,512], and the output data of the mask image encoder is mask_f[32,512]. Perform matrix multiplication on imgs_f and mask_f to obtain the similarity matrix. Then calculate the cross entropy loss function. The degraded region estimator and the mask image encoder are continuously optimized through the loss function until the set maximum iteration round of 20 epochs is reached to obtain the trained degraded region estimator.

[0081] During training, a random mask generator is used to pre-generate 100 different masks and the DIV2K validation set is used as the validation set. In order to evaluate the performance of the degraded region estimator, the feature nearest match accuracy is proposed. The calculation idea of ​​the feature nearest match accuracy is as follows: the low-quality image (LQ) is generated by fusing the mask with the degraded low-resolution image and the clean low-resolution image, so each low-quality image LQ has a corresponding mask. The number of masks used for testing is specified as 100. A low-quality image LQ is input into the degraded region estimator to obtain its degraded region features, and 100 mask masks are input into the mask image estimator to obtain 100 features. Then, the distance between the low-quality image LQ feature and the 100 mask image features is calculated, and the feature with the closest distance is selected as the predicted mask mask, which is compared with the true corresponding mask, and finally the matching accuracy is calculated. The number of iterations is determined by observing the value of the accuracy (the iteration round is obtained when the optimal and stable accuracy value is displayed) to avoid overfitting.

[0082] The test experiment settings include three different image datasets and three mask datasets with different distributions, constituting 9 experimental data. Each experimental data combines an image dataset and a mask dataset for testing. This design is used to verify the recognition and generalization ability of the degraded region estimator on masks with different distributions on different datasets. The three different image datasets are the DIV2K low-resolution test set, the AIM19 dataset, and the NTIRE20 dataset. The first mask dataset is 100 mask images obtained by the random mask generator, called RandMask100. The second mask dataset is 100 mask images containing 1 to 10 random rectangles, called BoxMask100. The third mask dataset is 100 mask images containing 1 to 10 random circles, called CirMask100. The performance test of the degraded region estimator is shown in Table 1 below:

[0083] Table 1

[0084] Dataset RandMask100 BoxMask100 CirMask100 DIV2K 0.98 0.91 0.93 AIM19 0.96 0.92 0.95 NTIRE20 0.97 0.90 0.94

[0085] Since the masks of the training set are generated by a random mask generator and have curved arc characteristics, the circular mask test set is closer in shape to the training set. In this case, the model will be more likely to generalize to the circular test set. Although the performance of the model trained based on the random mask generator on the rectangular test set is slightly reduced, it still demonstrates excellent generalization ability and is able to maintain high performance on mask datasets with different shape distributions, demonstrating its adaptability and robustness in dealing with mask shape changes and unseen data.

[0086] Step 3: Training of the super-resolution model. First, use a random mask generator to generate N random masks mask′, and then use the random mask mask′ to fuse the LR image block in the SR training data set and the clean low-resolution image block Clean_LR′ to generate a low-quality image LQ′. Then the low-quality image LQ′ is sent to the super-resolution model proposed by this method, and the result (super-resolution image) is output. Finally, the L1 loss function is used to make the super-resolution image approach the label image, that is, the corresponding HR image block, and optimize the super-resolution model.

[0087] In order to verify the effect of the method of the present invention, the RealSR_V3 training set is used for training. Tests are performed on the AIM19, NTIRE20, DRealSR, and RealSR_V3 datasets. PSNR and SSIM are used for indicator calculation. PSNR is calculated on the Y channel. The test results of this method and EDSR on the four datasets of AIM19, NTIRE20, DRealSR, and RealSR_V3 are shown in Table 2.

[0088] Table 2

[0089]

[0090]

[0091] Two images of RealSR_V3 are extracted and masked to simulate the inconsistency of the degradation area. In order to avoid the same distribution as the mask during training, the mask processing uses a central circle and a lower right corner rectangle respectively. The image processing is as follows Figure 5 The effect comparison is as follows. Figure 6 As shown. Figure 6 It can be seen that when the EDSR model processes low-resolution images with spatially different degradation, it will over-resolve the less degraded areas due to the global optimization tendency, resulting in unnatural phenomena such as over-sharp textures. However, this method can perform targeted super-resolution on degraded areas.

[0092] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A super-resolution image reconstruction method based on regional degradation perception, characterized in that: The method comprises the following steps: Step 1: construct a Dse training dataset for the degradation region estimator and a SR training dataset for the super-resolution model respectively; Step 2: construct a degraded region estimator and a mask image encoder, and use a random mask generator to generate N random masks; then use the random masks and the Dse training data set obtained in step 1 to train the degraded region estimator and the mask image encoder; Step 3: Build a super-resolution model and use a random mask generator to generate N random masks mask'; then use the random mask mask' and the SR training data set obtained in step 1 to train the super-resolution model. After the training is completed, the super-resolution model outputs the reconstructed image. Among them, the EDSR model combined with the characteristics of the degraded area is used as the super-resolution model.

2. The method for super-resolution image reconstruction based on regional degradation perception according to claim 1, characterized in that: In step 1, the construction of the Dse training data set for the degraded region estimator is as follows: a clean low-resolution image Clean_LR of uniform size is cropped from the clean low-resolution data set, and then a degraded low-resolution image Degraded_LR is obtained through a degradation operation, and then each clean low-resolution image Clean_LR is paired with its corresponding degraded low-resolution image Degraded_LR to form several data pairs, and the data set composed of all the data pairs is used as the training data set for the degraded region estimator; The SR training dataset for the super-resolution model is constructed as follows: the high-resolution images HR and low-resolution images LR of the real-world super-resolution dataset are cropped to generate several HR image blocks of uniform size and several LR image blocks of uniform size, and each HR image block and its corresponding LR image block are ensured to come from the same position; each HR image block is then bicubicly downsampled to obtain several clean low-resolution image blocks Clean_LR'; the HR image blocks, the corresponding LR image blocks and the corresponding clean low-resolution image blocks Clean_LR' are then paired one by one to form several data pairs, and the dataset consisting of all the data pairs is used as the training dataset for the super-resolution model.

3. The method for super-resolution image reconstruction based on regional degradation perception according to claim 2, characterized in that: In step 1, the clean low-resolution dataset uses the DIV2K low-resolution dataset; the real-world super-resolution dataset uses the RealSR_V3 dataset; the size of the HR image block is four times that of the LR image block; the degradation operations include Gaussian blur, adding noise, and JPEG compression.

4. The method for super-resolution image reconstruction based on regional degradation perception according to claim 1, characterized in that: In step 2, the degradation region estimator is composed of a shallow feature extraction module, a deep feature extraction module and a global feature extraction module connected in sequence; The shallow feature extraction module consists of a convolutional layer; The deep feature extraction module consists of five residual groups connected sequentially, a residual connection, and a convolutional layer; each residual group consists of four convolutional layers; The global feature extraction module consists of five ViTs connected sequentially; In step 2, the mask image encoder consists of five ViTs connected sequentially.

5. The method for super-resolution image reconstruction based on regional degradation perception according to claim 1, characterized in that: In step 2, the specific process of using the random mask generator to generate a random mask is as follows: first, initialize an all-zero mask matrix for storing the mask; then randomly generate N curve starting points on the mask matrix, and then generate N curves of different lengths through the curve generation function, and then draw a circle with this point as the center and a radius of 3 to the maximum circular radius for each point on the curve; finally, use a Gaussian filter to smooth the edge of the mask, and the generated smoothed mask matrix is ​​the random mask; the random mask is used to simulate the degradation mode of different regions; the mask value of 0 represents a non-degraded region, and the mask value of 1 represents a degraded region; The specific operation of the curve generation function is as follows: given a curve starting point, the current position is set as the starting point, a moving direction is randomly selected from the left, lower left, lower, lower right and right directions, one step is moved in the moving direction, that is, it is set to 1, and then the current position is set as the new moving position; at each movement, whether it exceeds the boundary of the mask matrix is ​​determined; if it exceeds, it stays where it is and directly enters the next loop; if it does not exceed, it continues to move; the above operations are repeated until the number of cycles is equal to the maximum length of the curve, and then the movement stops.

6. The method for super-resolution image reconstruction based on regional degradation perception according to claim 1, characterized in that: In step 2, the specific steps of training the degraded region estimator and the mask image encoder are: using a random mask mask to generate a low-quality image LQ; then inputting the low-quality image LQ into the degraded region estimator to extract the degraded region features; at the same time, inputting the random mask mask into the mask image encoder to obtain the mask image features; then performing a matrix multiplication operation on the degraded region features and the mask image features; finally, using the loss function to shorten the distance of matching features and lengthen the distance of non-matching features, thereby optimizing the parameters of the degraded region estimator and the mask image encoder; Among them, the generation of low-quality image LQ is as follows: LQ=mask*degraded_lr+(1-mask)*clean_lr (1) In formula (1), Degraded_LR is the degraded low-resolution image, and Clean_LR is the clean low-resolution image.

7. The method for super-resolution image reconstruction based on regional degradation perception according to claim 6, characterized in that: In step 2, the degraded region features are obtained as follows: the low-quality image LQ is subjected to feature extraction by the shallow feature extraction module of the degraded region estimator, and the shallow feature F is output. SF ; Then the shallow feature F SF The first residual group of the deep feature extraction module transmitted to the degradation region estimator is connected to the residual of the deep feature extraction module; then the shallow feature F SF After the residual groups are connected sequentially, the sum of the input and output of the previous residual group is used as the input of the next residual group; then the output of the last residual group is combined with the shallow feature F at the residual connection. SF Add element by element; then enhance the features through the convolution layer to obtain the deep features F DF ; Finally, the deep feature F DF The image is input into the global feature extraction module of the degradation region estimator and processed by five layers of ViT to obtain the degradation region features of the image. Among them, the calculation formula of the deep feature extraction module is: In formula (2), F SF Indicates shallow features; H i represents the i-th residual group, i = 1 to 5; H Conv Represents the weight parameter of a convolutional layer; F DF Indicates deep features; F i represents the result of adding the input and output of the i-th residual group; The calculation formula for each residual group is: In formula (3), Conv h represents the convolutional layer; F i and F i+1 represents the input and output of the i+1th residual group; x head Conv h The output result; ReLU6 represents the ReLU6 activation function; f1 represents the convolutional layer; f2 represents the convolutional layer; Conv g Represents a convolutional layer.

8. The method for super-resolution image reconstruction based on regional degradation perception according to claim 1, characterized in that: In step 3, the super-resolution model is composed of the degraded region estimator, the gate module Gate, the linear layer Fc, the dynamic network branch DynamicNet and the EDSR model after training obtained in step 2; the EDSR model is composed of a shallow feature extraction module, a deep feature extraction module and an upsampling module connected in sequence; The calculation formula of the EDSR model combined with the characteristics of the degraded area is as follows: In formula (4), Conv_first represents the shallow feature extraction module of EDSR; x first represents the output of Conv_first; Dse represents the degraded region estimator; Fc represents the linear layer; Repeat represents the operation of replicating the vector of shape [batchsize,64,1,1] in both height and width dimensions; x dse represents the processing result of the low-quality image LQ′ after Dse, Fc, and Repeat in sequence; x fd Represents x first and x dse The addition result of ; Gate represents the gate control module; gate represents the output result of Gate; DynamicNet represents the dynamic network branch; x dyn Represents the output result of DynamicNet; Conv_body represents the deep feature extraction module of EDSR; x body Represents deep features; res represents x body 、gate[0]*x dyn and x fd The sum of the three; Upsample is the upsampling module of EDSR; hr represents the output result of Upsample.

9. The method for super-resolution image reconstruction based on regional degradation perception according to claim 1, characterized in that: In step 3, the process of training the super-resolution model is: using mask mask' to generate a low-quality image LQ', and then inputting the low-quality image LQ' into the super-resolution model; in the super-resolution model, the low-quality image LQ' passes through a shallow feature extraction module to obtain shallow features x first ; At the same time, the trained degradation region estimator extracts the low-quality image LQ' into a dimension N dim Characteristics of degraded areas; Then the dimension is N dim The degraded region feature is input to the gate module Gate to obtain gate; at the same time, a learnable linear layer is used to obtain the dimension N dim The degraded region feature is transformed into 64 dimensions, and then replicated in the height and width dimensions to obtain the new degraded region feature x dse ; Then the shallow feature x first With the new degenerate region feature x dse The added results are input into the deep feature extraction module of EDSR and the dynamic network branch DynamicNet to obtain the deep features x body and branch feature x dyn ; Then branch feature x dyn Multiply with the first element of gate followed by shallow feature x first and deep features x body Add them together to get the added result res; finally, the added result res is input into the up-sampling module to get the reconstructed image; Among them, the generation of low-quality image LQ' is as follows: LQ′=mask′*LR+(1-mask′)*clean_lr′(5) In formula (5), LR represents the low-resolution image; Clean_LR' represents the clean low-resolution image block; The calculation formula of the EDSR model is as follows: The gate control module Gate contains a three-layer fully connected network, which is used to transform the input dimension into N dim The degraded region features are gradually reduced in dimension; Among them, the dimension transformation of the degraded region features is as follows: x dse =Repeat(Fc(Dse(LQ′)))(7).

10. The method for super-resolution image reconstruction based on regional degradation perception according to claim 9, characterized in that: In step 3, the dynamic network branch DynamicNet consists of a convolutional layer with a kernel size of 3×3, a convolutional layer with a kernel size of 1×1, a convolutional layer with a kernel size of 3×3, a convolutional layer with a kernel size of 1×1, and a convolutional layer with a kernel size of 3×3, each convolutional layer uses an adaptive activation function; the formula of the adaptive activation function is as follows: x act =x*Sigmoid(Conv(x))(8) In formula (8), Conv represents the convolutional layer; Sigmoid is an activation function; * represents element-by-element multiplication; x act Represents the output result after the input x uses the adaptive activation function.

Images