Multi-scale dense connection super-resolution reconstruction method based on random degradation

Abstract

The invention provides a multi-scale dense connection super-resolution reconstruction method based on random degradation. The method mainly comprises a multiple random degradation model and an image super-resolution reconstruction network. The multiple random degradation model carries out parameter random and sequence random degradation operations on high-resolution images for multiple times, and then low-resolution images are obtained to form a training data set. The image super-resolution reconstruction network is composed of a shallow feature extraction module, a multi-scale feature extraction module, a deep feature extraction module and an up-sampling module, and training is carried out by using a data set obtained through preprocessing of multiple random degradation models, so that the reconstruction performance of the network in a real scene is improved. Meanwhile, a discriminator is introduced to optimize the model in adversarial learning. According to the method, the reconstruction effect of an image super-resolution reconstruction algorithm in a real scene is improved, and the method has good practical value.

Description

Technical Field

[0001] This application relates to the technical field of image super-resolution reconstruction, and particularly to a multi-scale dense connection super-resolution reconstruction method based on random degradation. Background Art

[0002] Vision is the most information-rich form of human perception, usually carried by images. With the rapid development of information technology and the popularization of intelligent devices, people's requirements for image quality are constantly increasing. Image clarity directly affects the effectiveness of information transmission, and the resolution of the image is a core indicator among them. Low-resolution images can only present basic information such as the outlines of things, while high-resolution images, with a larger pixel density and finer textures, can carry more detailed features. To break through the resolution limit, the image super-resolution reconstruction technology has emerged. This technology restores high-resolution visual information from low-resolution images by reversely learning the prior knowledge of the image degradation process.

[0003] Currently, most image super-resolution reconstruction studies use synthetic datasets for training and have achieved good results on known degradation datasets. However, the degradation process of low-resolution images in synthetic datasets is single and overly ideal, and cannot fully cover the image degradation in real scenarios. Models trained on synthetic datasets often perform poorly when dealing with real low-resolution images. To address this problem, in order to better apply the image super-resolution reconstruction technology to real scenarios, it is necessary to study the relatively single degradation process in synthetic datasets and design corresponding image super-resolution reconstruction algorithms. Summary of the Invention

[0004] This application provides a multi-scale dense connection super-resolution reconstruction method based on random degradation, which can be used to solve the technical problem that most existing reconstruction algorithms have poor generalization ability in real scenarios.

[0005] Compared with the prior art, the present invention has the following remarkable advantages: 1. A multiple random degradation model is designed to simulate the image degradation process in real scenarios, so that the training dataset can cover as many image degradation processes as possible, providing more effective prior information for the reconstruction network and improving the generalization ability and robustness of the model in practical applications. 2. A multi-scale feature fusion module is proposed to make full use of structural information. Three parallel convolutions of 3*3, 5*5, and 7*7 are used to extract feature information from different scales, enhancing the network's ability to express contour details. 3. An SE-RRDB module is designed using the idea of residual dense connection, and a channel attention mechanism is introduced to enable the network to assign different weight values β to different channels when learning feature maps, improving the network's sensitivity to important features and enhancing the network's feature expression ability. Brief Description of the Drawings

[0006] Figure 1 Flowchart of the multi-scale dense connection super-resolution reconstruction algorithm based on random degradation provided for this application.

[0007] Figure 2 General network structure diagram provided for this application.

[0008] Figure 3 Structural diagram of the random degradation model provided for this application.

[0009] Figure 4 Structural diagram of the image super-resolution reconstruction network provided for this application.

[0010] Figure 5 Structural diagram of the multi-scale feature extraction module provided for this application.

[0011] Figure 6 Structural diagram of the SE-RRDB module provided for this application.

[0012] Figure 7 Structural diagram of the discriminator module provided for this application. Detailed implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe the implementation manners of this application in detail with reference to the accompanying drawings.

[0014] The following will first introduce the embodiments of this application with reference to the accompanying drawings.

[0015] Combined with Figure 1 , the present invention designs a multi-scale dense connection super-resolution reconstruction method based on random degradation. The method specifically includes two parts: a multiple random degradation model and an image super-resolution reconstruction network. For a high-resolution image, first use the multiple random degradation model for degradation operations. The obtained low-resolution image will sequentially pass through a 3*3 shallow feature extraction module, a multi-scale dense connection module, a deep feature extraction module, and an upsampling module to obtain the final reconstructed image.

[0016] The network structure of the multi-scale dense connection super-resolution reconstruction method based on random degradation is as Figure 2 shown, and the method includes the following steps:

[0017] Step A, as Figure 3 shown, for the high-resolution image I HR , by randomly shuffling various known degradation factors, simulate the degradation process of the low-resolution image in the real scene, obtain the low-resolution image with a corresponding downsampling factor of 4, and then form a paired data set for network training; the multiple random degradation model is divided into two parts: a degradation library and a random shuffling strategy, including:

[0018] Step A1: For the high-resolution image I HR Applying ideal degradation operations is a common method for synthesizing training datasets. The ideal degradation operations are as follows:

[0019]

[0020] where I HR is the high-resolution image, is a convolution operation used to perform a convolution operation with a blur kernel k on the high-resolution image I HR ↓, and s is a downsampling operation, and n is a noise addition operation; is the low-resolution image obtained after the ideal degradation operation. From the ideal degradation operation, it can be seen that the three key factors affecting image quality are blur, noise, and downsampling.

[0021] To enable the training dataset to cover more degradation spaces and better cope with the degradation process in real scenarios, a degradation library is built, which includes blur, noise, and downsampling operations, and a compression operation is additionally introduced. At the same time, these four degradation operations are made as rich as possible to maximize the practicality of the model. Among them, the blur operation selects Gaussian blur B Gaussion and motion blur B Motion , the noise operation selects Gaussian noise N Gaussian , Poisson noise N Poisson , and salt-and-pepper noise N Pepper , the downsampling operation selects bilinear interpolation and bicubic interpolation to achieve the purpose of 4-fold downsampling, and the compression operation selects JPEG compression.

[0022] In terms of setting degradation parameters, for the blur kernel setting, the Gaussian blur kernel follows an isotropic Gaussian distribution, and the blur kernel size is randomly selected from {5×5, 7×7,..., 23×23}, and the value range of σ is from 0.2 to 5.5; the rotation angle range of the motion blur kernel is from 0 to 180°; the downsampling operation randomly selects bilinear interpolation and bicubic interpolation to achieve the purpose of 4-fold downsampling; for the noise intensity setting, the variance of Gaussian noise N Gaussian is low noise when it is in [0.01, 0.1], medium noise when it is in [0.1, 0.5], and high noise level when it is in [0.5, 1.0], and the selected applied noise is between [0.01, 0.5]; the intensity of Poisson noise N Poisson is related to the image illumination intensity, and λ is randomly selected from 0 to 125; the noise density of salt-and-pepper noise N Pepper is randomly set to 1%, 5%, and 10%; the compression index of compression noise N Jpeg is randomly selected from 10 to 80;

[0023] Step A2: Use the random shuffling strategy to randomly select and sort the parameters of multiple degradation factors in a random order, and superimpose multiple degradations to randomly permute and combine different degradation processes. This greatly expands the degradation space of the degradation model, and the ideal degradation operation is only a special case in the random degradation model. In addition, the random degradation model may simultaneously include multiple blur kernels and noises to simulate the degradation situation in the real scene. At the same time, the blur kernel and downsampling operation can change the characteristics of the noise, so the diversity of degradation factors is increased; the combined random degradation processes include but are not limited to:

[0024]

[0025] where, I HR is the high-resolution image, is the convolution operation, which is used to perform the convolution operation with the blur kernel k on the high-resolution image I HR , ↓ s is the downsampling operation, n is the noise addition operation, is the low-resolution image obtained through the ideal degradation operation;

[0026] Considering that overly complex multiple degradations will randomly generate many degradation processes that do not exist in the real scene, resulting in a sharp increase in the number of model parameters and thus bringing pressure to model training. Select a two-fold degradation operation, where each layer of the degradation process is independent and random, and under the constraint of the downsampling operation, a 4-fold downsampling operation is achieved;

[0027] Step B, as Figure 4 shown, construct the overall image super-resolution reconstruction network, including a shallow feature extraction module, a multi-scale feature extraction module, a deep feature extraction module, and an upsampling module; use a 3*3 convolution to perform shallow feature extraction on the low-resolution image, and then use parallel convolutions of 3*3, 5*5, and 7*7 to extract feature information of different scales. The deep feature extraction module uses 23 improved SE-RRDB modules for feature mapping, and finally uses two sub-pixel convolutions as the upsampling module to generate the final reconstructed image; including:

[0028] Step B1: For a low-resolution image I LR , first use a 3*3 convolution to extract the shallow features of the low-resolution image in the low-dimensional space, reduce the network parameters and adjust the number of channels of the feature map, as follows:

[0029] X1 = Conv 3×3 (I LR )

[0030] Among them, X1 represents the shallow features obtained after the low-resolution image passes through the shallow feature extraction module, Conv(·) represents the convolution operation, and I LR is the input low-resolution image;

[0031] Step B2: As Figure 5 shown, the feature X1 is obtained through the shallow feature extraction module, and then the multi-scale feature fusion module is used to extract feature information of different dimensions: parallel convolutions of 3*3, 5*5, and 7*7 are used to extract features, and after splicing and adjustment of the feature dimensions, the feature output X2 is obtained. The method is as follows:

[0032] m1 = f(Conv 3×3 (Conv 3×3 (X1)))

[0033] m2 = f(Conv 5×5 (Conv 5×5 (X1)))

[0034] m3 = f(Conv 7×7 (Conv 7×7 (X1)))

[0035] E1 = Conv 1×1 (Concat(m1, m2, m3))

[0036] X2 = X1 + E1 + E2

[0037] Among them, Conv(·) represents the convolution operation, f(·) represents the Leaky ReLU operation, Concat(·) is used to splice features of different scales, X1 is the input of the multi-scale feature fusion module, X2 is the output of the multi-scale feature fusion module, and E1 and E2 are intermediate features;

[0038] Step B3: As Figure 6 shown, a deep feature extraction module composed of 23 SE-RRDBs is used for feature mapping to enhance the learning ability of the network; the SE-RRDB module is improved on the basis of the Residual in Residual Dense Block (RRDB). Each RRDB includes three Residual Dense Blocks (RDBs), and each Residual Dense Block RDB contains five convolutional layers and four Leaky ReLU activation layers; the overall structure of the RRDB is divided into three levels of residual connections: dense connection, local residual connection, and residual nested structure;

[0039] Among them, for the densely connected RDB, in the traditional convolutional neural network, each layer is only connected to its directly preceding and succeeding layers. In the RDB, each layer in the network not only receives the input from the previous layer but also simultaneously receives the outputs of all the previous layers as additional inputs, and finally, an additional convolutional layer is used to adjust the number of channels of the feature map. The formula is as follows:

[0040] F i = H i ([F i-1 , F i-2 , …, F0])

[0041] Among them, H i represents the i-th layer convolutional operation, and [·] represents channel concatenation.

[0042] Local residual connection: The input and output of each RDB are added together, and one branch needs to be scaled by a weight factor to alleviate gradient vanishing and stabilize training. The formula is as follows:

[0043] RDB out = RDB in + β × H RDB (RDB in )

[0044] Among them, RDB in is the input of the densely connected block RDB, HRDB is the RDB operation, β is the weight factor, usually set to 0.2, and RDB out is the output of the local residual connection.

[0045] Residual nested structure: 3 RDB modules are cascaded to form an RRDB module, and the output of each RDB module is directly passed to the subsequent layer through a skip connection to obtain the final output. The formula is as follows:

[0046] RRDB out = RRDB in + β × (RDB3(RDB2(RDB1(RRDB in ))))

[0047] Among them, RRDB in is the input feature, that is, X2 obtained by the multi-scale feature extraction module, β is the weight factor, RDB(·) is the densely connected operation, and RRDB out is the output feature;

[0048] When determining the SE-RRDB module, by introducing the channel attention mechanism (Squeeze-and-Excitation Networks, SE), different weight values β are assigned to different channels to improve the network's sensitivity to important features, better capture and emphasize high-frequency information, and thus optimize the overall performance.

[0049] The channel attention mechanism can re-weight the importance of each channel by aggregating global information. Specifically, for a feature map of dimension C×H×W as input, the feature vector F to which the channel attention mechanism is finally applied att , goes through three steps:

[0050] Step B31, given a feature map F ∈ R C×H×W , a global average pooling operation that compresses the spatial dimension of each channel into a scalar value, with the formula as follows:

[0051]

[0052] where C is the number of channels, H and W are the height and width of the feature map respectively, and F GAP is the scalar value of each channel obtained after the global average pooling operation; this step effectively summarizes the spatial information of the entire feature map and generates a statistical description of each channel.

[0053] Step B32, use two consecutive fully connected layers or 1x1 convolutional layers, with a non-linear activation function inserted in the middle; the task of the first fully connected layer is to reduce the feature dimension to C / r, and the second is used to restore to the original number of channels C, with the formula as follows:

[0054] F1 = W2(δ(W1(F GAP )))

[0055] where W1 and W2 represent the weight matrices of the first and second layers respectively, δ is the ReLU activation function, and r is the scaling ratio, set to 16; this step not only reduces the computational complexity but also enhances the expressive power through the bottleneck structure.

[0056] Step B33, the output is processed by the Sigmoid activation function to produce a weight vector in the range [0,1], representing the degree of importance of each channel, with the formula as follows:

[0057] M c = σ(F1)

[0058] where σ represents the Sigmoid operation; the weight vector will be applied to the original feature map, and channel re-weighting is achieved through element-wise multiplication, with the formula as follows:

[0059] Fatt = M c ⊙ F

[0060] where ⊙ represents element-wise multiplication;

[0061] Finally, the RRDB module with channel attention mechanism enables the network to assign different weight values β to different channels when learning feature maps, rather than using fixed and uniform weights; the formula is as follows:

[0062] RDB out = RDB in + SE out × H RDB (RDB in )

[0063] RRDB out = RRDB in + SE out × (RDB3(RDB2(RDB1(RRDB in ))))

[0064] where SE out represents the weight factor assigned by the channel attention mechanism to different feature information;

[0065] Step B4: Complete the upsampling operation through two sub-pixel convolutions and generate the final RGB reconstructed image through a 3*3 convolution.

[0066] Step C, as Figure 7 shown, use the PatchGAN discriminator to discriminate the reconstructed image and optimize the network using the generative adversarial idea;

[0067] Using the PatchGAN discriminator makes the model pay more attention to the local structural consistency rather than the global consistency of the whole image, thereby improving the quality of the reconstructed image, especially in terms of texture details and local features. The discriminator network is divided into six layers. The first five layers all contain a convolutional layer and an activation function, and the parameter is set to 0.2; except for the first layer without using the BN layer, the remaining four layers all use the BN layer; finally, after a convolutional operation, an N*N feature map is obtained to judge the authenticity of each small patch.

[0068] Step D, use the weighted adversarial loss, the mean absolute value loss L1 and the perceptual loss for supervision, and then generate an image that is more in line with the human eye perception effect; including:

[0069] Step D1: Obtain the adversarial loss: The training processes of the reconstruction network G and the discriminator network D are described as a process of solving the extreme value of a binary function. The formula is as follows:

[0070]

[0071] Among them, represents the adversarial loss, D(Y) is the discriminator's judgment result on the real image, and D(G(X)) is the discriminator's judgment result on the reconstructed image;

[0072] The discriminator D continuously discriminates the quality of the generated image under the prior information of the input HR image, while the generator G continuously improves its own generation quality and tries to deceive the discriminator network as much as possible. In this game, the two force the network to continuously obtain better reconstruction results; for the discriminator, when calculating the discriminator loss, the training parameters of the current generator will be fixed, as shown in the formula:

[0073]

[0074] Among them, represents the discriminator loss, D(Y) is the discriminator's judgment result on the real image, and D(G(X)) is the discriminator's judgment result on the reconstructed image;

[0075] For the generator, when calculating the generator loss, the training parameters of the current discriminator will be fixed, and the formula is as follows:

[0076]

[0077] Among them, represents the generator loss, and D(G(X)) is the discriminator's judgment result on the reconstructed image;

[0078] Step D2: Obtain the pixel loss: The pixel loss refers to calculating the error between the predicted image and the target image I at each pixel position and summing up the errors into a scalar value; Use loss to calculate the similarity at the pixel level of the image. For an image with size H×W×C, the pixel loss formula is as follows:

[0079]

[0080] Among them, H is the height, W is the width, and C is the number of channels;

[0081] Step D3: Obtain the perceptual loss: The role of the perceptual loss is to improve the quality of the generated image by capturing the high-level semantic information of the image. Send the high-resolution image I HR and the reconstructed image I SR into the pre-trained VGG-19 network, and use the high-level feature maps extracted by the network to calculate the reconstructed image I SR and the high-resolution image I HRThe difference between them, that is, calculating the Euclidean distance of these intermediate features, is used to measure the gap at the semantic level between the generated image and the real image;

[0082] Suppose there is a pre-trained convolutional neural network φ that can map the input image to feature representations at multiple different levels; the perceptual loss formula is as follows:

[0083]

[0084] Among them, φ l represents the feature extractor of the l-th layer of the network, C l , H l , W l are the number of channels, height, and width of the feature map of the corresponding layer respectively, φ l (I SR ) and φ l (I HR ) are the feature representations of the reconstructed image I SR and the high-resolution image I HR at the l-th layer respectively;

[0085] Step D4: The overall loss formula of the entire network is as follows:

[0086]

[0087] Among them, represents the adversarial loss, represents the pixel loss, represents the perceptual loss, α = 0.001, δ = 1.0, μ = 0.05.

[0088] The present invention first uses a multiple random degradation model to process high-resolution images to obtain a paired dataset for training. Then, the low-resolution image sequentially passes through a shallow feature extraction module, a multi-scale feature extraction module, a deep feature extraction module, and an upsampling module to obtain a reconstructed image. In addition, a PatchGAN discriminator is used to optimize the generation quality of the image. During the network training process, the adversarial loss, pixel loss, and perceptual loss are used to constrain the network to enable the network to reconstruct an image with more perfect details. The design of this application is reasonable and easy to implement, and it can be effectively applied to the image super-resolution reconstruction task under real scenarios, improving the image quality of the reconstructed image. It can be applied to special image fields such as medical images and remote sensing images, the security monitoring field, and the small target detection field, etc., and has good practical value.

[0089] The above-described embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A multi-scale densely connected super-resolution reconstruction method based on random degradation, characterized in that: The method comprises: Step A, use multiple random degradation models to degrade the high-resolution image to obtain a data set for network training: HR , by randomly shuffling various known degradation factors, the degradation process of low-resolution images in real scenes is simulated, and the corresponding low-resolution images with a downsampling factor of 4 are obtained, which then form a paired data set for network training; the multiple random degradation model is divided into two parts: the degradation library and the random shuffling strategy; Step B, constructing an overall image super-resolution reconstruction network; the image super-resolution reconstruction network includes a shallow feature extraction module, a multi-scale feature extraction module, a deep feature extraction module and an upsampling module; a 3*3 convolution is used to perform shallow feature extraction on the low-resolution image, and then 3*3, 5*5 and 7*7 parallel convolutions are used to extract feature information of different scales. The deep feature extraction module uses 23 improved SE-RRDB modules for feature mapping, and finally two sub-pixel convolutions are used as upsampling modules to generate the final reconstructed image; Step C: Use the PatchGAN discriminator to discriminate the reconstructed image and optimize the network using the generative adversarial idea; In step D, weighted adversarial loss, mean absolute value loss L1 and perceptual loss are used for supervision to generate images that are more consistent with human eye perception.

2. The method according to claim 1, characterized in that: Step A, using multiple random degradation models to degrade the high-resolution image, thereby obtaining a data set for network training, including: Step A1: For high-resolution image I HR Apply the ideal degradation operation as follows: Among them, I HR For high-resolution images, is a convolution operation used for high-resolution image I HR Perform a convolution operation with blur kernel k,↓ s is the downsampling operation, and n is the noise adding operation; is a low-resolution image obtained after an ideal degradation operation; Build a degradation library, including blur, noise and downsampling operations, and introduce additional compression operations; select Gaussian blur for blur operation B Gaussion and motion blur B Motion , the noise operation selects Gaussian noise N Gaussian , Poisson noise N Poisson and salt and pepper noise N Pepper , bilinear interpolation is selected for downsampling operation and bicubic interpolation To achieve the purpose of 4 times downsampling, the compression operation selects JPEG compression; In terms of degradation parameter settings, for blur kernel settings, the Gaussian blur kernel obeys isotropic Gaussian distribution, the blur kernel size is randomly selected from {5×5, 7×7, …, 23×23}, and the value range of σ is 0.2 to 5.5; the rotation angle range of the motion blur kernel is 0 to 180°; the downsampling operation randomly selects bilinear interpolation and bicubic interpolation to achieve 4 times downsampling; for noise intensity settings, Gaussian noise N Gaussian The variance of is low noise in [0.01, 0.1], medium noise in [0.1, 0.5], and high noise level in [0.5, 1.0]. The noise applied is selected to be between [0.01, 0.5]; Poisson noise N Poisson The intensity of the noise is related to the image illumination intensity, and λ is randomly selected between 0 and 125; the salt and pepper noise N Pepper The noise density is randomly set to 1%, 5% and 10%; the compression noise N Jpeg The compression index is randomly selected from 10 to 80; Step A2: Use random shuffling strategy to account for multiple degradation factors The parameters are randomly selected and sorted, and multiple degradations are superimposed to randomly arrange and combine different degradation processes. The blur kernel and downsampling operations can change the characteristics of the noise, thereby increasing the diversity of degradation factors. The combined random degradation processes include: Among them, I HR For high-resolution images, is a convolution operation used for high-resolution image I HR Perform a convolution operation with blur kernel k,↓ s is the downsampling operation, n is the noise adding operation, is a low-resolution image obtained after an ideal degradation operation; A double degradation operation is selected, and the degradation process of each layer is independent and random. Under the constraint of the downsampling operation, a 4-fold downsampling operation is achieved.

3. The method according to claim 1, characterized in that: Step B, constructing the overall image super-resolution reconstruction network, including: Step B1: For a low-resolution image I LR , first use a 3*3 convolution to extract shallow features of low-resolution images in low-dimensional space, reduce network parameters and adjust the number of channels of feature maps, as follows: X1=Conv 3×3 (I LR ) Among them, X1 represents the shallow features obtained after the low-resolution image passes through the shallow feature extraction module, Conv(·) represents the convolution operation, and I LR is the input low-resolution image; Step B2: The feature X1 is obtained through the shallow feature extraction module, and then the feature information of different dimensions is extracted through the multi-scale feature fusion module: 3*3, 5*5 and 7*7 parallel convolutions are used to extract features, and after splicing and feature dimension adjustment, the feature output X2 is obtained. The method is as follows: m1=f(Conv 3×3 (Conv 3×3 (X1))) m2=f(Conv 5×5 (Conv 5×5 (X1))) m3=f(Conv 7×7 (Conv 7×7 (X1))) E1=Conv 1×1 (Concat(m1,m2,m3)) X2=X1+E1+E2 Among them, Conv(·) represents the convolution operation, f(·) represents the Leaky ReLU operation, Concat(·) is used to concatenate features of different scales, X1 is the input of the multi-scale feature fusion module, X2 is the output of the multi-scale feature fusion module, and E1 and E2 are intermediate features; Step B3: Use a deep feature extraction module consisting of 23 SE-RRDBs to perform feature mapping to enhance the learning ability of the network; the SE-RRDB module is improved on the basis of the residual in residual dense block (RRDB), each RRDB includes three residual dense blocks (RDB), each residual dense block RDB contains five convolutional layers and four leaky ReLU activation layers; the overall structure of RRDB is divided into three levels of residual connection: dense connection, local residual connection, and residual nested structure; Each layer in the densely connected RDB network not only receives input from the previous layer, but also receives the output of all previous layers as additional input, and finally adjusts the number of channels of the feature map through an additional convolutional layer; the formula is as follows: F i =H i ([F i-1 ,F i-2 ,…,F0]) Among them, H i represents the i-th layer convolution operation, [·] represents channel concatenation; Local residual connection, the input and output of each RDB are added, and one branch needs to be scaled by a weight factor to alleviate the gradient disappearance and stabilize the training; the formula is as follows: RDB out =RDB in +β×H RDB (RDB in ) Among them, RDB in is the input of the densely connected block RDB, H RDB is an RDB operation, β is a weight factor, usually set to 0.2, RDB out is the output of the local residual connection; The residual nested structure uses three RDB modules to form an RRDB module through cascading. The output of each RDB module is directly passed to the subsequent layer through skip connection to obtain the final output; the formula is as follows: RRDB out =RRDB in +β×(RDB3(RDB2(RDB1(RRDB in )))) Among them, RRDB in is the input feature, i.e., X2 obtained by the multi-scale feature extraction module, β is the weight factor, RDB(·) is the dense connection operation, and RRDB out is the output feature; When determining the SE-RRDB module, different weight values ​​β are assigned to different channels by introducing the channel attention mechanism (Squeeze-and-Excitation Networks, SE); Step B4: The upsampling operation is completed through two sub-pixel convolutions, and the final RGB reconstructed image is generated through a 3*3 convolution.

4. The method according to claim 3, characterized in that: For a feature map of the input C×H×W dimension, the feature vector F with the channel attention mechanism applied at the end att , after three steps: Step B31, given a feature map F∈R C×H×W , the global average pooling operation compresses the spatial dimension of each channel into a scalar value, the formula is as follows: Where C is the number of channels, H and W are the height and width of the feature map, respectively, and F GAP is the scalar value of each channel obtained after the global average pooling operation; Step B32, use two consecutive fully connected layers or 1x1 convolutional layers with a nonlinear activation function inserted in between; the task of the first fully connected layer is to reduce the feature dimension to C / r, and the second one is used to restore to the original number of channels C, the formula is as follows: F1=W2(δ(W1(F GAP ))) Where W1 and W2 represent the weight matrices of the first and second layers respectively, δ is the ReLU activation function, and r is the scaling factor, which is set to 16; Step B33: The output is processed by the Sigmoid activation function to generate a weight vector in the interval [0,1], which represents the importance of each channel. The formula is as follows: M c =σ(F1) Among them, σ represents the Sigmoid operation; the weight vector will be applied to the original feature map, and the channel will be re-weighted by element-by-element multiplication. The formula is as follows: F att =M c ⊙F Among them, ⊙ represents element-by-element multiplication; Finally, the RRDB module with the channel attention mechanism enables the network to assign different weight values ​​β to different channels when learning feature maps, instead of using fixed, uniform weights; the formula is as follows: <h2 style=";text-align:left;direction:ltr">RDB<h2 style=";text-align:left;direction:ltr"> out <h2 style=";text-align:left;direction:ltr"> <RDB<h2 style=";text-align:left;direction:ltr"> in <h2 style=";text-align:left;direction:ltr"> +SE<h2 style=";text-align:left;direction:ltr"> out <h2 style=";text-align:left;direction:ltr"> ×H<h2 style=";text-align:left;direction:ltr"> RDB <h2 style=";text-align:left;direction:ltr"> (RDB<h2 style=";text-align:left;direction:ltr"> in <h2 style=";text-align:left;direction:ltr"> ) RRDB out =RRDB in +IF out ×(RDB3(RDB2(RDB1(RDB in )))) Among them, SE out Represents the weight factor assigned by the channel attention mechanism to different feature information.

5. The method according to claim 1, characterized in that: Step C, PatchGAN discriminator The discriminator network is divided into six layers. The first five layers contain a convolution layer and an activation function with the parameter set to 0.

2. Except for the first layer, the other four layers all use the BN layer. Finally, a convolution operation is performed to obtain an N*N feature map to judge the authenticity of each small patch.

6. The method according to claim 1, characterized in that: Step D: Use weighted adversarial loss, mean absolute value loss L1 and perceptual loss for supervision to generate images that are more consistent with human eye perception. include: Step D1: Obtaining adversarial loss: The training process of the reconstruction network G and the identification network D is described as the process of solving the extreme value of a binary function. The formula is as follows: in, represents adversarial loss, D(Y) is the discriminant’s result on the real image, and D(G(X)) is the discriminant’s result on the reconstructed image; The discriminator D continuously identifies the quality of the generated image under the prior information of the input HR image, while the generator G continuously improves its own generation quality and tries to deceive the discriminator network as much as possible. In this game, the two force the network to continuously obtain better reconstruction results. For the discriminator, when calculating the discriminator loss, the training parameters of the current generator are fixed, as shown in the formula: in, represents the discriminator loss, D(Y) is the discriminator's judgment result on the real image, and D(G(X)) is the discriminator's judgment result on the reconstructed image; For the generator, when calculating the generator loss, the training parameters of the current discriminator are fixed, and the formula is as follows: in, represents the generator loss, D(G(X)) is the discriminator’s decision on the reconstructed image; Step D2: Get pixel loss: Pixel loss refers to calculating the predicted image and the error of the target image I at each pixel position, and summarize the error into a scalar value; use Loss is used to calculate the similarity of the image pixel level. For an image of size H×W×C, The pixel loss formula is as follows: Among them, H is the height, W is the width, and C is the number of channels; Step D3: Obtaining perceptual loss: The role of perceptual loss is to improve the quality of the generated image by capturing the high-level semantic information of the image. HR and the reconstructed image I SR The pre-trained VGG-19 network is used to calculate the reconstructed image I using the high-level feature map extracted by the network. SR and high resolution image I HR The difference between them, that is, calculating the Euclidean distance of these intermediate features, to measure the gap between the generated image and the real image at the semantic level; Assume that there is a pre-trained convolutional neural network φ that can map the input image to multiple levels of feature representation; the perceptual loss formula is as follows: Among them, φ l represents the feature extractor of the network layer l, C l , H l , W l are the number of channels, height and width of the feature map of the corresponding layer, φ l (I SR ) and φ l (I HR ) are the reconstructed images I SR and high resolution image I HR Feature representation at layer l; Step D4: The overall loss formula of the entire network is as follows: in, Represents resistance to loss, represents pixel loss, represents the perceptual loss, α=0.001, δ=1.0, μ=0.05.

Images