An image blind super-resolution network model, method, device and storage medium

Abstract

This invention discloses an image blind super-resolution network model, method, device, and storage medium. The image blind super-resolution network model includes a degenerate representation and initial feature extraction sub-network (DRIFENet), a feature fusion transformation sub-network (FFTNet), and an image reconstruction sub-network (IRNet). DRIFENet performs degenerate representation encoding and initial feature map extraction from the input LR image. FFTNet obtains high-value features that fuse spatial local information and degenerate representation information through cascaded feature transformation modules. IRNet uses the transformed high-value features to complete the super-resolution reconstruction of the image. The image blind super-resolution network model disclosed in this invention utilizes degenerate representation information while introducing spatial local information, and incorporates a feature fusion mechanism in the reconstruction unit module. It achieves the best reconstruction effect among several comparison algorithms and achieves a good balance between image reconstruction effect and running time.

Description

Technical Field

[0001] This invention belongs to the field of image blind super-resolution technology, specifically relating to an image blind super-resolution network model, method, device and storage medium. Background Technology

[0002] Super-resolution (SR) image reconstruction refers to the technique of reconstructing a corresponding high-resolution (HR) image from a low-resolution (LR) image. This technique has received widespread attention due to its significant applications in aerospace, medical imaging, and image communication. Image super-resolution reconstruction technology has evolved through methods based on interpolation, reconstruction, shallow neural networks, and deep convolutional neural networks (CNNs). Among these, CNN-based methods offer better image representation and reconstruction capabilities and are currently the most active research direction. Deep CNN-based image super-resolution methods typically assume that the image degradation model is known. When the actual degradation model of the image differs from the degradation model assumed by the deep CNN, the performance of image super-resolution reconstruction significantly degrades. To address this issue, researchers have proposed blind image super-resolution reconstruction techniques.

[0003] Blind super-resolution image reconstruction refers to the super-resolution reconstruction of low-resolution images under the condition that the degradation model of the image is unknown. For blind super-resolution image reconstruction, Kai Zhang et al. used several degradation models (including multiple Gaussian kernels and various noises) as the entire training data when training a deep CNN model. This model has stronger adaptability than a single assumed degradation model. However, when the actual degradation model is not included in the degradation models of the training data, the super-resolution reconstruction performance is still poor. To improve the performance of blind super-resolution image reconstruction, researchers have proposed the idea of ​​estimating degradation models. Jinjin Gu et al. proposed an iterative method for estimating degradation models, which can accurately estimate the degradation model of the image and has high blind super-resolution performance. However, this method uses multiple iterations to estimate the degradation model, resulting in a long algorithm runtime, which cannot meet the needs of practical applications. To improve the efficiency of blind super-resolution image reconstruction, Longguang Wang et al. proposed a blind super-resolution image reconstruction method based on degradation representation learning, which has high performance and efficiency in blind super-resolution image reconstruction. However, this method does not utilize the spatial local information of the image during blind super-resolution reconstruction, which limits further improvement in super-resolution reconstruction performance. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides an image blind super-resolution network model that, while utilizing degenerate representation information, introduces spatial local information to enhance high-value information in the image blind super-resolution process, thereby improving the model's super-resolution reconstruction performance.

[0005] The technical solution of this invention is: an image blind super-resolution network model, comprising:

[0006] The degradation expression and initial feature extraction subnetwork is used to obtain the degradation expression level R and initial feature F0 of the low-resolution LR image through a convolutional network, and output them to the feature fusion transformation subnetwork;

[0007] The feature fusion transformation subnetwork is used to learn the degradation information and spatial local information of the LR image based on the degradation expression level R and the initial feature F0, extract high-value features that fuse spatial local information and degradation expression information, and output the extracted high-value features to the image reconstruction subnetwork.

[0008] The image reconstruction subnetwork is used to reconstruct the extracted high-value features through a convolutional network to obtain the reconstructed high-resolution HR image.

[0009] Furthermore, the degradation expression and initial feature extraction subnetwork includes a degradation expression branch, which inputs the LR image into a 6-layer convolutional network to obtain the degradation expression level R of the LR image; and an initial feature branch, which inputs the LR image into a single-layer convolutional network to obtain the initial feature F0 of the LR image.

[0010] Furthermore, the feature fusion transformation subnetwork includes N cascaded feature transformation modules, each feature transformation module consisting of an SLDC cascade block and a feature fusion block, where N is a positive integer; the i-th SLDC cascade block, where i is a positive integer between 1 and N, is used to receive the degraded expression R and the input features of this stage. Output features are obtained by learning from degradation information and spatial local information. And output to the feature fusion block of this level, wherein the input features of the first SLDC cascade block The Represented as In the formula, H(·) represents the model of the SLDC cascade block;

[0011] The i-th feature fusion block, where i is a positive integer between 1 and N, includes a feature fusion layer and a convolutional layer, used to combine the initial feature F0 and the output features of the current SLDC cascade block. And the outputs of the i-1 feature fusion blocks preceding this level [F1, F2, ..., F i-1 The features are then fused, and then passed through a convolutional layer to obtain the output F of the i-th feature fusion block.i The F i Input features of the (i+1)th SLDC cascade block The F i Represented as In the formula This represents the parameters of the convolutional layer of the i-th feature fusion block, and * indicates the convolution operation;

[0012] The output F of the Nth feature fusion block N After passing through a convolutional layer, a residual learning model is constructed by the identity mapping between the adder and the initial feature F0, resulting in the output F of the feature fusion transformation subnetwork. FFTNet The F FFTNet Represented as F FFTNet =f FFTNet (F0)+F0, where f FFTNet (·) represents the feature fusion transformation subnetwork model.

[0013] Furthermore, the image reconstruction subnetwork includes: a subpixel convolutional layer and a convolutional network layer, and the feature fusion transform subnetwork outputs F. FFTNet The image Y is obtained by sequentially passing through a subpixel convolutional layer and a convolutional network layer to obtain a high-resolution reconstructed HR image. SR The Y SR Represented as Y SR =f IRNet (F FFTNet ), where f IRNet (·) represents the image reconstruction subnetwork model.

[0014] Furthermore, each of the SLDC cascaded blocks includes: n cascaded SLDC blocks, and each SLDC block includes a spatial local information learning branch, a first degradation feature information learning branch, and a second degradation feature information learning branch;

[0015] The spatial local information learning branch sequentially passes the input feature U through the first convolutional layer, the ReLU activation function, the second convolutional layer, and the sigmoid activation function to obtain a spatial local feature map s. Then, the input feature U is multiplied by the spatial local feature map s to obtain the modulated feature U1.

[0016] The first degenerate feature information learning branch passes the initial degenerate expression R through two fully connected layers and structural transformations to obtain the parameters of the third and fourth convolutional layers. Then, the feature U passes through the third and fourth convolutional layers to obtain the feature U2 after degenerate expression modulation.

[0017] The second degradation feature information learning branch passes the degradation expression R through two fully connected layers and a sigmoid activation function to obtain the channel modulation component v. Then, the channel modulation component v is multiplied by the input feature U by channel to obtain the degradation expression modulated feature U3.

[0018] After feature aggregation, features U1, U2, and U3 are transformed by the fifth convolutional layer and aggregated with the residual branch to obtain the output feature U of the SLDC block. out .

[0019] Furthermore, the spatial local feature map s is represented as In the formula, ρ(·) represents the sigmoid activation function, δ(·) represents the ReLU activation function, * represents the convolution operation, and W s 1 and W represents the parameters of the first convolutional layer. s 2 and Indicates the parameters of the second convolutional layer;

[0020] The feature U1 is represented as U1=f(U,s), where f(·) represents the element-wise multiplication of the local feature map s with the input feature U;

[0021] The feature U2 is represented as In the formula and This represents the parameters of the third convolutional layer. and This represents the parameters of the fourth convolutional layer, and * represents the convolution operation;

[0022] The feature U3 is expressed as U3=g(U,v), where g(·) represents the element-wise multiplication of the channel adjustment component v with the input feature U;

[0023] The feature U out Represented as U out =W SLDC *[U1+U2+U3]+b SLDC +U, where W SLDC b SLDC This represents the parameters of the fifth convolutional layer, and * represents the convolution operation.

[0024] This invention also provides a blind super-resolution image method, which inputs a low-resolution LR image into any of the blind super-resolution image network models in the above embodiments, and obtains a high-resolution HR image according to the following steps: S1, obtain the degradation expression R and initial features F0 of the low-resolution LR image through a convolutional network;

[0025] S2. Based on the degradation expression level R and the initial feature F0, learn the degradation information and spatial local information of the LR image, and extract high-value features that integrate spatial local information and degradation expression information;

[0026] S3. The extracted high-value features are reconstructed using a convolutional network to obtain a reconstructed high-resolution HR image.

[0027] This invention also provides an electronic device that uses the method described in the above embodiments to achieve blind super-resolution of images.

[0028] This invention also provides a computer storage medium storing at least one program instruction, which is loaded and executed by a processor to implement the image blind super-resolution method described in the above embodiments.

[0029] The various embodiments of the present invention have the following beneficial effects: While utilizing degraded representation information, the present invention introduces spatial local information, enhancing high-value information in the blind super-resolution process of images. Furthermore, a feature fusion mechanism is introduced in the reconstruction unit module to improve the fusion capability of feature information at different levels. Through the cascading of SLDC cascade blocks and feature fusion blocks, as well as the front-end to back-end fusion link, not only are the learned high-value features and image degradation information strengthened, but the information learned at the front end is also preserved, resulting in high-value information learning ability and multi-level information fusion capability. Experimental results show that the present invention improves the PSNR value under different degradation model conditions and has high reconstruction efficiency, recovering more image details in the visual effect comparison of reconstructed images. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1(a) is a schematic diagram of the overall structure of the network model in an embodiment of the present invention.

[0032] Figure 1(b) is a schematic diagram of the feature transformation module in the feature fusion transformation subnetwork of the network model in the embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the SLDC block structure in the network model of an embodiment of the present invention.

[0034] Figure 3This is a comparison chart showing the reconstruction effects of the network model of this invention and other algorithm models on "zebra" in Set14. Detailed Implementation

[0035] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1: This invention proposes a blind super-resolution network model for images, specifically a deep convolutional neural network model based on spatial local information and degradation construction for blind super-resolution image reconstruction. This model is abbreviated as SLDCN (Spanish Local and Degradation Construction Network). SLDCN uses degradation representation information while further introducing spatial local construction information, and employs a feature fusion mechanism in its unit modules to enhance the propagation of high-value features within the network. The structure of the deep convolutional neural network model based on spatial local information and degradation construction (SLDCN) is shown in Figure 1(a). This model consists of three sequentially connected parts: a degradation representation and initial feature extraction sub-network (DRIFENet), a feature fusion transformation sub-network (FFTNet), and an image reconstruction sub-network (IRNet). The SLDCN model consists of three sub-networks: DRIFENet, which encodes degenerate representations and extracts initial feature maps from the input LR image; FFTNet, which uses cascaded feature transformation modules to obtain high-value features that fuse spatial local information and degenerate representation information; and IRNet, which uses the transformed high-value features to perform super-resolution reconstruction of the image. The SLDCN model will be described in detail below, consisting of three parts: the degenerate representation and initial feature extraction sub-network (DRIFENet), the feature fusion transformation sub-network (FFTNet), and the image reconstruction sub-network (IRNet).

[0037] Degradation Representation and Initial Feature Extraction Subnetwork (DRIFENet): The DRIFENet subnetwork consists of two parts: degradation representation and initial feature extraction. To improve the training efficiency of the model, we borrow Wang's method (Longguang Wang, Yingqian Wang, et al. Unsupervised degradation representation learning for blind super-resolution[C] / / Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition.2021:10576-10585), instead of estimating the accurate degradation information of the LR image at the pixel level, we use a 6-layer convolutional network to obtain the degradation representation R of the LR image. This representation, together with the initial feature F0 generated by the convolutional network of the LR image, is fed into the FFTNet subnetwork for feature transformation.

[0038] Feature Fusion Transformer Subnetwork (FFTNet): The FFTNet subnetwork consists of N cascaded feature transformation modules. These modules are formed by cascading Spatial Local and Degradation Construction (SLDC) blocks and feature fusion blocks, resulting in high-value extraction of initial features from the LR image and learning of degradation information. The structure diagram of the SLDC cascaded blocks and feature fusion blocks in the FFTNet subnetwork, i.e., the structure diagram of the feature transformation modules, is shown in Figure 1(b). Each feature transformation module consists of one SLDC cascaded block and one feature fusion block.

[0039] The SLDC cascaded block consists of n cascaded SLDC blocks. The SLDC blocks autonomously learn spatial information from image features, improving the model's focus on high-value features. Additionally, the model rapidly and efficiently learns degradation information in LR images to achieve blind super-resolution. The structure diagram of the SLDC block is shown below. Figure 2 As shown, Figure 2 In this context, "Conv" represents a convolutional layer, and "ReLU" and "sigmoid" represent activation operations. Let the features sent from the previous SLDC block to the current SLDC block be U = [u1, u2, ..., u...]. cThe feature map contains C feature maps of size H×W. To enable the model to learn high-value spatial distribution information and improve its ability to reconstruct high-frequency details, the idea of ​​the local computation model in HU (Channel-wise and spatial feature modulation network for single image super-resolution[J].IEEE Transactions on Circuits and Systems for Video Technology, vol.30, no.11, pp.3911-3927, Nov.2020) is used to construct a high-value spatial local information learning branch. This branch contains a first convolutional layer, a ReLU activation function, a second convolutional layer, and a sigmoid activation function connected in sequence. Through the convolutional network and activation function, the spatial local feature map s = R is obtained. 1×H×W This process can be represented by the following formula:

[0040]

[0041] In the above formula, ρ(·) represents the sigmoid activation function, δ(·) represents the ReLU activation function, * represents the convolution operation, and W s 1 and W represents the parameters of the first convolutional layer. s 2 and Let represent the parameters of the second convolutional layer. The modulated feature U1 is obtained by modulating the input feature U using this spatial local feature map s, and is expressed by the following formula:

[0042] U1=f(U,s) (2)

[0043] In the above formula, f(·) represents the element-wise multiplication of the local feature map s with the input feature U.

[0044] To enable the model to quickly learn image degradation information, two degradation feature learning branches were constructed. The first degradation feature learning branch sequentially passes the initial degradation expression R through two fully connected layers and a structural transformation to obtain the parameters of the third and fourth convolutional layers. Then, the feature U is sequentially passed through the third and fourth convolutional layers to obtain the degradation expression-modulated feature U2, expressed by the following formula:

[0045]

[0046] In the above formula, and This represents the parameters of the third convolutional layer. and This represents the parameters of the fourth convolutional layer, and * represents the convolution operation.

[0047] The second branch for learning degraded feature information passes the degraded expression R through two fully connected layers and a sigmoid activation function to obtain a channel modulation component v. Then, the channel modulation component v is multiplied by the input feature U by channel to obtain the degraded expression modulated feature U3, which is expressed by the following formula:

[0048] U3=g(U,v) (4)

[0049] In the above formula, g(·) represents the element-wise multiplication of the channel adjustment component v with the input feature U.

[0050] Features U1, U2, and U3, after feature aggregation, are transformed by the fifth convolutional layer and aggregated with the residual branch to obtain the output feature U of the SLDC block. out , can be represented as:

[0051] U out =W SLDC *[U1+U2+U3]+b SLDC +U (5)

[0052] In the above formula, W SLDC b SLDC This represents the parameters of the fifth convolutional layer, and * represents the convolution operation.

[0053] Since each feature transformation module consists of an SLDC cascade block and a feature fusion block, and the FFTNet subnetwork includes N cascaded feature transformation modules, the FFTNet subnetwork also includes N SLDC cascade blocks and N feature fusion blocks.

[0054] Let R and ... be the inputs of the i-th SLDC cascade block (i is a positive integer and 1 ≤ i ≤ N) in the FFTNet subnetwork. As shown in Figure 1(b), the output of the nth SLDC block is obtained after n cascaded SLDC blocks. This can be expressed by the formula:

[0055]

[0056] In the above formula, H(·) represents the model of the SLDC cascade block, and the input features of the first SLDC cascade block are...

[0057] Output of the nth SLDC block The output of the fusion block with the preceding i-1 features [F1, F2, ..., F] i-1The initial feature F0 is concatenated along the channel domain in the feature fusion layer of the feature fusion block, and then passed through a convolutional layer to obtain the output F of the i-th feature fusion block. i , can be represented as:

[0058]

[0059] In the above formula This represents the parameters of the convolutional layer of the i-th feature fusion block, where * denotes the convolution operation. The output F of the i-th feature fusion block... i Input features of the (i+1)th SLDC cascade block The output F of the Nth feature fusion block N After passing through a convolutional layer, a residual learning model is constructed by the identity mapping between the adder and the initial feature F0, resulting in the output F of the feature fusion transformation subnetwork. FFTNet , can be represented as:

[0060] F FFTNet =f FFTNet (F0)+F0 (8)

[0061] In the above formula, f FFTNet (·) represents the feature fusion transformation sub-network model. Through the cascading of SLDC blocks and feature transformation modules, the high-value spatial local information and image degradation information learned by the model are enhanced. Through the fusion of feature fusion blocks and the front-end to back-end fusion links, the model not only learns the feature information of its own module but also retains the information learned by the front-end modules.

[0062] Image Reconstruction Subnetwork (IRNet): IRNet consists of one subpixel convolutional layer and one convolutional network layer, yielding a high-resolution reconstructed HR image, which can be represented as:

[0063] Y SR =f IRNet (F FFTNet (9)

[0064] In the above formula, Y SR f represents the output high-resolution reconstructed image. IRNet (·) represents the image reconstruction subnetwork model.

[0065] Example 2: The present invention also provides a blind super-resolution image method, specifically, which involves inputting a low-resolution LR image into the blind super-resolution network model of the above embodiment, and obtaining a high-resolution HR image according to the following steps:

[0066] S1. Obtain the degraded expression level R and initial features F0 of the low-resolution LR image through a convolutional network;

[0067] S2. Based on the degradation expression level R and the initial feature F0, learn the degradation information and spatial local information of the LR image, and extract high-value features that integrate spatial local information and degradation expression information;

[0068] S3. The extracted high-value features are reconstructed using a convolutional network to obtain a reconstructed high-resolution HR image.

[0069] Experiments and Analysis:

[0070] To verify the effectiveness of the image blind super-resolution network model in this embodiment of the invention, images from different scenes were selected as the test set. The experimental results of several algorithms, including RCAN, SRMD, MZSR, IKC, and DASR, were compared and analyzed with those of this invention.

[0071] We used 800 images from the DIV2K dataset as the training dataset for the model proposed in this invention, and used three benchmark datasets—Set5, Set14, and B100—as the test datasets for model effectiveness. Peak signal-to-noise ratio (PSNR) was used as the evaluation metric for image reconstruction quality; a higher PSNR value indicates better reconstruction results.

[0072] The LR images used in the experiment were obtained by convolving HR images with a Gaussian kernel, followed by bicubic interpolation downsampling and noise addition. The Gaussian kernel size was set to 21×21. When the sampling factor was ×2, the Gaussian kernel parameter σ was selected from [0.2, 2.0], and when the sampling factor was ×4, the Gaussian kernel parameter σ was selected from [0.2, 3.6], and the Gaussian noise variance was selected from [0, 15]. Thirty-two HR images were randomly selected from the DIV2K dataset and augmented using random flipping and rotation transformations. The 32 Gaussian kernel parameters and Gaussian noise parameters were randomly rotated to generate LR images corresponding to the HR images. Two 48×48 LR image blocks were randomly cropped from each LR image, resulting in a total of 64 pairs of LR and HR image blocks that constituted the training set of the model.

[0073] During model training, the Adam algorithm was used to optimize the model, with hyperparameters β1, β2, and ε set to 0.9, 0.999, and 10, respectively. -8The degradation representation level R was obtained according to the method described in Wang (Longguang Wang, Yingqian Wang, et al. Unsupervised degradation representation learning for blind super-resolution[C] / / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2021:10576-10585). The model was iterated 1000 times, and the learning rate was initialized to 10. -4 The learning rate is halved every 250 iterations. The model is trained using the L1 loss function. LR images are first denoised using the DnCNN algorithm before training and testing.

[0074] Objective Quantitative Results and Analysis: We compared the PSNR values ​​of the SLDCN model of this invention with those of the RCAN, SRMD, MZSR, IKC, and DASR algorithms on three test datasets: Set5, Set14, and B100, at sampling factors of ×2 and ×4. The MZSR model was only used for super-resolution with a sampling factor of ×2, the IKC model was only used for super-resolution with a sampling factor of ×4, and the SRMD and MZSR models used the degradation prediction module from the IKC model. The comparison results are shown in Tables 1 and 2.

[0075] Table 1. Comparison of PSNR values ​​for different algorithm models when the sampling factor is ×2.

[0076]

[0077]

[0078] Table 2 shows the comparison of PSNR values ​​for different algorithm models when the sampling factor is ×4.

[0079]

[0080] As shown in Tables 1 and 2, when the Gaussian kernel width is set to 0, the degradation is bicubic degradation, and the RCAN model exhibits the best super-resolution performance in this case. When the Gaussian kernel width is not 0, the performance of the RCAN model decreases significantly, indicating that the model has low adaptability to different degradation types. SRMD+ degradation prediction, MZSR+ degradation prediction, IKC, and DASR models all possess degradation prediction capabilities and exhibit some adaptability to different image degradation types. The SLDCN model of this invention combines spatial local information and degradation prediction in its construction. It demonstrates higher PSNR values ​​in super-resolution reconstructions with Gaussian kernel widths of 0.8 and 1.6 × 2, and in super-resolution reconstructions with Gaussian kernel widths of 1.6 and 3.2 × 4, indicating that the model of this invention has superior blind super-resolution performance.

[0081] Model Running Time Comparison: Table 3 shows the average running time of the SLDCN algorithm model of this invention compared with several other algorithms including RCAN, SRMD, MZSR, IKC, and DASR. As can be seen from the table, the SRMD algorithm has the highest running efficiency, but it lacks blind super-resolution reconstruction capability. When the image degradation model is inconsistent with the algorithm's actual degradation model, the algorithm's super-resolution reconstruction capability is significantly reduced. The IKC algorithm has the lowest running efficiency because it iteratively estimates the image degradation model. The SLDCN algorithm of this invention has a lower running efficiency than the DASR algorithm, but significantly higher than the RCAN algorithm.

[0082] Table 3 Average running time of different algorithm models

[0083] RCAN SRMD MZSR IKC DASR SLDCN(ours) time 168ms 8ms 92ms 521ms 75ms 87ms

[0084] Visual Effect Comparison: This invention compares the blind super-resolution performance of several algorithm models in the "zebra" image of the Set14 test dataset at a noise level of 5, a Gaussian kernel width of 1.6, and a sampling factor of ×4. The visual effects are as follows: Figure 3 As shown in the figure, RCAN's super-resolution performance is poor because the image degradation model is not a bicubic degradation model. The degradation models estimated by SRMD and IKC algorithms deviate significantly from the actual degradation models, and their super-resolution performance is also unsatisfactory. The algorithm of this invention, while utilizing degradation representation information, introduces spatial local information and incorporates a feature fusion mechanism in the reconstruction unit module. Among several algorithms, it achieves the best reconstruction effect and a good balance between image reconstruction performance and running time.

[0085] If the image blind super-resolution method described in this embodiment of the invention is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the image reconstruction method described in this embodiment of the invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, external hard drives, ROM, RAM, magnetic disks, or optical disks.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An image blind super-resolution network model, characterized in that, include: A degraded expression and initial feature extraction subnetwork is used to obtain the degraded expression level of low-resolution LR images through a convolutional network. and initial features And output to the feature fusion transformation subnetwork; Feature fusion transformation subnetwork, used to adjust based on the degraded expression level and initial features The degradation information and spatial local information of the LR image are learned, and high-value features that integrate spatial local information and degradation expression information are extracted. The extracted high-value features are then output to the image reconstruction sub-network. The image reconstruction subnetwork is used to reconstruct the extracted high-value features through a convolutional network to obtain the reconstructed high-resolution HR image; The feature fusion transformation subnetwork includes, N cascaded feature transformation modules, each of which consists of an SLDC cascade block and a feature fusion block, where N is a positive integer, and the SLDC cascade block is a spatial local and degradation construction cascade block; The i-th SLDC cascade block, where i is a positive integer between 1 and N, is used to receive the degenerate expression level. and the input features of this level Output features are obtained through learning from degradation information and spatial local information. The input features of the first SLDC cascade block are then output to the feature fusion block at this level. = The Represented as In the formula A model representing an SLDC cascade block; The i-th feature fusion block, where i is a positive integer between 1 and N, includes a feature fusion layer and a convolutional layer, used to fuse the initial features. Output characteristics of this level SLDC cascade block And the output of the i-1 feature fusion blocks preceding this level. The features are fused and then passed through a convolutional layer to obtain the output of the i-th feature fusion block. The Input features of the (i+1)th SLDC cascade block ; The Represented as In the formula , This represents the parameters of the convolutional layer for the i-th feature fusion block. This represents the convolution operation; The output of the Nth feature fusion block After passing through a convolutional layer, the initial features are added by an adder. The identity mappings constitute the residual learning model, and the output of the feature fusion transformation subnetwork is obtained. ; The Represented as In the formula This represents the feature fusion transformation subnetwork model.

2. The image blind super-resolution network model according to claim 1, characterized in that, The degraded expression and initial feature extraction subnetwork includes, The degradation expression branch inputs the LR image into a 6-layer convolutional network to obtain the degradation expression level of the LR image. ; The initial feature branch inputs the LR image into a single-layer convolutional network to obtain the initial features of the LR image. .

3. The image blind super-resolution network model according to claim 1, characterized in that, The image reconstruction subnetwork includes: One subpixel convolutional layer and one convolutional network layer, with the feature fusion transformation subnetwork output. The image is then passed through a subpixel convolutional layer and a convolutional network layer in sequence to obtain a high-resolution reconstructed HR image. The Represented as In the formula This represents the image reconstruction subnetwork model.

4. The image blind super-resolution network model according to claim 1, characterized in that, Each of the SLDC cascade blocks includes: n cascaded SLDC blocks, wherein the SLDC blocks are used to achieve autonomous learning of spatial information in image features, improve the model’s attention to high-value features, and complete the fast and efficient learning of LR image degradation information. Each SLDC block includes a spatial local information learning branch, a first degradation feature information learning branch, and a second degradation feature information learning branch. The spatial local information learning branch will input features Spatial local feature maps are obtained by sequentially passing the first convolutional layer, the ReLU activation function, the second convolutional layer, and the sigmoid activation function. Then utilize this spatial local feature map For input features The modulated features are obtained after multiplication. ; The first degradation feature information learning branch will initialize the degradation expression level. After passing through two fully connected layers and structural transformations, the parameters of the third and fourth convolutional layers are obtained. Then, the features are... The features obtained after degradation expression modulation are obtained by sequentially passing through the third and fourth convolutional layers. ; The second degradation feature information learning branch will learn the degradation expression level. After passing through two fully connected layers and a sigmoid activation function, the channel modulation component is obtained. Then the channel adjustment component Input features The degenerate expression modulated features are obtained by multiplying by the channels. ; The features ,feature ,feature After feature aggregation, the output features of the SLDC block are obtained by further transformation through the fifth convolutional layer and aggregation with the residual branches. .

5. The image blind super-resolution network model according to claim 4, characterized in that, The spatial local feature map Represented as In the formula This represents the sigmoid activation function. Represents the ReLU activation function. This represents the convolution operation. and This represents the parameters of the first convolutional layer. and Indicates the parameters of the second convolutional layer; The features Represented as In the formula Representing local feature maps Input features Element-by-element multiplication; The features Represented as In the formula and This represents the parameters of the third convolutional layer. and This represents the parameters of the fourth convolutional layer. This represents the convolution operation; The features Represented as In the formula Indicates channel adjustment component Input features Element-by-element multiplication; The features Represented as In the formula , This represents the parameters of the fifth convolutional layer. This represents the convolution operation.

6. A blind super-resolution method for images, characterized in that, Input the low-resolution LR image into the image blind super-resolution network model as described in any one of claims 1-5, and obtain the high-resolution HR image by following these steps: S1. Obtaining the degraded expression levels of low-resolution LR images through convolutional networks. and initial features ; S2, based on the described degraded expression level and initial features Learning from the degradation information and spatial local information of the LR image, high-value features that integrate spatial local information and degradation representation information are extracted; S3. The extracted high-value features are reconstructed using a convolutional network to obtain a reconstructed high-resolution HR image.

7. An electronic device, characterized in that, Blind super-resolution of images is achieved using the method described in claim 6.

8. A computer storage medium, characterized in that, The storage medium stores at least one program instruction, which is loaded and executed by a processor to implement the image blind super-resolution method as described in claim 6.

Images