An image super-resolution reconstruction method based on a lightweight IRN

Abstract

The present application belongs to the technical field of image processing, and particularly relates to an image super-resolution reconstruction method based on a light IRN. The method comprises the following steps: step 1, selecting a training set; step 2, constructing a light IRN network; step 3, taking high-resolution images in the training set as input data, and training the IRN network constructed in step 2; step 4, using the IRN network trained in step 3 to perform super-resolution reconstruction of images, and reversely outputting a super-resolution image. The wavelet domain high-frequency sub-band interpolation is used to improve the latent variables of the model, can save and process the edge values of the LR image, and enhance the embedding high-frequency information capacity. The attention mechanism and the extended convolution are used to design the feature extraction block, reduce the parameter quantity of the model, and improve the attention to the image details during training.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to an image super-resolution reconstruction method based on a lightweight IRN. Background Technology

[0002] Single-image super-resolution reconstruction (SISR) involves obtaining a magnified or original-size high-resolution (HR) image from a scaled-down single low-resolution (LR) image. Because LR images lack many image details, ill-posedness arises when reconstructing super-resolution (SR) images. Therefore, improving ill-posedness is a key research focus and challenge in SISR.

[0003] In 2014, Dong et al. first applied convolutional neural networks to image super-resolution tasks and proposed the SRCNN algorithm. This algorithm reconstructs images through three steps: feature extraction of image blocks, nonlinear mapping, and reconstruction, laying the foundation for the application of convolutional neural networks in image super-resolution.

[0004] Since the introduction of SRCNN, many scholars have focused on super-resolution tasks based on deep learning, continuously proposing new optimization algorithms to improve the performance of reconstructed images. Xiao Jinsheng et al. improved the SRCNN algorithm by adjusting the convolutional kernel size and adding pooling layers, reducing the number of parameters and computational complexity. Gong Lanlan et al., based on SRCNN, increased the number of feature extraction layers and used AdaDelta and SGD methods to optimize network parameters, accelerating the model's training speed. Kim et al. successively introduced residual structures and recurrent neural networks, proposing VDSR and DRCN algorithms respectively, further deepening the network structure, expanding the network's receptive field, and alleviating the gradient vanishing or exploding problem in deep networks. Inspired by generative adversarial networks, Ledig et al. introduced SRGAN to the super-resolution reconstruction problem, proposing adversarial loss and content loss as loss functions, obtaining reconstructed images with more realistic textures. Inspired by invertible networks, Xiao et al. proposed the IRN algorithm, which uniformly models image downscaling and upscaling tasks, captures and embeds lost high-frequency information, is compatible with image downscaling operations, and obtains reconstructed images with good visual quality.

[0005] However, experiments revealed that the latent variables of the IRN model could not preserve the high-frequency information of the LR image, resulting in mediocre embedding of high-frequency information and affecting the model's performance in reconstructing the SR image.

[0006] Based on the component study of the IRN model in this paper, we improve the IRN model in two aspects. First, we improve the latent variables of the model by using wavelet domain high-frequency subband interpolation, which can preserve and process the edge values ​​of LR images and enhance the ability to embed high-frequency information. Second, we use an attention mechanism and extended convolution to design feature extraction blocks, which reduces the number of model parameters and improves the attention to image details during training. Summary of the Invention

[0007] The purpose of this invention is to address the problem that current IRN models cannot preserve high-frequency information of LR images through their latent variables, resulting in mediocre high-frequency information embedding and affecting the performance of the model in reconstructing SR images. This invention provides a lightweight IRN-based image super-resolution reconstruction method, improving IRN in two ways. First, wavelet domain high-frequency subband interpolation is used to improve the model's latent variables, which can preserve and process LR image edge values, enhancing the ability to embed high-frequency information. Second, this invention uses an attention mechanism and extended convolution to design feature extraction blocks, reducing the number of model parameters and improving attention to image details during training.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A lightweight IRN-based image super-resolution reconstruction method includes the following steps:

[0010] Step 1: Select the training set;

[0011] Step 2: Construct a lightweight IRN network;

[0012] Step 3: Use the high-resolution images in the training set as input data to train the IRN network constructed in Step 2;

[0013] Step 4: Use the IRN network trained in Step 3 to perform super-resolution reconstruction of the image.

[0014] Preferably, in step 1, the specific operation of selecting the training set is as follows: using the DIV2K dataset as the training set, the DIV2K dataset contains 800 high-resolution images and corresponding low-resolution images sampled by bicubic interpolation. Random translation, flipping, and cropping operations are performed on the DIV2K training set images to expand the training set to 32593 images, and these images are uniformly 480×480 pixels, serving as the final training set. The high-resolution image samples {P} in the training set... h1 ,P h2 , …P hi , …,P hn} as input data for the network's forward pass, using low-resolution image samples {P l1 ,P l2 , …P li , …,P ln} is used as data for calculating the forward error function of the network.

[0015] Preferably, the specific operations for constructing the IRN network in step 2 include:

[0016] Step 2.1: A wavelet transform and eight identical invertible building blocks are stacked to form a downscaling module;

[0017] Step 2.2: The two downscaling modules from Step 2.1 form a lightweight IRN network, with high-resolution image samples {P} in the training set. h1 ,P h2 ,…P hi , …P hn The first downscaling module outputs a set of downscaled low-resolution feature images, which serves as the input to the second downscaling module. The second downscaling module outputs a set of low-resolution images {Y}. o1 ,Y o2 ,…,Y on}

[0018] Preferably, the specific operations of step 2.1 include:

[0019] Step 2.1.1: Wavelet transform is used as the first layer of the downscaling module to transform the high-resolution image samples P of the training set. hi Decompose:

[0020] (2)

[0021] Where LL: low-frequency component, HH: horizontal low-frequency and vertical high-frequency component, HL: horizontal high-frequency and vertical low-frequency component, HH: diagonal high-frequency component; H and G are the filter coefficient matrices corresponding to the one-dimensional scaling function and the one-dimensional wavelet function, respectively, and HH*+GG*=I, the superscript symbol * indicates the conjugate transpose of the matrix, and the components LL, LH, HL, and HH are concatenated into a tensor X according to formula (2):

[0022] (3)

[0023] Among them, F cat This represents the concatenation function, where 1 in the function indicates concatenation in one dimension.

[0024] Step 2.1.2: Input tensor X into the first reversible building block. The operation formulas for the reversible building block are represented by (3) and (4):

[0025] (4)

[0026] (5)

[0027] In this context, formula (3) represents the forward computation process of the reversible building block, and formula (4) represents the reverse computation process of the reversible building block; k1 and k2 represent the internal coupling computation formulas of the reversible building block, K represents the output of the reversible building block, exp represents the exponential function with the natural constant e as the base, X[:,:3] represents the first 3 dimensions of tensor X, X[:,3:] represents the last 3 dimensions of tensor X, and ⊙ represents the multiplication between tensors. This indicates division between tensors. Represents a feature extraction network, and They are the same;

[0028] Step 2.1.3: Use the output of the first reversible building block as the input of the second building block, and so on, stacking 8 reversible building blocks to form a downscaling module.

[0029] Preferably, the specific operations of the feature extraction network in step 2.1.2 include:

[0030] Step 2.1.2.1: Pass the input tensor X through three densely connected convolutional layers. Each densely connected layer accepts the features of all the preceding layers as input. The operation of the convolutional layers is represented by formulas (5) and (6):

[0031] (6)

[0032] (7)

[0033] F1 represents the output of the first convolutional layer, F i F represents the output of the i-th convolutional layer. i-1 W represents the output of the (i-1)th convolutional layer, where the output of the previous layer serves as the input of the next layer. c(1) W represents the weights of the first convolutional layer. c(i) This represents the weights of the i-th convolutional layer. Each convolutional weight corresponds to a filter bank of size n and a size of 3×3. * represents the convolution operation, b1 represents the bias of the first layer, and b... i σ represents the bias of the i-th layer, and σ represents the leaky ReLU activation function.

[0034] Step 2.1.2.2: Input the output data from Step 2.1.2.1 into the attention mechanism. The operation is as follows: Data F i Input an average pooling layer and a max pooling layer respectively. The operations of the average pooling layer and the max pooling layer are the same as those of the convolutional layer. The parameters of the pooling layer are not updated. As shown in Equation (5), the outputs of the average pooling layer and the max pooling layer are then passed through two convolutional layers and activation layers. The operations are shown in Equations (5) and (6). The output of the previous layer is used as the input of the next layer.

[0035] Step 2.1.2.3: The outputs of the channels on both sides of the average pooling layer and the max pooling layer are concatenated into a tensor by formula (2). The tensor is then added to the input data of step 2.1.2.1 and used as the output of the feature extraction network.

[0036] Preferably, step 3 specifically includes the following operations:

[0037] Step 3.1: Forward Training: Step 2 yields the low-resolution image Y o and low-resolution images P in the training set l The forward error l is calculated using formula (7). forw :

[0038] (8)

[0039] Where: l forw H represents the forward error, H and W represent the height and width of the image, respectively, and i and j represent the pixel coordinates of the image.

[0040] Step 3.2: Low-resolution image Y o Image obtained by upsampling by 2x using bilinear interpolation The bilinear interpolation upsampling operation is represented by equation (8):

[0041] (9)

[0042] f(x,y) represents the coordinates of the unknown point at (x,y), Q 11 (x1,y1),Q 12 (x1,y2),Q 21 (x2,y1),Q 22 (x2, y2) represents the image Y. o Four known pixel coordinates, x and y represent the coordinates of the unknown pixel, f(Q) 11 f(Q) 12 f(Q) 21 f(Q) 22 () represents the pixel value at the pixel coordinate;

[0043] image Formula (1) is decomposed into low-frequency component LL, horizontal low-frequency and vertical high-frequency component HH, horizontal high-frequency and vertical low-frequency component HL, and diagonal high-frequency component HH. The low-frequency component LL and the image The correction factor α is calculated using the formula:

[0044] (10)

[0045] The LH, HL, and HH components are corrected using the correction factor α and formula (10):

[0046] (11)

[0047] The revised The latent variables are obtained by inverse wavelet transform of the components and LL components. The operation is as shown in formula (11):

[0048] (12)

[0049] Step 3.3: Reverse Training: Transferring latent variables Inversely inputting the lightweight IRN network yields a high-resolution image X. sr High-resolution image X sr And the input image P for forward training in step 3.1 h The reverse error l is calculated using formula (12). recon :

[0050] (13)

[0051] l recon H represents the inverse error, and W represents the height and width of the image, respectively. i and j represent the pixel coordinates of the image, respectively.

[0052] Step 3.4: Based on the forward error l calculated in Step 3.1 forw And the inverse error l calculated in step 3.3 recon Calculate the total error l total :

[0053] ,

[0054] Then, the Adam gradient descent method and backpropagation of the network are used to minimize the total error l. total Adjusting network weights, the process of updating network weights is as follows:

[0055] (14)

[0056] (15)

[0057] Δ k This represents the previous weight update value, l represents the layer number, k represents the number of network iterations, and β represents the learning rate. This represents the weights at the k-th iteration of the l-th layer. This represents taking the partial derivative with respect to the corresponding weights in the error function.

[0058] Preferably, step 4 specifically includes: processing the image P that needs to be super-resolution reconstructed. l The image is input to a lightweight IRN network, and the super-resolution image Ps is output in reverse.

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] (1) The present invention improves the latent variables of the model by wavelet domain high-frequency subband interpolation, which can save and process the edge values ​​of LR images and enhance the ability to embed high-frequency information.

[0061] (2) The present invention uses an attention mechanism and an extended convolutional design to extract features, which reduces the number of model parameters and improves the attention to image details during training. Attached Figure Description

[0062] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0063] In the attached diagram:

[0064] Figure 1 This is a flowchart of the method of the present invention;

[0065] Figure 2 A block diagram of a lightweight IRN model;

[0066] Figure 3 This is a diagram of the feature extraction network structure.

[0067] Figure 4 For the latent variable plot of the model;

[0068] Figure 5 Comparison results for boobies: (a) SRCNN method, (b) SRGAN method, (c) EDSR method, (d) IRN method, (e) the method of this invention, (f) real image;

[0069] Figure 6 The comparison results for the buildings are as follows: (a) SRCNN method, (b) SRGAN method, (c) EDSR method, (d) IRN method, (e) the method of this invention, and (f) real image. Detailed Implementation

[0070] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0071] Example:

[0072] See attached document Figure 1-6 As shown, an image super-resolution reconstruction method based on a lightweight IRN is characterized by the following steps:

[0073] Step 1: Select the training set. Specifically, use the DIV2K dataset as the training set. The DIV2K dataset contains 800 high-resolution images and corresponding low-resolution images obtained through bicubic interpolation. Perform random translation, flipping, and cropping operations on the DIV2K training set images to expand the training set to 32,593 images, all uniformly sized to 480×480 pixels. This will serve as the final training set. The high-resolution image samples in the training set {P} h1 ,P h2 , …P hi , …,P hn} as input data for the network's forward feed, using low-resolution image samples {P l1 ,P l2 , …P li , …,P ln} is used as data for calculating the forward error function of the network.

[0074] Step 2: Construct a lightweight IRN network;

[0075] Step 2.1: A wavelet transform and eight identical invertible building blocks are stacked to form a downscaling module;

[0076] Step 2.1.1: Wavelet transform is used as the first layer of the downscaling module to transform the high-resolution image samples P of the training set. hi Decompose:

[0077] (16)

[0078] Where LL: low-frequency component, HH: horizontal low-frequency and vertical high-frequency component, HL: horizontal high-frequency and vertical low-frequency component, HH: diagonal high-frequency component; H and G are the filter coefficient matrices corresponding to the one-dimensional scaling function and the one-dimensional wavelet function, respectively, and HH*+GG*=I, the superscript symbol * indicates the conjugate transpose of the matrix, and the components LL, LH, HL, and HH are concatenated into a tensor X according to formula (2):

[0079] (17)

[0080] Among them, F cat This represents the concatenation function, where 1 in the function indicates concatenation in one dimension.

[0081] Step 2.1.2: Input tensor X into the first reversible building block. The operation formulas for the reversible building block are represented by (3) and (4):

[0082] (18)

[0083] (19)

[0084] In this context, formula (3) represents the forward computation process of the reversible building block, and formula (4) represents the reverse computation process of the reversible building block; k1 and k2 represent the internal coupling computation formulas of the reversible building block, K represents the output of the reversible building block, exp represents the exponential function with the natural constant e as the base, X[:,:3] represents the first 3 dimensions of tensor X, X[:,3:] represents the last 3 dimensions of tensor X, and ⊙ represents the multiplication between tensors. This indicates division between tensors. Represents a feature extraction network, and They are the same;

[0085] like Figure 3 As shown in the feature extraction network structure diagram, the specific operations of the feature extraction network in step 2.1.2 include:

[0086] Step 2.1.2.1: Pass the input tensor X through three densely connected convolutional layers. Each densely connected layer accepts the features of all the preceding layers as input. The operation of the convolutional layers is represented by formulas (5) and (6):

[0087] (20)

[0088] (twenty one)

[0089] F1 represents the output of the first convolutional layer, F i F represents the output of the i-th convolutional layer. i-1 W represents the output of the (i-1)th convolutional layer, where the output of the previous layer serves as the input of the next layer. c(1) W represents the weights of the first convolutional layer. c(i) This represents the weights of the i-th convolutional layer. Each convolutional weight corresponds to a filter bank of size n and a size of 3×3. * represents the convolution operation, b1 represents the bias of the first layer, and b... i σ represents the bias of the i-th layer, and σ represents the leaky ReLU activation function. σ(y) = max(0,a) + leak*min(0,a), where leak is a very small constant.

[0090] Step 2.1.2.2: Input the output data from Step 2.1.2.1 into the attention mechanism. The operation is as follows: Data F i Input an average pooling layer and a max pooling layer respectively. The operations of the average pooling layer and the max pooling layer are the same as those of the convolutional layer. The parameters of the pooling layer are not updated. As shown in Equation (5), the outputs of the average pooling layer and the max pooling layer are then passed through two convolutional layers and activation layers. The operations are shown in Equations (5) and (6). The output of the previous layer is used as the input of the next layer.

[0091] Step 2.1.2.3: The outputs of the channels on both sides of the average pooling layer and the max pooling layer are concatenated into a tensor by formula (2). The tensor is then added to the input data of step 2.1.2.1 and used as the output of the feature extraction network.

[0092] Step 2.1.3: Use the output of the first reversible building block as the input of the second building block, and so on, stacking 8 reversible building blocks to form a downscaling module.

[0093] Step 2.2: The two downscaling modules from Step 2.1 form a lightweight IRN network, with high-resolution image samples {P} in the training set. h1 ,P h2 ,…P hi , …P hn The first downscaling module outputs a set of downscaled low-resolution feature images, which serves as the input to the second downscaling module. The second downscaling module outputs a set of low-resolution images {Y}. o1 ,Y o2 ,…,Y on}

[0094] Step 3: Use the high-resolution images in the training set as input data to train the IRN network constructed in Step 2;

[0095] Step 3.1: Forward Training: Step 2 yields the low-resolution image Y o and low-resolution images P in the training set l The forward error l is calculated using formula (7). forw :

[0096] (twenty two)

[0097] Where: l forw H represents the forward error, H and W represent the height and width of the image, respectively, and i and j represent the pixel coordinates of the image.

[0098] Step 3.2: Low-resolution image Y o Image obtained by upsampling by 2x using bilinear interpolation The bilinear interpolation upsampling operation is represented by equation (8):

[0099] (twenty three)

[0100] f(x,y) represents the coordinates of the unknown point at (x,y), Q 11 (x1,y1),Q 12 (x1,y2),Q 21 (x2,y1),Q 22 (x2, y2) represents the image Y.o Four known pixel coordinates, x and y represent the coordinates of the unknown pixel, f(Q) 11 f(Q) 12 f(Q) 21 f(Q) 22 () represents the pixel value at the pixel coordinate;

[0101] image Formula (1) is decomposed into low-frequency component LL, horizontal low-frequency and vertical high-frequency component HH, horizontal high-frequency and vertical low-frequency component HL, and diagonal high-frequency component HH. The low-frequency component LL and the image The correction factor α is calculated using the formula:

[0102] (twenty four)

[0103] The LH, HL, and HH components are corrected using the correction factor α and formula (10):

[0104] (25)

[0105] The revised The latent variables are obtained by inverse wavelet transform of the components and LL components. The operation is as shown in formula (11):

[0106] (26)

[0107] Step 3.3: Reverse Training: Transferring latent variables Inversely inputting the lightweight IRN network yields a high-resolution image X. sr High-resolution image X sr And the input image P for forward training in step 3.1 h The reverse error l is calculated using formula (12). recon :

[0108] (27)

[0109] l recon H represents the inverse error, and W represents the height and width of the image, respectively. i and j represent the pixel coordinates of the image, respectively.

[0110] Step 3.4: Based on the forward error l calculated in Step 3.1 forw And the inverse error l calculated in step 3.3 recon Calculate the total error l total :

[0111] ,

[0112] Then, the Adam gradient descent method and backpropagation of the network are used to minimize the total error l.total Adjusting network weights, the process of updating network weights is as follows:

[0113] (28)

[0114] (29)

[0115] Δ k This represents the previous weight update value, l represents the layer number, k represents the number of network iterations, and β represents the learning rate. This represents the weights at the k-th iteration of the l-th layer. This represents taking the partial derivative with respect to the corresponding weights in the error function.

[0116] Step 4: Use the IRN network trained in Step 3 to perform super-resolution reconstruction of the image. The image P that needs super-resolution reconstruction... l The image is input to a lightweight IRN network, and the super-resolution image Ps is output in reverse.

[0117] Simulation experiment:

[0118] Experimental parameter settings:

[0119] The experimental platform hardware configuration consisted of an Intel i5 10600KF processor, 8GB of RAM, and an NVIDIA GeForce RTX 2060 graphics card. The experimental environment was Ubuntu 18.04, the deep learning framework was PyTorch, and the CUDA accelerated version was CUDA Toolkit 7.6.5. The forward and backward loss functions of the trained model were L2 and L1, respectively, and the total number of model iterations was 5 × 10⁻⁶. 5 Initial learning rate 2×10 -4 The learning rate decays to [10k, 20k, 30k, 40k] iterations with a decay rate γ=0.5. The network parameters are then updated using the optimizer Adam until the model converges.

[0120] Training process:

[0121] This invention uses the DIV2K dataset as the training set, which contains 800 high-resolution images. Due to the limited number of images in the DIV2K training set and the need to maintain consistency across all experimental training data, this paper expands the training set to 32,593 images by randomly shifting, flipping, and cropping them, and uniformly assigning each image a 480×480 pixel resolution to meet the required training data volume. This paper uses widely used benchmark datasets Set5, Set14, BSD100, and Urban100 as the test set, and qualitatively evaluates the model using Peak Signal-to-Noise Ratio (PSNR) on the RGB channels and Structural Similarity (SSIM).

[0122] Experimental results and analysis:

[0123] The improved algorithm is compared with SRCNN, SRGAN, EDSR, and IRN methods to evaluate its performance. The quality of the reconstructed image is evaluated from two aspects: subjective human visual perception and objective data evaluation. Subjective human visual perception evaluation is based on human visual observation of the image, while objective data evaluation uses current mainstream image quality metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) to assess the image. This invention also calculates the model complexity of each method and compares their performance.

[0124] The objective evaluation results of each algorithm are shown in Tables 1 and 2. From the comparison results of PSNR and SSIM in Tables 1 and 2, it can be seen that the PSNR and SSIM of the proposed method and the IRN method are higher than those of the other methods on each test set. Compared with the IRN method, the PSNR of the proposed method is 0.005 dB higher, and the SSIM is 0.005 dB higher. This indicates that the image reconstruction method of the proposed method has the lowest distortion.

[0125] Table 1. Comparison of PSNR of reconstructed images

[0126]

[0127] Table 2. SSIM Comparison Results of Reconstructed Images

[0128]

[0129] This invention selects a subset of reconstructed images for subjective visual evaluation and comparison. The selected images are all from different scenes to avoid randomness. Figure 5 As shown in 6 and 7. From Figure 5 (a) Figure 6 (a) It can be seen that the reconstructed image by the SRCNN method is the most blurry and has less edge information. Figure 5 (b)(c) Figure 6 (b) and (c) show that EDSR and SRGAN methods produce more image texture details, with SRGAN showing better visual effects and higher clarity. Figure 5 (d)(e) (f), Figure 6 (d), (e), and (f) show that the proposed method and the IRN method have the best image visual effects, and the effects are the same.

[0130] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. An image super-resolution reconstruction method based on a lightweight IRN, characterized in that: Includes the following steps: Step 1: Select the training set; Step 2: Build a lightweight IRN network; The specific steps for building an IRN network include: Step 2.1: A wavelet transform and eight identical invertible building blocks are stacked to form a downscaling module; Step 2.2: The two downscaling modules from Step 2.1 form a lightweight IRN network, with high-resolution image samples {P} in the training set. h1 ,P h2 ,…P hi , …P hn The first downscaling module outputs a set of downscaled low-resolution feature images, which serves as the input to the second downscaling module. The second downscaling module outputs a set of low-resolution images {Y}. o1 ,Y o2 ,…,Y on The specific operations of step 2.1 include: Step 2.1.1: Wavelet transform is used as the first layer of the downscaling module to transform the high-resolution image samples P of the training set. hi Decompose: （1） Where LL: low-frequency component, HH: horizontal low-frequency and vertical high-frequency component, HL: horizontal high-frequency and vertical low-frequency component, HH: diagonal high-frequency component; H and G are the filter coefficient matrices corresponding to the one-dimensional scaling function and the one-dimensional wavelet function, respectively, and HH*+GG*=I, the superscript symbol * indicates the conjugate transpose of the matrix, and the components LL, LH, HL, and HH are concatenated into a tensor X according to formula (2): （2） Among them, F cat This represents the concatenation function, where 1 in the function indicates concatenation in one dimension. Step 2.1.2: Input tensor X into the first reversible building block. The operation formulas for the reversible building block are represented by (3) and (4): （3） （4） In this context, formula (3) represents the forward computation process of the reversible building block, and formula (4) represents the reverse computation process of the reversible building block; k1 and k2 represent the internal coupling computation formulas of the reversible building block, K represents the output of the reversible building block, exp represents the exponential function with the natural constant e as the base, X[:,:3] represents the first 3 dimensions of tensor X, X[:,3:] represents the last 3 dimensions of tensor X, and ⊙ represents the multiplication between tensors. This indicates division between tensors. Represents a feature extraction network, and They are the same; Step 2.1.3: Use the output of the first reversible building block as the input of the second building block, and so on, stacking 8 reversible building blocks to form a downscaling module; Step 3: Use the high-resolution images in the training set as input data to train the IRN network constructed in Step 2; the specific operations of Step 3 include: Step 3.1: Forward Training: Step 2 yields the low-resolution image Y o and low-resolution images P in the training set l The forward error l is calculated using formula (7). forw : （7） Where: l forw H represents the forward error, H and W represent the height and width of the image, respectively, and i and j represent the pixel coordinates of the image. Step 3.2: Low-resolution image Y o Image obtained by upsampling by 2x using bilinear interpolation The bilinear interpolation upsampling operation is represented by equation (8): （8） f(x,y) represents the coordinates of the unknown point at (x,y), Q 11 (x1,y1),Q 12 (x1,y2),Q 21 (x2,y1),Q 22 (x2, y2) represents the image Y. o Four known pixel coordinates, x and y represent the coordinates of the unknown pixel, f(Q) 11 f(Q) 12 f(Q) 21 f(Q) 22 () represents the pixel value at the pixel coordinate; image Formula (1) is decomposed into low-frequency component LL, horizontal low-frequency and vertical high-frequency component HH, horizontal high-frequency and vertical low-frequency component HL, and diagonal high-frequency component HH. The low-frequency component LL and the image The correction factor α is calculated using the formula: （9） The LH, HL, and HH components are corrected using the correction factor α and formula (10): （10） The revised Low-resolution images are obtained by inverse wavelet transform of the LL component and the LL component. The operation is as shown in formula (11): （11） Step 3.3: Reverse Training: Transform the low-resolution image Inversely inputting the lightweight IRN network yields a high-resolution image X. sr High-resolution image X sr And the input image P for forward training in step 3.1 h The reverse error l is calculated using formula (12). recon : （12） l recon H represents the inverse error, and W represents the height and width of the image, respectively. i and j represent the pixel coordinates of the image, respectively. Step 3.4: Based on the forward error l calculated in Step 3.1 forw And the inverse error l calculated in step 3.3 recon Calculate the total error l total : ， Then, the Adam gradient descent method and backpropagation of the network are used to minimize the total error l. total Adjusting network weights, the process of updating network weights is as follows: （13） （14） Δ k This represents the previous weight update value, l represents the layer number, k represents the number of network iterations, and β represents the learning rate. This represents the weights at the k-th iteration of the l-th layer. This represents taking the partial derivative with respect to the corresponding weights in the error function; Step 4: Use the IRN network trained in Step 3 to perform super-resolution reconstruction of the image.

2. The image super-resolution reconstruction method based on a lightweight IRN according to claim 1, characterized in that: In step 1, the specific operation for selecting the training set is as follows: The DIV2K dataset is used as the training set. The DIV2K dataset contains 800 high-resolution images and corresponding low-resolution images sampled using bicubic interpolation. Random translation, flipping, and cropping operations are performed on the DIV2K training set images to expand the training set to 32,593 images, all uniformly sized to 480×480 pixels, which serve as the final training set. The high-resolution image samples in the training set {P} h1 ,P h2 , …P hi , …,P hn } as input data for the network's forward feed, using low-resolution image samples {P l1 ,P l2 , …P li , …,P ln } is used as data for calculating the forward error function of the network.

3. The image super-resolution reconstruction method based on a lightweight IRN according to claim 2, characterized in that: The specific operations of the feature extraction network in step 2.1.2 include: Step 2.1.2.1: Pass the input tensor X through three densely connected convolutional layers. Each densely connected layer accepts the features of all the preceding layers as input. The operation of the convolutional layers is represented by formulas (5) and (6): （5） （6） F1 represents the output of the first convolutional layer, F i F represents the output of the i-th convolutional layer. i-1 W represents the output of the (i-1)th convolutional layer, where the output of the previous layer serves as the input of the next layer. c(1) W represents the weights of the first convolutional layer. c(i) This represents the weights of the i-th convolutional layer. Each convolutional weight corresponds to a filter bank of size n and a size of 3×3. * represents the convolution operation, b1 represents the bias of the first layer, and b... i σ represents the bias of the i-th layer, and σ represents the leaky ReLU activation function. Step 2.1.2.2: Input the output data from Step 2.1.2.1 into the attention mechanism. The operation is as follows: Data F i Input an average pooling layer and a max pooling layer respectively. The operations of the average pooling layer and the max pooling layer are the same as those of the convolutional layer. The parameters of the pooling layer are not updated. As shown in Equation (5), the inputs of the average pooling layer and the max pooling layer are then passed through two convolutional layers and activation layers. The operations are shown in Equations (5) and (6). The output of the previous layer is used as the input of the next layer. Step 2.1.2.3: The outputs of the channels on both sides of the average pooling layer and the max pooling layer are concatenated into a tensor by formula (2). The tensor is then added to the input data of step 2.1.2.1 and used as the output of the feature extraction network.

4. The image super-resolution reconstruction method based on a lightweight IRN according to claim 3, characterized in that: Step 4 specifically includes: processing the image P that needs to be super-resolution reconstructed. l The image is input to a lightweight IRN network, and the super-resolution image Ps is output in reverse.

Images