Super-resolution reconstruction method of remote sensing images based on lightweight generative model

Abstract

The invention belongs to the technical field of image generation, and discloses a remote sensing image super-resolution reconstruction method based on a lightweight generation model. A generation adversarial super-resolution network total model based on remote sensing images is constructed, which includes a generation network and a discriminant network. A super-resolution image closer to a real image is generated through continuous training confrontation between the generation network and the discriminant network. The generation network integrates the saliency mechanism of attention, and redistributes the weights of different channel features, so that the model can adaptively learn the feature distribution of the remote sensing image. The two network model parameters are continuously iterated and optimized through a loss function and an optimizer, so as to achieve a dynamic balance between the two, so that the probability distribution of the generated image is approximately the same as the data distribution of the real image, so as to alleviate the ill-posed problem of image super-resolution reconstruction.

Description

Technical Field

[0001] The invention belongs to the technical field of image generation, relates to a remote sensing image generation method, and in particular to a remote sensing image super-resolution reconstruction method based on a lightweight generation model. Background Art

[0002] Since remote sensing images are mostly imaged at long distances, one pixel can often cover a surface area of ​​several square meters or even thousands of square meters. Therefore, the images obtained have the characteristics of complex ground object types and blurred target elements, which makes it difficult to extract and interpret ground objects in the later stage, resulting in certain limitations in the research and application of images in related fields. Therefore, generating remote sensing images that are close to real images is the key to overcoming this limitation.

[0003] At present, the method of realizing image super-resolution reconstruction is to use deep residual convolution generative adversarial network. Most of the generative networks use the structure of multiple residual modules in series, and the generated images and natural real images are used as the input of the discriminator. Through the continuous training and confrontation between the generative network and the discriminative network, the images generated by the generative network are getting closer and closer to the real images. In addition, there are also convolutional neural network super-resolution reconstruction methods based on different structures. For example, the method based on residual structure reconstructs the high-frequency detail information of the image by learning the residual between the high-resolution image and the stretched and enlarged low-resolution image; the method based on dense structure concatenates the features of each previous layer of the network as the input of the subsequent network, so that the image features are fully transmitted and efficient network training is achieved; the method based on recursive structure learns the residuals of each level between the low-resolution image and the high-resolution image, and then adds them to the original low-resolution image to obtain the corresponding reconstruction results of each level, and performs weighted calculation on this result to obtain the final super-resolution image.

[0004] At present, most of the super-resolution reconstruction methods based on deep residual convolution generative adversarial networks are carried out on the basis of ordinary images, and there are still some challenges in the super-resolution research of remote sensing images. Specifically, remote sensing images are rich in surface information, and the features of ground objects and the texture of target objects are relatively obvious. In addition, since remote sensing images are mostly imaged at long distances and have a large spatial scale, they are prone to problems such as low image resolution and large image deformation. This makes it difficult to directly apply the super-resolution method of ordinary images to remote sensing images. Therefore, for remote sensing images, a new network model needs to be proposed to improve the resolution of images.

[0005] Compared with ordinary images, remote sensing images have different image scales and spatial resolutions, and they have rich texture features and strong autocorrelation. Therefore, it is difficult to reconstruct high-definition remote sensing images. At the same time, from the current research status, the higher the image accuracy, the more network layers there are, and the more difficult it is to train. Therefore, on the basis of ensuring the accuracy of the image, thinking about how to solve the problems of large network parameters and difficult training has also become a major challenge.

[0006] To this end, the present invention is designed from aspects such as establishing a network model, adjusting relevant learning strategies, and rationally planning data sets to accurately extract remote sensing image features, thereby achieving the goal of restoring image edge texture and improving perceived quality. In addition, while ensuring unchanged accuracy, the network structure is improved to reduce the number of parameters and achieve a lightweight design. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention provides a remote sensing image super-resolution reconstruction method based on a lightweight generative model, designs a generative adversarial super-resolution network total model based on remote sensing images, which is composed of a generative network and a discriminative network. Through the continuous training confrontation between the generative network and the discriminative network, a super-resolution image closer to the real image is generated. This generative network integrates the saliency mechanism of attention, named the attention generative network AGN, re-weights the features of different channels, fully considers the interdependence between different channels, and adopts a method of treating different importance levels differently, so that the model can adaptively learn the feature distribution of remote sensing images, and continuously iterates and optimizes the parameters of the two network models through loss functions and optimizers, in order to achieve a dynamic balance between the two, so that the probability distribution of the generated image is approximately the same as the data distribution of the real image, so as to alleviate the ill-posed problem of image super-resolution reconstruction. In addition, the respective advantages of deep separable convolution and residual network are fully utilized to realize image feature learning and reduce network redundancy, realize network lightweight design, and solve the problems of many network layers and large number of parameters in most super-resolution reconstruction methods based on deep learning.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0009] The remote sensing image super-resolution reconstruction method based on the lightweight generation model includes the following steps:

[0010] Step 1: Build a lightweight generative network structure, named Attention Generative Network (AGN). This generative network structure integrates the channel attention mechanism saliency model and replaces the ordinary convolution with the depthwise separable convolution.

[0011] Step 2: Construct a generative adversarial super-resolution network model based on remote sensing images. The model includes a generative network and a discriminative network. Through continuous training and adversarial training between the generative network and the discriminative network, a super-resolution image that is closer to the real image is generated.

[0012] Furthermore, the attention generation network AGN includes a shallow feature extraction layer f1, a channel attention layer f2, a middle feature extraction layer f3, and an upsampling layer f4;

[0013] The attention generation network AGN is represented by f AGN , input a low-resolution image into the network, this low-resolution image is marked as LR, and the output super-resolution image is marked as SR, we can get:

[0014] SR=f AGN (LR)

[0015] After the entire process of the attention generation network AGN, it can be expressed as:

[0016] SR=f4(f3(f2(f1(LR))+f1(LR))+f1(LR))

[0017] Furthermore, the channel attention layer f2 is composed of a CAdepth block, which is composed of a channel attention basic block CA_Block and a Depth_Block composed of two layers of depth-separable convolution structures; the channel attention basic block CA_Block is composed of an average pooling layer, a convolution layer and a Sigmoid function; the input feature passes through the channel attention basic block CA_Block, is scaled to obtain the output, and then the input and output are multiplied to obtain the final output of the channel attention basic block CA_Block.

[0018] Furthermore, the average pooling layer of the channel attention basic block CA_Block obtains the statistics of the spatial feature representation of each channel. The first convolution layer and the second convolution layer use the same size of convolution kernel, but the number of output channels of the first convolution layer is reduced by four times, and the output channels of the second convolution layer are expanded to the original number of channels. The reconstructed channel representation is then input into the Sigmoid function, weights are assigned to different channel features, dependencies between different channels are established, and finally the output is multiplied by the input channel structure to obtain the output.

[0019] Furthermore, the depth-wise separable convolution structure adopted by the channel attention layer f2 is used to realize image feature extraction through two convolution processes, channel-by-channel and point-by-point. The spatial dimensions of the input feature map are first fused using channel-by-channel convolution. Each convolution operation is processed with its corresponding single input channel, and then the obtained feature map channel information is fused using point-by-point convolution.

[0020] Furthermore, during upsampling, sub-pixel convolution operations are used to map remote sensing image features from low-resolution space to high-resolution space to obtain square feature maps of the upsampling factor. Then, pixels are extracted from the same position in each feature map for pixel reorganization. The reorganized pixels are the sub-pixels on the new feature map, and finally an enlarged image with a multiple of the upsampling factor is obtained.

[0021] Furthermore, the generative network and the discriminative network of the generative adversarial super-resolution network overall model are optimized by a super-resolution reconstruction algorithm. The specific steps of the super-resolution reconstruction algorithm are as follows:

[0022] Step 1: Input training sample pair P = {X, Y}: The high-resolution remote sensing image samples in the training set are [X1, X2, …X n ], after random cropping, we get high-resolution image samples X = [x1, x2, … x n ], after downsampling, the corresponding low-resolution image samples Y = [y1, y2, ... y n ];

[0023] Step 2: Set the low-resolution image sample Y = [y1, y2, ... n ] Generate the network according to the batch input and obtain the corresponding super-resolution image S = [s1, s2, ...s n ];

[0024] Step 3: Input the super-resolution image sample S obtained in Step 2 and the cropped high-resolution image sample X into the discriminant network; then the discriminant network gives its discriminant result, that is, the probability that the input image is a natural image, which is recorded as p and s 、p x ; Then calculate the discriminant loss d loss , and back-propagate the results to update the discriminant network parameters;

[0025] Step 4: Set the generated network gradient to zero and calculate the generated network loss g loss , and back-propagate the results to update the generated network parameters;

[0026] Next, the low-resolution image sample Y is input into the generative network, which generates the corresponding super-resolution image S and calculates the generative network loss g after updating the parameters. loss , and recalculate the discriminant loss d after updating the parameters loss .

[0027] Step 5: Return to Step 3 and Step 4, and continuously update the parameters of different batches of data until this round of training is completed.

[0028] Furthermore, in Step 3, the discriminant loss d loss The calculation formula is as follows:

[0029] d loss =1-p x +p s

[0030] In Step 4, the generated network loss g loss The calculation formula is as follows:

[0031] g loss =MSE loss +VGG loss +Adversarial loss

[0032] Among them, MSE loss Used to calculate high-definition images And generate images The difference between pixels; CGG loss Used to calculate and generate images and high definition images I HR Euclidean distance between feature representations; Adversarial loss Generate images for discriminative model judgment is the probability of high-definition images in the original data set; the calculation formulas for the above three network losses are as follows:

[0033]

[0034]

[0035]

[0036] in, represents the generation of convolutional neural network, θ G Represents network weights and bias terms; refers to the feature map obtained by the VGG-19 network, W i,j and H i,j Represents the dimension of the VGG-19 network feature map; Represented as a probability value, that is, the discriminant model determines the generated image is the probability of the HD image in the original dataset.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] (1) The present invention proposes a novel generative adversarial super-resolution network model based on remote sensing images. The attention generation network (AGN) integrates the saliency mechanism of channel attention, redistributes the weights of different channel features, fully considers the interdependence between different channels, and adopts a method of treating different importance levels differently, so that the model can learn the feature distribution of remote sensing images in a targeted manner.

[0039] Through the loss function and optimizer, the parameters of the two network models, the generation network and the discriminant network, are continuously iterated and optimized, so that the probability distribution of the generated image is approximately the same as the data distribution of the real image, alleviating the ill-posed problem of image super-resolution reconstruction.

[0040] (2) The present invention utilizes the respective advantages of deep separable convolution and residual network to realize image feature learning and reduce network redundancy, thereby achieving lightweight network design. First, the ordinary convolution of the generated network is replaced with deep separable convolution to reduce the number of model parameters, improve the performance of the network model, and help achieve the goal of lightweight network design. Then, the skip connection function of the residual is used to add the input features to the output features of the channel attention. By passing the feature map of the shallow network to the subsequent network layer, the connection between the features of the network layers is strengthened, helping the network to recover more image detail information.

[0041] The experimental results show that the model reconstruction effect of the present invention is good, the network training is better, and it can fully mine information from remote sensing image data, restore image high-frequency information, and improve the image edge contour and texture expression ability. In addition, under the condition of the same image reconstruction effect, the present invention uses fewer parameters and less calculation, which can play an important practical role in remote sensing applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0043] Figure 1 A network structure diagram of the overall network model of the present invention;

[0044] Figure 2 A network architecture diagram of a generation network of the present invention;

[0045] Figure 3 is a flow chart of the channel attention method of the present invention;

[0046] Figure 4 This is a flow chart of the upsampling method of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0048] The present invention studies a remote sensing image super-resolution reconstruction method based on a generative adversarial network with an attention mechanism. First, the characteristics of remote sensing images are analyzed to mine effective feature information of remote sensing images. Second, a lightweight network design method is designed to reduce network redundancy and improve network performance. In this regard, this embodiment proposes a remote sensing image super-resolution reconstruction method based on a lightweight generative model, comprising the following steps:

[0049] Step 1: Build a lightweight generative network structure, named Attention Generative Network (AGN). This generative network structure integrates the channel attention mechanism saliency model, fully exploits the edge texture and other detail features of remote sensing images, restores the high-frequency information of the image, and achieves better reconstruction effect. At the same time, it replaces the ordinary convolution with the depthwise separable convolution, effectively reducing the number of model parameters and helping to achieve the goal of lightweight network design.

[0050] Step 2: Construct a general model of a generative adversarial super-resolution network based on remote sensing images. The model includes a generative network (the attention generative network AGN in step 1) and a discriminative network. Through continuous training and adversarial training between the generative network and the discriminative network, a super-resolution image that is closer to the real image is generated.

[0051] The following describes each part separately:

[0052] 1. Network structure of the overall network model

[0053] The network structure is as follows Figure 1 As shown in the figure, the input image is a remote sensing image training dataset with an image size of 256×256. After four times downsampling, the low-resolution image is 64×64. Then, the image is input into the generative network to obtain a reconstructed super-resolution image with an image size of 256×256. The reconstructed image and the corresponding real image are input into the discriminative network, so that the discriminative network distinguishes the generated image from the real image, and the network is optimized by the game between the generative network and the discriminative network.

[0054] 1. Generate a network

[0055] The generative network is an end-to-end simple network structure, which adopts the post-up-sampling method. The attention generative network AGN of the present invention includes a shallow feature extraction layer f1, a channel attention layer f2, a middle feature extraction layer f3, and an up-sampling layer f4. The network architecture is as follows: Figure 2 As shown:

[0056] In this embodiment, the attention generation network AGN is represented as f AGN, input a low-resolution image into the network, this low-resolution image is marked as LR, and the output super-resolution image is marked as SR, we can get:

[0057] SR=f AGN (LR)

[0058] In this embodiment, the shallow feature extraction layer is composed of a 9×9 convolution kernel, the channel attention layer is composed of a CAdepth block, which is composed of a channel attention basic block CA_Block and a Depth_Block composed of two layers of depth-separable convolution structures; the middle feature extraction layer is composed of a 3×3 convolution kernel; the upsampling layer is composed of two sub-pixel convolution upsampling blocks; finally, after a layer of 9×9 convolution kernel, a complete network architecture is obtained.

[0059] After the entire process of the attention generation network AGN, it can be expressed as:

[0060] SR=f4(f3(f2(f1(LR))+f1(LR))+f1(LR))

[0061] Channel attention method: It should be noted that the channel attention basic block CA_Block described in this embodiment is composed of an average pooling layer, a convolution layer and a Sigmoid function. Among them, the statistics of the spatial feature representation of each channel are obtained through the average pooling layer. The first convolution layer uses a convolution kernel of size 1×1. The number of output channels is reduced by four times, the dimension is reduced, and the complexity of the model can be limited. The second layer of convolution also uses a convolution kernel of the same size. The output channel is expanded to the original number of channels, and then the reconstructed channel representation is input into the Sigmoid function. The weights of different channel features are assigned, and the dependency relationship between different channels is established. The utilization of image features is improved, the generalization ability of the model is enhanced, and finally the output is multiplied by the input channel structure. In short, the input feature passes through the channel attention basic block CA_Block, is scaled to obtain the output, and then the input and output are multiplied to obtain the final output of the channel attention basic block CA_Block. The specific process is as follows Figure 3 shown.

[0062] Depthwise separable convolution: The depthwise separable convolution structure used in the channel attention layer f2 of this embodiment is mainly used to realize image feature extraction through two convolution processes, channel-by-channel and point-by-point. The idea is: first use channel-by-channel convolution to fuse the spatial dimensions (i.e., height and width) of the input feature map, each convolution operation is processed with its corresponding single input channel, and then use point-by-point convolution to fuse the obtained feature map channel information.

[0063] The process of using depthwise separable convolution can be expressed as follows: After the CA_Block module, the size of the input feature map is 64×64×64 (length×width×number of channels). The first step is to perform channel-by-channel convolution, changing the single convolution kernel 3×3×64 in the conventional convolution to 64 convolution kernels of size 3×3×1, so that each convolution kernel is convolved with each channel of the corresponding input feature map, and 64 output features of size 64×64×1 are obtained. These 64 feature maps are stacked together to obtain an output feature map of size 64×64×64. The second step is to perform pointwise convolution, using a 1×1×64 convolution kernel to convolve with the output feature map obtained in the first step, and obtain a feature map of size 64×64×1. Let the number of 1×1 convolution kernels be 64, and then a 64×64×64 output feature map can be obtained through the depthwise separable convolution process.

[0064] Upsampling method: During upsampling, sub-pixel convolution operation is used to map the remote sensing image features from low-resolution space to high-resolution space, and a square feature map of the upsampling factor is obtained. Then, pixels are extracted from the same position in each feature map for pixel reorganization. The reorganized pixels are the sub-pixels on the new feature map, and finally an enlarged image of the upsampling factor multiple is obtained. Specifically, after performing a convolution operation on an input image of size H×W, n 2 (n is the upsampling factor) a feature map of size H×W, and then the feature map is pixel-reorganized to obtain an enlarged image of size nH×nW. The process is as follows: Figure 4 shown.

[0065] 2. Super-resolution reconstruction algorithm

[0066] The generative network and discriminative network of the overall model of the generative adversarial super-resolution network are optimized through the super-resolution reconstruction algorithm. Assume that the high-resolution remote sensing image samples in the training set are [X1, X2, …X n ], after random cropping, we get high-resolution image samples X = [x1, x2, … x n ], after a certain scale factor down sampling operation, the corresponding low-resolution image sample Y = [y1, y2, ... y n ]. After the above operations, the training sample pair (ie, the high-resolution and low-resolution image pair) is P = {X, Y}.

[0067] The specific steps of the super-resolution reconstruction algorithm are as follows:

[0068] Step 1: Input training sample pair P = {X, Y}:

[0069] Step 2: Set the low-resolution image sample Y = [y1, y2, ... n] Generate the network G according to the batch input and obtain the corresponding super-resolution image S = [s1, s2, ...s n ];

[0070] Step 3: Input the super-resolution image sample S obtained in Step 2 and the cropped high-resolution image sample X into the discriminant network D; then the discriminant network D gives its discriminant result, that is, the probability that the input image is a natural image, which is recorded as p s 、p x ; Then calculate the discriminant loss d loss , and back-propagate the results to update the discriminant network parameters. Its discriminant loss d loss The calculation formula is as follows:

[0071] d loss =1-P x +P s

[0072] Step 4: Set the gradient of the generated network G to zero and calculate the generated network loss g loss , and back propagate the results to update the generated network parameters. Its generated network loss g loss The calculation formula is as follows:

[0073] g loss =MSE loss +VGG loss +Adversarial loss

[0074] Among them, MSE loss It is a loss calculation method commonly used in the field of image processing and is used to calculate high-definition images. And generate images The difference between pixels; VGG loss Used to calculate and generate images and high definition images I HR Euclidean distance between feature representations; Adversarial loss Generate images for discriminative model judgment is the probability of high-definition images in the original data set. The calculation formulas for the above three network losses are as follows:

[0075]

[0076]

[0077]

[0078] in, represents the generation of convolutional neural network, θ G Represents network weights and bias terms; refers to the feature map obtained by the VGG-19 network, W i,j and H i,j Represents the dimension of the VGG-19 network feature map;

[0079] Represented as a probability value, that is, the discriminant model determines the generated image is the probability of the HD image in the original dataset.

[0080] Next, the low-resolution image sample Y is input into the generative network G, which generates the corresponding super-resolution image S and calculates the generative network loss g after updating the parameters. loss , and recalculate the discriminant loss d after updating the parameters loss .

[0081] Step 5: Return to Step 3 and Step 4, and continuously update the parameters of different batches of data until this round of training is completed.

[0082] Verification of the technical effect of the present invention:

[0083] We selected a remote sensing image dataset with sufficient samples, selected 600 remote sensing images of the airplane class in the NWPU-RESISC45 dataset for training, selected 200 remote sensing images in the RSOD-Dataset for verification, and conducted a series of test experiments on public datasets such as RSOD-Dataset, NWPU-RESISC45, NWPU-VHR, and UC Merced Land-Use Data Set. Finally, we used the two evaluation indicators, PSNR and SSIM, to evaluate and compare the experimental results, and conducted comparative experiments and ablation experiments with the same benchmark methods to verify and optimize.

[0084] The experimental results show that the overall effect of the super-resolution image of the AGN network proposed in the present invention is higher than that of the original SRGAN network. The average PSNR is about 0.2dB higher than that of SRGAN. For a single test image, its PSNR value is about 1dB higher than that of SRGAN, and its SSIM is about 0.5 higher. Compared with the SRGAN network model, the number of parameters of the AGN network model is also greatly reduced, by 49%, its floating point calculation amount is reduced by 26%, and the memory space occupied is reduced by 20%. Therefore, the AGN network model is more lightweight and more efficient.

[0085] In summary, the present invention proposes a novel generative adversarial super-resolution network model AGN based on remote sensing images. This network integrates the saliency mechanism of attention, redistributes the weights of different channel features, fully considers the interdependence between different channels, and adopts different treatment methods for different importance levels, so that the model can learn the feature distribution of remote sensing images in a targeted manner, continuously iterate and optimize the two network model parameters through loss functions and optimizers, and restore the high-frequency details of remote sensing images. Experiments show that the super-resolution reconstruction model proposed in this paper has good reconstruction effect, better network training, can fully mine information in remote sensing image data, restore image high-frequency information, and improve the image edge contour and texture expression ability.

[0086] The present invention utilizes the respective advantages of deep separable convolution and residual network to realize image feature learning and reduce network redundancy. First, the ordinary convolution of the generated network is replaced with deep separable convolution to reduce the number of model parameters, improve the performance of the network model, and help achieve the goal of lightweight network design. Then, the input features are added to the output features of the channel attention using the skip connection function of the residual, and the connection between the features of the shallow network is strengthened by passing the feature map of the shallow network to the subsequent network layer, helping the network to recover more image detail information. By adjusting the loss function, continuously experimenting and optimizing the model, the network structure is adjusted to reduce the number of parameters while ensuring that the accuracy remains unchanged, and a lightweight design is achieved. The experimental results show that under the same image reconstruction effect, the present invention uses fewer parameters and less calculation, which can play an important practical significance in remote sensing applications.

[0087] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Any changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A remote sensing image super-resolution reconstruction method based on a lightweight generative model, characterized in that: The following steps are involved: Step 1: Build a lightweight generative network structure, named Attention Generative Network (AGN). This generative network structure integrates the channel attention mechanism saliency model and replaces the ordinary convolution with the depthwise separable convolution. Step 2: construct a generative adversarial super-resolution network model based on remote sensing images, which includes the generative network and the discriminative network described in step 1. Through continuous training adversarial between the generative network and the discriminative network, a super-resolution image closer to the real image is generated. The generative network and the discriminative network of the generative adversarial super-resolution network overall model are optimized by a super-resolution reconstruction algorithm. The specific steps of the super-resolution reconstruction algorithm are as follows: Step 1: Input training sample pair P = {X, Y}: The high-resolution remote sensing image samples in the training set are [X1, X2, …X n ], after random cropping, we get high-resolution image samples X = [x1, x2, … x n ], after downsampling, the corresponding low-resolution image samples Y = [y1, y2, ... y n ]; Step 2: Set the low-resolution image sample Y = [y1, y2, ... n ] Generate the network according to the batch input and obtain the corresponding super-resolution image S = [s1, s2, ...s n ]; Step 3: Input the super-resolution image sample S obtained in Step 2 and the cropped high-resolution image sample X into the discriminant network; then the discriminant network gives its discriminant result, that is, the probability that the input image is a natural image, which is recorded as p and s 、p x ; Then calculate the discriminant loss d loss , and back-propagate the results to update the discriminant network parameters; In Step 3, the discriminant loss d loss The calculation formula is as follows: d loss =1-p x +p s In Step 4, the generated network loss g loss The calculation formula is as follows: g loss =MSE loss +VGG loss +Adversarial loss Among them, MSE loss Used to calculate high-definition images And generate images The difference between pixels; VGG loss Used to calculate and generate images and high definition images I HR Euclidean distance between feature representations; Adversarial loss Generate images for discriminative model judgment is the probability of high-definition images in the original data set; the calculation formulas for the above three network losses are as follows: in, represents the generation of convolutional neural network, θ G Represents network weights and bias terms; refers to the feature map obtained by the VGG-19 network, W i,j and H i,j Represents the dimension of the VGG-19 network feature map; Represented as a probability value, that is, the discriminant model determines the generated image is the probability of the high-definition image in the original dataset; Step 4: Set the generated network gradient to zero and calculate the generated network loss g loss , and back-propagate the results to update the generated network parameters; Next, the low-resolution image sample Y is input into the generative network, which generates the corresponding super-resolution image S and calculates the generative network loss g after updating the parameters. loss , and recalculate the discriminant loss d after updating the parameters loss ; Step 5: Return to Step 3 and Step 4, and continuously update the parameters of different batches of data until this round of training is completed.

2. The remote sensing image super-resolution reconstruction method based on a lightweight generation model according to claim 1, characterized in that: The attention generation network AGN includes a shallow feature extraction layer f1, a channel attention layer f2, a middle feature extraction layer f3, and an upsampling layer f4; The attention generation network AGN is represented by f AGN , input a low-resolution image into the network, this low-resolution image is marked as LR, and the output super-resolution image is marked as SR, we can get: SR=f AGN (LR) After the entire process of the attention generation network AGN, it can be expressed as: SR=f4(f3(f2(f1(LR))+f1(LR))+f1(LR)).

3. The remote sensing image super-resolution reconstruction method based on a lightweight generation model according to claim 2 is characterized in that: The channel attention layer f2 is composed of a CAdepth block, which is composed of a channel attention basic block CA_Block and a Depth_Block composed of two layers of depth-separable convolution structures; the channel attention basic block CA_Block is composed of an average pooling layer, a convolution layer and a Sigmoid function; The input feature passes through the channel attention basic block CA_Block, is scaled to get the output, and then the input and output are multiplied to get the final output of the channel attention basic block CA_Block.

4. The remote sensing image super-resolution reconstruction method based on a lightweight generation model according to claim 3 is characterized in that: The average pooling layer of the channel attention basic block CA_Block obtains the statistics of the spatial feature representation of each channel. The first convolution layer and the second convolution layer use the same size of convolution kernel, but the number of output channels of the first convolution layer is reduced by four times, and the output channels of the second convolution layer are expanded to the original number of channels. The reconstructed channel representation is then input into the Sigmoid function, and the weights of different channel features are assigned to establish the dependency relationship between different channels. Finally, the output is multiplied by the input channel structure to obtain the output.

5. The remote sensing image super-resolution reconstruction method based on a lightweight generation model according to claim 3 is characterized in that: The depth-wise separable convolution structure adopted by the channel attention layer f2 is used to realize image feature extraction through two convolution processes, channel-by-channel and point-by-point. The spatial dimensions of the input feature map are first fused using channel-by-channel convolution. Each convolution operation is processed on its corresponding single input channel, and then the obtained feature map channel information is fused using point-by-point convolution.

6. The remote sensing image super-resolution reconstruction method based on a lightweight generation model according to claim 5, characterized in that: During upsampling, sub-pixel convolution operations are used to map remote sensing image features from low-resolution space to high-resolution space to obtain square feature maps of the upsampling factor. Then, pixels are extracted from the same position in each feature map for pixel reorganization. The reorganized pixels are the sub-pixels on the new feature map, and finally an enlarged image with a multiple of the upsampling factor is obtained.

Images