Image super-resolution method and device based on two-stage generative adversarial network

Abstract

The present invention discloses an image super-resolution method and device based on a two-stage generative adversarial network, aiming to solve the problem of detail loss and noise in super-resolution reconstruction of remote sensing images in the prior art. The method improves the resolution of low-resolution remote sensing images and enhances the naturalness and detail richness of images through a two-stage generative adversarial network. In the first stage, a first generative adversarial network is trained using real low-resolution images and low-resolution images generated by bicubic downsampling to generate low-resolution images similar to the bicubic style. In the second stage, a second generative adversarial network is trained using the low-resolution images generated in the first stage and real high-resolution images to generate high-resolution images similar to real high-resolution images. The method of the present invention not only improves the spatial resolution of the image, but also enhances the visual quality and application value of the image.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image super-resolution method and device based on a two-stage generative adversarial network. Background Art

[0002] With the rapid development of remote sensing technology, obtaining high-resolution remote sensing images has become increasingly important for geographic information systems, urban planning, environmental monitoring and other fields. However, due to sensor limitations, cost considerations and bandwidth limitations for data transmission, there are certain challenges in directly obtaining high-resolution images. In order to solve this problem, image super-resolution technology has emerged. Its core goal is to improve the spatial resolution of low-resolution images through algorithms to approach or reach the quality of high-resolution images.

[0003] Traditional image super-resolution techniques mainly rely on interpolation methods, such as nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. Although these methods are simple to implement, they often cannot effectively restore the detailed information of the image and are prone to introduce unnatural artifacts. With the development of deep learning technology, super-resolution methods based on convolutional neural networks (CNNs) have shown significant performance improvements. However, these methods usually require pairs of high-resolution and low-resolution images for training, which are often difficult to obtain in practical applications. In addition, most of the existing super-resolution methods based on deep learning use an end-to-end network structure to directly predict high-resolution images from low-resolution images, but this method often ignores the problem of detail loss and noise in the image degradation process, resulting in the generated high-resolution images being visually unnatural and difficult to meet the high-quality requirements in practical applications. The present invention aims to solve the above problems in the prior art and provides a novel two-stage super-resolution method. By introducing the generative adversarial network (GAN) technology, it can not only effectively improve the resolution of low-resolution remote sensing images, but also enhance the naturalness and detail richness of the images. Summary of the invention

[0004] The purpose of the present invention is to address the deficiencies of the prior art and provide an image super-resolution method and device based on a two-stage generative adversarial network to reduce the impact of detail loss and noise in low-resolution images on high-resolution image reconstruction.

[0005] The object of the present invention is to achieve the following technical solution: an image super-resolution method based on a two-stage generative adversarial network, comprising:

[0006] Obtain the corresponding real high-resolution image and real low-resolution image, and use the bicubic interpolation method to convert the real high-resolution image into a bicubic style low-resolution image;

[0007] Using the real low-resolution image and its corresponding bicubic style low-resolution image to train a first generative adversarial network; the first generative adversarial network includes a first generator and a first discriminator; the trained first generator is used to generate a synthetic low-resolution image;

[0008] Using the synthetic low-resolution image and its corresponding real high-resolution image to train a second generative adversarial network; the second generative adversarial network includes a second generator and a second discriminator;

[0009] The trained first generator and the trained second generator are used to perform high-resolution reconstruction of the image to be processed.

[0010] Furthermore, the first generator is composed of a convolution module, a residual cascade module, and a convolution module connected in sequence in a head-to-tail order.

[0011] Furthermore, the second generator is composed of a convolution module, a residual cascade module, a convolution module and an upsampling module connected in sequence in a head-to-tail order.

[0012] Furthermore, the upsampling module includes a convolution layer, a PixelShuffle layer, a convolution layer and a PixelShuffle layer and are connected in sequence.

[0013] Furthermore, the residual cascade module is composed of several residual blocks connected end to end, and the residual block includes several multi-scale residual attention blocks and a convolutional neural network.

[0014] Furthermore, the multi-scale residual attention block includes a large-scale residual attention block, a medium-scale residual attention block and a small-scale residual attention block; the large-scale residual attention block, the medium-scale residual attention block and the small-scale residual attention block are each composed of a convolutional layer, a ReLU activation function layer and a convolutional layer connected in sequence.

[0015] Furthermore, the loss function of the first generator is composed of the addition of pixel loss and adversarial loss.

[0016] Furthermore, the loss function of the second generator is composed of the addition of pixel loss, perceptual loss and adversarial loss; wherein the perceptual loss is the sum of the element-by-element differences between the feature vectors of the synthetic low-resolution image and its corresponding real high-resolution image after passing through the feature extractor.

[0017] The present invention also provides an image super-resolution device based on a two-stage generative adversarial network, comprising one or more processors for implementing the above-mentioned image super-resolution method based on a two-stage generative adversarial network.

[0018] The present invention also provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it is used to implement the above-mentioned image super-resolution method based on a two-stage generative adversarial network.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] 1. Traditional remote sensing image super-resolution methods (such as those based on adversarial generation) often directly reconstruct low-resolution images into high-resolution images. However, traditional methods ignore the problem of detail loss and noise in the image degradation process. The present invention considers a two-stage adversarial generation method, first reconstructing the true resolution image into a bicubic style image, thereby partially restoring the detail information in the image (such as the boundaries of the objects), and then reconstructing it into a synthetic resolution image based on the bicubic style image, thereby obtaining a better reconstruction effect.

[0021] 2. Both the first generator and the second generator of the present invention adopt an innovative network structure, including a convolution module, a residual cascade module and a multi-scale residual attention block, to extract image features in an all-round way. In addition, the design of the loss function takes into account pixel loss, adversarial loss and perceptual loss to further improve the quality of image reconstruction.

[0022] 3. In remote sensing images, different objects have different spatial forms and boundaries. By introducing a multi-scale multi-attention residual block model based on the residual cascade module, on the one hand, the ability to reconstruct multi-scale object features is guaranteed; on the other hand, the residual cascade capability ensures that the model can still be effectively trained even when the network is deep. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0024] Figure 1 is a flowchart of an image super-resolution method based on a two-stage generative adversarial network provided by an embodiment of the present invention;

[0025] Figure 2 are example images of three different images (real low-resolution images, real high-resolution images, and bicubic-style low-resolution images) provided by an embodiment of the present invention;

[0026] Figure 3 It is a schematic diagram of a first generative adversarial network structure provided by an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of a residual block structure provided by an embodiment of the present invention;

[0028] Figure 5 It is a schematic diagram of a multi-scale residual attention block structure provided by an embodiment of the present invention;

[0029] Figure 6 is a schematic diagram of a second generative adversarial network structure provided by an embodiment of the present invention;

[0030] Figure 7 is a schematic diagram of the structure of an upsampling module provided by an embodiment of the present invention;

[0031] Figure 8 This is a comparison diagram of super-resolution effects of remote sensing images provided by an embodiment of the present invention;

[0032] Fig. 9 It is a schematic diagram of a hardware structure provided by an embodiment of the present invention.

[0033] Among them, the real low-resolution image, the real high-resolution image, the synthetic low-resolution image and the synthetic high-resolution image are respectively abbreviated as the real low-resolution image, the real high-resolution image, the synthetic low-resolution image and the synthetic high-resolution image in the figure. DETAILED DESCRIPTION

[0034] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0035] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0036] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0037] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0038] The present invention provides an image super-resolution method based on a two-stage generative adversarial network. Figure 1 As shown, the following steps are included:

[0039] (1) Obtain the corresponding real high-resolution image and real low-resolution image, and use the bicubic interpolation method to convert the real high-resolution image into a bicubic style low-resolution image;

[0040] Specifically, obtain a series of real high-resolution images Compared with the real low-resolution image The resulting real data set :

[0041]

[0042] Among them, each and One-to-one correspondence; then, bicubic interpolation is used to convert each real high-resolution image Convert to bicubic style low-resolution image (like Figure 2 shown):

[0043]

[0044] in, represents the bicubic downsampling method. Then, the real dataset Expanded to include the processed data set , which will be used in the following steps The data set is used to train a two-stage generative adversarial network (the two-stage generative adversarial network corresponds to the first generative adversarial network and the second generative adversarial network):

[0045]

[0046] (2) using the real low-resolution image and its corresponding bicubic style low-resolution image to train a first generative adversarial network; the first generative adversarial network includes a first generator and a first discriminator; the trained first generator is used to generate a synthetic low-resolution image;

[0047] Specifically, train the first generative adversarial network ( Figure 3 ), the purpose is to obtain the trained first generator With the first discriminator :

[0048]

[0049]

[0050] in, To synthesize low-resolution images. For the need to be The image to be judged can be or One of a kind. The first discriminator is based on the image If the scoring is The higher it is, the more likely the first discriminator thinks the input image is a bicubic style low-resolution image.

[0051] (3) using the synthetic low-resolution image and its corresponding real high-resolution image to train a second generative adversarial network; the second generative adversarial network includes a second generator and a second discriminator;

[0052] Specifically, the second generative adversarial network is trained to obtain a trained second generator With the second discriminator :

[0053]

[0054]

[0055] in, To synthesize high-resolution images. For the need to be The image to be judged can be or One of them. The second discriminator is based on the image If the scoring is The higher it is, the more likely the second discriminator thinks its input image is a real high-resolution image.

[0056] (4) Using the trained first generator and second generator, high-resolution reconstruction of the image to be processed is performed.

[0057] Specifically, through the trained first generator With the second generator , the real low-resolution image to be processed , and finally generate a synthetic high-resolution image :

[0058]

[0059] In one embodiment, the first generator The convolution modules , residual cascade module , convolution module Connect them in the order of head to tail, with the output of the previous module being the input of the next module, such as Figure 3 As shown:

[0060]

[0061] In one embodiment, the convolution module and Both are single-layer convolutional neural networks.

[0062] In one embodiment, the residual cascade module Several residual blocks Connect head to tail, such as Figure 3 As shown in FIG. 1 , the specific number of residual blocks can be adjusted according to actual applications.

[0063] In one embodiment, the residual block Composed of multiple multi-scale residual attention blocks With a convolutional layer Connect in order, such as Figure 4 As shown. Among them, It is an element-by-element addition operation. Multi-scale residual attention block The specific number can be adjusted according to the actual application.

[0064] In one embodiment, the multi-scale residual attention blocks are large-scale residual attention blocks , mid-scale residual attention block , small-scale residual attention block .like Figure 5 As shown in the figure, the three attention blocks of different scales are composed of a convolutional layer, a ReLU activation function layer, and a convolutional layer connected in sequence:

[0065]

[0066]

[0067]

[0068] The different convolutional layers , , , , , The image features will be extracted from different spatial scales. It is an element-by-element addition operation. With the residual connection mechanism, it can fully extract the features of objects of different sizes in the image for subsequent image generation.

[0069] In one embodiment, the first generator The loss function It consists of two parts, namely pixel loss and fighting losses :

[0070]

[0071]

[0072]

[0073] Where N represents the total number of pixels in the image; Represents the weight of the adversarial loss, which can be adjusted according to the actual task.

[0074] In one embodiment, the first discriminator It consists of four convolution modules. , , , And a fully connected module They are connected in sequence, where the output of the previous module is the input of the next module, such as Figure 3 As shown:

[0075]

[0076] Among them, each convolution module contains a convolution layer and a LeakyReLU layer.

[0077] In one embodiment, the first discriminator The loss function It is expressed as follows:

[0078]

[0079] In one embodiment, the second generator The convolution modules , residual cascade module , convolution module And upsampling module Connect them in the order of head to tail, with the output of the previous module being the input of the next module, such as Figure 6 As shown:

[0080]

[0081] in, , , Similar to the convolution module and residual cascade module in the first generator. However, compared with the first generator, the second generator has an additional upsampling module. , this is because low-dimensional information needs to be mapped to high-dimensionality.

[0082] In one embodiment, the convolution module and Both are single-layer convolutional neural networks.

[0083] In one embodiment, the residual cascade module Cascade module with the residual in the first generator The structure is exactly the same. There are also several residual blocks inside At the same time, these residual blocks It also contains multiple multi-scale residual attention blocks MRAB (∙) and a convolutional neural network CNN (∙), which are connected in sequence. Through the innovative use of the multi-scale residual attention mechanism, the second generator can also meet the needs of high-resolution reconstruction of objects at different spatial scales.

[0084] In one embodiment, the upsampling module By a convolutional layer , a PixelShuffle layer , a convolutional layer , a PixelShuffle layer Connect to each other in the order of first place, such as Figure 7 As shown. The output of the previous layer is the input of the next layer:

[0085]

[0086] The Pixelshuffle layer rearranges a low-resolution image with a large number of channels into a high-resolution image with a small number of channels. By combining Pixelshuffle with the convolution layer, high-quality image reconstruction can be achieved.

[0087] In one embodiment, the second generator The loss function It is composed of three parts: pixel loss , Perceptual Loss , and fight against losses :

[0088]

[0089]

[0090]

[0091]

[0092] in, and is the corresponding coefficient term, which can be adjusted according to the actual work. The pixel loss and adversarial loss here are similar to the loss term of the first generator. To calculate the perceptual loss, the pre-trained neural network is used as the feature extractor, and the sum of the element-by-element difference between the feature vectors of the two images after the feature extractor is the perceptual loss. Here, the pre-trained VGG-19 network is used as the feature extractor, and Refers to the operation of passing an image through a feature extractor to form a feature vector. The reason why perceptual loss is added to the second generator but not to the first generator is mainly because the generation tasks of the two stages are different. In the first stage, the goal of the generation task is to reduce the blur and noise of the image, rather than directly generate high-resolution details. In the second stage, the generated image of the generation task needs to have textures and details similar to those of the real high-resolution image, so perceptual loss is introduced to evaluate and improve the perceptual quality of the generated image, making it visually closer to the real high-resolution image.

[0093] In one embodiment, the second discriminator It consists of four convolution modules. , , , And a fully connected module They are connected in sequence, where the output of the previous module is the input of the next module, such as Figure 6 As shown:

[0094]

[0095] Among them, each convolution module contains a convolution layer and a LeakyReLU layer.

[0096] In one embodiment, the second discriminator The loss function It is expressed as follows:

[0097]

[0098] Embodiment 1:

[0099] With the rapid development of remote sensing technology, obtaining high-resolution remote sensing images has become increasingly important for geographic information systems, urban planning, environmental monitoring and other fields. However, due to the limitations of sensors, cost considerations and bandwidth limitations of data transmission, there are certain challenges in directly obtaining high-resolution images. This embodiment uses the above method to reconstruct low-resolution remote sensing images with high resolution; specifically, the following steps are included:

[0100] Step 1: Use PAN images and MS images covering the same area from the WorldView-3 satellite. The spatial resolution of PAN images is 0.31 meters, which is used to construct a real high-resolution image dataset (512 pixels × 512 pixels). The spatial resolution of MS images is 1.24 meters (128 pixels × 128 pixels), which is used to construct a real low-resolution image dataset. We collected 2048 corresponding PAN images and MS images. Then, the bicubic downsampling method is used on the PAN images to construct bicubic style low-resolution images.

[0101] In step 2, a first generative adversarial network is constructed, and its specific structure can be found in the content of the above method embodiment, which will not be repeated here;

[0102] For the first generator : Residual Cascade Module Internal residual block The number of residual blocks is 5. Internal multi-scale residual attention block The number of convolution modules is 10. The convolutional layer in , uses 64 And the convolution kernel has a stride of 1. Convolution module The convolutional layer in , uses 3 And the convolution kernel has a stride of 1.

[0103] For the first generator Multi-scale residual attention blocks in: Convolutional layers and , using 64 And the convolution kernel has a stride of 1. Convolutional layer and , using 64 And the convolution kernel has a stride of 1. Convolutional layer and , using 64 And the convolution kernel has a stride of 1.

[0104] For the first discriminator : Convolution module The convolutional layer in , uses 64 And the convolution kernel has a stride of 1. Convolution module The convolutional layer in , uses 128 And the convolution kernel has a stride of 1. Convolution module The convolutional layer in , uses 256 And the convolution kernel has a stride of 1. Convolution module The convolutional layer in , uses 512 And the convolution kernel has a stride of 1.

[0105] For the first generator The loss term weight is: is 0.001.

[0106] The first generative adversarial network constructed is trained using the MS image and its corresponding bicubic style low-resolution image until and Convergence. Through training , a series of real low-resolution remote sensing images can be converted into synthetic low-resolution remote sensing images.

[0107] Step 3: construct a second generative adversarial network. The specific structure of the network can be found in the above method embodiment and will not be described in detail here.

[0108] For the second generator : Residual Cascade Module Internal residual block The number of residual blocks is 5. Internal multi-scale residual attention block The number of convolution modules is 20. The convolutional layer in , uses 64 And the convolution kernel has a stride of 1. Convolution module The convolutional layer in , uses 3 And the convolution kernel with a stride of 1. In addition, Internal multi-scale residual attention block Configuration and Same.

[0109] For the second generator Upsampling module: convolutional layer 256 used And the convolution kernel has a stride of 1. Convolutional layer 512 used And the convolution kernel has a stride of 1. PixelShuffle layer and The upsampling factor is 2.

[0110] For the second discriminator : Convolution module The convolutional layer in , uses 64 And the convolution kernel has a stride of 2. Convolution module The convolutional layer in , uses 128 And the convolution kernel has a stride of 2. Convolution module The convolutional layer in , uses 256 And the convolution kernel has a stride of 2. Convolution module The convolutional layer in , uses 512 And the convolution kernel has a stride of 1.

[0111] For the second generator The loss term weight is: is 0.001, is 0.01.

[0112] The second generative adversarial network constructed is trained using synthetic low-resolution remote sensing images and their corresponding PAN images until and Convergence. Through training , a series of synthetic low-resolution remote sensing images can be converted into synthetic high-resolution remote sensing images.

[0113] Step 4: After obtaining the real low-resolution remote sensing image to be processed, the trained and , generate synthetic high-resolution remote sensing images, thereby achieving high-resolution reconstruction of remote sensing images, the effect is as follows Figure 8 shown.

[0114] Corresponding to the aforementioned embodiment of an image super-resolution method based on a two-stage generative adversarial network, the present invention also provides an embodiment of an image super-resolution device based on a two-stage generative adversarial network.

[0115] In summary, the method of the present invention not only improves the spatial resolution of the image, but also enhances the visual quality and application value of the image, and is suitable for multiple fields such as geographic information systems, urban planning and environmental monitoring.

[0116] See also Fig. 9 An embodiment of the present invention provides an image super-resolution device based on a two-stage generative adversarial network, comprising one or more processors for implementing an image super-resolution method based on a two-stage generative adversarial network in the above embodiment.

[0117] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0118] An embodiment of an image super-resolution device based on a two-stage generative adversarial network of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From a hardware perspective, if Fig. 9 As shown, it is a hardware structure diagram of an image super-resolution device based on a two-stage generative adversarial network of the present invention, in which any device with data processing capability is located, except Fig. 9 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0119] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0120] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0121] An embodiment of the present invention also provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, an image super-resolution method based on a two-stage generative adversarial network in the above embodiment is implemented.

[0122] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0123] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. An image super-resolution method based on a two-stage generative adversarial network, characterized in that: include: Obtain the corresponding real high-resolution image and real low-resolution image, and use the bicubic interpolation method to convert the real high-resolution image into a bicubic style low-resolution image; Using the real low-resolution image and its corresponding bicubic style low-resolution image to train a first generative adversarial network; the first generative adversarial network includes a first generator and a first discriminator; The trained first generator is used to generate synthetic low-resolution images; Using the synthetic low-resolution image and its corresponding real high-resolution image to train a second generative adversarial network; the second generative adversarial network includes a second generator and a second discriminator; The trained first generator and the trained second generator are used to perform high-resolution reconstruction of the image to be processed.

2. The method according to claim 1, characterized in that The first generator is composed of a convolution module, a residual cascade module, and a convolution module connected in sequence in a head-to-tail order.

3. The method according to claim 1, characterized in that The second generator is composed of a convolution module, a residual cascade module, a convolution module and an upsampling module connected in sequence in a head-to-tail order.

4. The method according to claim 3, characterized in that The upsampling module includes a convolution layer, a PixelShuffle layer, a convolution layer and a PixelShuffle layer which are connected in sequence.

5. The method according to claim 2 or 3, characterized in that: The residual cascade module is composed of several residual blocks connected end to end, and the residual block includes several multi-scale residual attention blocks and a convolutional neural network.

6. The method according to claim 5, characterized in that The multi-scale residual attention block includes a large-scale residual attention block, a medium-scale residual attention block and a small-scale residual attention block; the large-scale residual attention block, the medium-scale residual attention block and the small-scale residual attention block are respectively composed of a convolutional layer, a ReLU activation function layer and a convolutional layer connected in sequence.

7. The method according to claim 1, characterized in that The loss function of the first generator is composed of the sum of pixel loss and adversarial loss.

8. The method according to claim 1, characterized in that The loss function of the second generator is composed of the sum of pixel loss, perceptual loss and adversarial loss; wherein the perceptual loss is the sum of the element-by-element differences between the feature vectors of the synthetic low-resolution image and its corresponding real high-resolution image after passing through the feature extractor.

9. An image super-resolution device based on a two-stage generative adversarial network, characterized in that: It includes one or more processors for implementing an image super-resolution method based on a two-stage generative adversarial network as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, it is used to implement an image super-resolution method based on a two-stage generative adversarial network as described in any one of claims 1 to 8.

Images