A satellite super-resolution image generation method and device and a storage medium

Abstract

The application provides a satellite super-resolution image generation method and device and a storage medium, and belongs to the field of image processing.The method comprises the following steps: obtaining a low-resolution satellite image by respectively performing image preprocessing on original satellite images; obtaining a low-level satellite feature map by respectively performing first feature extraction on the low-resolution satellite image; obtaining a target satellite feature map by respectively performing feature extraction and analysis on the low-level satellite feature map through a trained model; and obtaining an image generation result by respectively performing image reconstruction on the target satellite feature map through a reconstruction model.The application solves the technical problem that the image quality is not high when a low-resolution satellite image is generated into a high-resolution satellite image, realizes end-to-end mapping between the low-resolution satellite image and the high-resolution satellite image, and thus generates a high-resolution satellite image with higher quality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method, apparatus, and storage medium for generating satellite super-resolution images. Background Technology

[0002] The spatial resolution of remote sensing satellite images is a crucial indicator of a country's remote sensing satellite technology development. To improve the resolution and clarity of satellite images, thereby enhancing the accuracy of subsequent classification and identification, it is necessary to process these low-quality, low-resolution image data using appropriate methods or approaches. The simplest measure to improve the spatial resolution of satellite images is to upgrade the hardware, such as increasing the focal length of the optical system or reducing the width of the CCD (charge-coupled device) array elements. However, this increases the difficulty of component manufacturing, raises production costs, and leads to increased size and weight of the remote sensing device, hindering its loading and carrying. Reducing the width of the CCD array elements not only demands extremely high precision in manufacturing processes but also results in an excessively wide image transmission signal bandwidth, increasing data volume and causing transmission difficulties. In summary, improving image quality through hardware upgrades is time-consuming, labor-intensive, and extremely costly, often proving impractical. Therefore, how to economically and effectively improve the resolution and image quality of remote sensing satellite images, and enhance the clarity and recognizability of targets within the images, remains a pressing issue that needs to be addressed. Satellite image super-resolution technology is a key solution to the above problems. It combines existing prior information to reconstruct the observed low-resolution satellite images into high-quality, high-fidelity, high-resolution images.

[0003] Due to its significant application value in improving image resolution and image quality, super-resolution technology has gained attention from academia and industry worldwide in recent years and has become a hot research topic.

[0004] Super-resolution techniques commonly include interpolation-based and reconstruction-based methods. Kernel-based and edge-feature-based interpolation methods are simple and efficient, but due to the inability to obtain prior information beyond the low-resolution image, the generated high-frequency detail information is insufficient, leading to aliasing effects in the reconstructed image and failing to meet practical application requirements. Reconstruction-based super-resolution methods can utilize complementary information between multiple frames, increasing the source of high-frequency information to some extent. However, it has been proven that in practical situations, the effective magnification of reconstruction-based super-resolution algorithms is only 1.6 times, which greatly limits the application prospects of these algorithms. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method, apparatus and storage medium for generating satellite super-resolution images, which addresses the shortcomings of the prior art.

[0006] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A method for generating satellite super-resolution images, comprising the following steps:

[0007] Import multiple raw satellite images, and perform image preprocessing on each of the raw satellite images to obtain low-resolution satellite images corresponding to each of the raw satellite images;

[0008] Each of the low-resolution satellite images is subjected to initial feature extraction to obtain a low-level satellite feature map corresponding to each of the original satellite images;

[0009] A training model is constructed, and feature extraction and analysis are performed on each of the low-level satellite feature maps using the training model to obtain target satellite feature maps corresponding to each of the original satellite images.

[0010] A reconstruction model is constructed, and image reconstruction is performed on the feature maps of each target satellite using the reconstruction model to obtain super-resolution satellite images corresponding to each original satellite image. All super-resolution satellite images are then used as the image generation result.

[0011] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: A satellite super-resolution image generation device, comprising:

[0012] The image preprocessing module is used to import multiple raw satellite images, perform image preprocessing on each of the raw satellite images respectively, and obtain a low-resolution satellite image corresponding to each of the raw satellite images;

[0013] The feature extraction module is used to perform initial feature extraction on each of the low-resolution satellite images to obtain a low-level satellite feature map corresponding to each of the original satellite images;

[0014] The feature extraction and analysis module is used to construct a training model, and to perform feature extraction and analysis on each of the low-level satellite feature maps through the training model to obtain target satellite feature maps corresponding to each of the original satellite images.

[0015] The image generation result acquisition module is used to construct a reconstruction model, and to reconstruct the feature maps of each target satellite using the reconstruction model to obtain super-resolution satellite images corresponding to each original satellite image, and to use all the super-resolution satellite images as the image generation result.

[0016] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a satellite super-resolution image generation device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the satellite super-resolution image generation method as described above.

[0017] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the satellite super-resolution image generation method as described above.

[0018] The beneficial effects of this invention are as follows: by preprocessing the original satellite images to obtain low-resolution satellite images, by performing initial feature extraction on the low-resolution satellite images to obtain low-level satellite feature maps, by using a trained model to perform feature extraction and analysis on the low-level satellite feature maps to obtain target satellite feature maps, and by using a reconstruction model to reconstruct the target satellite feature maps to obtain image generation results, this invention solves the technical problem of low-quality images when generating high-resolution satellite images from low-resolution satellite images. It achieves end-to-end mapping between low-resolution and high-resolution satellite images, thereby generating higher-quality high-resolution satellite images. Attached Figure Description

[0019] Figure 1 A schematic flowchart of a satellite super-resolution image generation method provided in an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of the hardware structure of a satellite super-resolution image generation method provided in an embodiment of the present invention;

[0021] Figure 3 This is a block diagram of a satellite super-resolution image generation device provided in an embodiment of the present invention. Detailed Implementation

[0022] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0023] Figure 1 This is a flowchart illustrating a satellite super-resolution image generation method provided in an embodiment of the present invention.

[0024] like Figure 1 As shown, a method for generating satellite super-resolution images includes the following steps:

[0025] Import multiple raw satellite images, and perform image preprocessing on each of the raw satellite images to obtain low-resolution satellite images corresponding to each of the raw satellite images;

[0026] Each of the low-resolution satellite images is subjected to initial feature extraction to obtain a low-level satellite feature map corresponding to each of the original satellite images;

[0027] A training model is constructed, and feature extraction and analysis are performed on each of the low-level satellite feature maps using the training model to obtain target satellite feature maps corresponding to each of the original satellite images.

[0028] A reconstruction model is constructed, and image reconstruction is performed on the feature maps of each target satellite using the reconstruction model to obtain super-resolution satellite images corresponding to each original satellite image. All super-resolution satellite images are then used as the image generation result.

[0029] It should be understood that the training phase of this invention uses the public satellite image dataset Draper (i.e., multiple original satellite images), the original image size and resolution of which are 3099×2329 pixels.

[0030] It should be understood that the paired image dataset (i.e., multiple of the original satellite images) was tested on the public satellite image dataset Draper, which contains more than a thousand satellite images taken in Southern California. The original images on Draper are 3099×2329 pixels in size and resolution.

[0031] It should be understood that the refined satellite image feature map (i.e., the target satellite feature map) is processed by the reconstruction model to generate the final high-resolution satellite image (i.e., the super-resolution satellite image).

[0032] In the above embodiments, low-resolution satellite images are obtained by preprocessing the original satellite images, and low-level satellite feature maps are obtained by first-time feature extraction of the low-resolution satellite images. Target satellite feature maps are obtained by feature extraction and analysis of the low-level satellite feature maps using a trained model, and image generation results are obtained by image reconstruction of the target satellite feature maps using a reconstruction model. This solves the technical problem of low-resolution satellite images generating high-resolution satellite images with low image quality, and realizes end-to-end mapping between low-resolution and high-resolution satellite images, thereby generating higher-quality high-resolution satellite images.

[0033] Optionally, as an embodiment of the present invention, the process of performing image preprocessing on each of the original satellite images to obtain a low-resolution satellite image corresponding to each of the original satellite images includes:

[0034] Each of the original satellite images is cropped to obtain a cropped satellite image corresponding to each of the original satellite images;

[0035] The cropped satellite images are downsampled for the first time using a bicubic interpolation algorithm to obtain low-resolution satellite images corresponding to the original satellite images.

[0036] It should be understood that bicubic interpolation (i.e., the bicubic interpolation algorithm) is a more complex interpolation method that can create smoother image edges than bilinear interpolation. Bicubic interpolation is commonly used in some image processing software, printer drivers, and digital cameras to enlarge the original image or certain areas of the original image. Adobe Photoshop CS provides users with two different bicubic interpolation methods: bicubic interpolation smoothing and bicubic interpolation sharpening.

[0037] It should be understood that the image (i.e., the original satellite image) is cropped to 192×192 pixels to obtain a high-resolution satellite image (i.e., the cropped satellite image), and then the high-resolution satellite image (i.e., the cropped satellite image) is downsampled to 48×48 pixels through bicubic interpolation to obtain the low-resolution satellite image, thereby obtaining a paired dataset, each image pair containing one low-resolution satellite image and one corresponding high-resolution satellite image (i.e., the original satellite image).

[0038] Specifically, the image (i.e., the original satellite image) is cropped to 192×192 pixels to become a high-resolution image (i.e., the cropped satellite image). Then, the high-resolution image (i.e., the cropped satellite image) is downsampled to 48×48 pixels through bicubic interpolation to become a low-resolution image. 1000 images are selected for training and validation, and 200 images are selected for testing.

[0039] In the above embodiments, cropped satellite images are obtained by cropping the original satellite images. Low-resolution satellite images are obtained by first downsampling the cropped satellite images using a bicubic interpolation algorithm, which provides more accurate data for subsequent processing. This achieves end-to-end mapping between low-resolution and high-resolution satellite images, thereby generating higher-quality high-resolution satellite images.

[0040] Optionally, as an embodiment of the present invention, the process of performing initial feature extraction on each of the low-resolution satellite images to obtain low-level satellite feature maps corresponding to each of the original satellite images includes:

[0041] Based on the first 3×3 convolutional layer, the low-resolution satellite images are subjected to initial feature extraction to obtain low-level satellite feature maps corresponding to each of the original satellite images.

[0042] It should be understood that feature extraction operations are performed on the target low-resolution image (i.e., the low-resolution satellite image) to obtain low-level features of the satellite image (i.e., the low-level satellite feature map).

[0043] It should be understood that the acquired low-resolution remote sensing satellite image (i.e., the low-resolution satellite image) is processed by a 3×3 convolutional layer (i.e., the first 3×3 convolutional layer) to extract features, so as to generate a coarse low-level remote sensing satellite feature map (i.e., the low-level satellite feature map).

[0044] In the above embodiments, based on the first 3×3 convolutional layer, a low-level satellite feature map is obtained by the initial feature extraction of the low-resolution satellite image, and the features of the image are initially extracted, thereby realizing the end-to-end mapping between the low-resolution satellite image and the high-resolution satellite image.

[0045] Optionally, as an embodiment of the present invention, the training model includes multiple cross-scale structure maintenance blocks and multiple local texture attention modules, wherein the number of cross-scale structure maintenance blocks and local texture attention modules is the same and they are arranged alternately.

[0046] The process of constructing a training model and performing feature extraction and analysis on each of the low-level satellite feature maps using the training model to obtain target satellite feature maps corresponding to each of the original satellite images includes:

[0047] The global structural features of each of the low-level satellite feature maps are extracted by the first cross-scale structure maintenance block to obtain the global structural feature map corresponding to each of the original satellite images.

[0048] The first local texture attention module extracts attention features from each of the global structure feature maps to obtain attention feature maps corresponding to each of the original satellite images. The attention feature maps are then input into the next cross-scale structure maintenance block until they pass through the last local texture attention module. All attention feature maps obtained after passing through the last local texture attention module are used as target satellite feature maps.

[0049] Preferably, the number of the cross-scale structure maintenance block and the number of the local texture attention module are both 15.

[0050] It should be understood that the extracted low-level satellite feature map (i.e., the low-level satellite feature map) is sent to the network backbone (i.e., the training model) composed of the cross-scale structure maintenance block and the local texture attention module to obtain a set of refined feature maps (i.e., the target satellite feature map).

[0051] It should be understood that, for example, the order is: cross-scale structure maintenance block, local texture attention module, cross-scale structure maintenance block and local texture attention module, with the cross-scale structure maintenance block ranked first as the first cross-scale structure maintenance block, the local texture attention module ranked second as the first local texture attention module, the cross-scale structure maintenance block ranked third as the next cross-scale structure maintenance block, and the local texture attention module ranked fourth as the next local texture attention module.

[0052] Specifically, the backbone of the network (i.e. the training model) is composed of 15 cross-scale structure maintenance blocks and local texture attention modules stacked together. The stacking mode is that one cross-scale structure maintenance block and one local texture attention module are stacked in parallel, and then the 15 groups are connected in series.

[0053] It should be understood that the cross-scale structure preservation block extracts multi-scale global structural information in the up-dimensional space of the image, while the local texture attention module obtains finer local texture details as a supplement.

[0054] In the above embodiments, the target satellite feature map is obtained by extracting and analyzing the features of each low-level satellite feature map through training model, extracting multi-scale global structural information, and obtaining more refined local texture details.

[0055] Optionally, as an embodiment of the present invention, the cross-scale structure maintenance block includes a second 3×3 convolutional layer, a feature fusion layer, and a first 1×1 convolutional layer;

[0056] The process of extracting global structural features from each of the low-level satellite feature maps using the first cross-scale structure maintenance block to obtain global structural feature maps corresponding to each of the original satellite images includes:

[0057] The second 3×3 convolutional layer is used to perform a second feature extraction on each of the low-level satellite feature maps to obtain satellite feature maps after the second feature extraction corresponding to each of the original satellite images.

[0058] The feature fusion layer performs feature fusion analysis on each of the satellite feature maps after the second feature extraction to obtain a fused satellite feature map corresponding to each of the original satellite images.

[0059] The first 1×1 convolutional layer performs dimensionality reduction on each of the fused satellite feature maps to obtain a global structural feature map corresponding to each of the original satellite images.

[0060] It should be understood that, given an input vector (i.e., the low-level satellite feature map), features are first extracted through a 3×3 convolutional layer (i.e., the second 3×3 convolutional layer), then through the feature fusion layer, and then through a 1×1 convolutional layer (i.e., the first 1×1 convolutional layer) for dimensionality reduction to obtain the output result (i.e., the global structure feature map).

[0061] In the above embodiments, the global structural feature map is obtained by extracting the global structural features of the low-level satellite feature map through the first cross-scale structure maintenance block. Multi-scale global structural information is extracted, and end-to-end mapping between low-resolution satellite images and high-resolution satellite images is realized, thereby generating higher-quality high-resolution satellite images.

[0062] Optionally, as an embodiment of the present invention, the feature fusion layer includes a first upsampling layer, a first downsampling layer, a second upsampling layer, a second downsampling layer, a third upsampling layer, a third downsampling layer, a third 3×3 convolutional layer, and a connection layer;

[0063] The process of performing feature fusion analysis on each of the satellite feature maps after the second feature extraction through the feature fusion layer to obtain the fused satellite feature map corresponding to each of the original satellite images includes:

[0064] The satellite feature maps after the second feature extraction are upsampled by the first upsampling layer and the preset first scale respectively to obtain the first upsampled satellite feature map corresponding to each of the original satellite images;

[0065] The first downsampling layer and the preset first scale are used to downsample each of the first upsampled satellite feature maps to obtain the first downsampled satellite feature map corresponding to each of the original satellite images.

[0066] The satellite feature maps after the second feature extraction are upsampled by the second upsampling layer and the preset second scale respectively to obtain the second upsampled satellite feature maps corresponding to each of the original satellite images;

[0067] The second downsampling layer and the preset second scale are used to downsample each of the second upsampled satellite feature maps to obtain the second downsampled satellite feature maps corresponding to each of the original satellite images.

[0068] The satellite feature maps after the second feature extraction are upsampled by the third upsampling layer and the preset third scale to obtain the third upsampled satellite feature maps corresponding to the original satellite images.

[0069] The satellite feature maps after third upsampling are downsampled by the third downsampling layer and the preset second scale respectively to obtain the satellite feature maps after third downsampling corresponding to each of the original satellite images;

[0070] The third 3×3 convolutional layer performs a third feature extraction on each of the first downsampled satellite feature maps, each of the second downsampled satellite feature maps, and each of the third downsampled satellite feature maps, to obtain a first satellite feature map to be added corresponding to each of the first downsampled satellite feature maps, a second satellite feature map to be added corresponding to each of the second downsampled satellite feature maps, and a third satellite feature map to be added corresponding to each of the third downsampled satellite feature maps.

[0071] The connection layer performs feature fusion on each of the satellite feature maps after the second feature extraction, the first satellite feature map to be added corresponding to each of the original satellite images, the second satellite feature map to be added corresponding to each of the original satellite images, and the third satellite feature map to be added corresponding to each of the original satellite images, to obtain the fused satellite feature map corresponding to each of the original satellite images.

[0072] Preferably, the preset first scale, the preset second scale, and the preset third scale are 2 times the scale, 4 times the scale, and 8 times the scale, respectively.

[0073] It should be understood that the upsampling process involves sampling analog signals. Sampling converts a signal that is continuous in both time and amplitude into a discrete signal in both time and amplitude under the action of sampling pulses. Therefore, sampling is also called the discretization process of waveforms. Upsampling is the process of resampling a digital signal. The sampling rate of the resampling is compared with the original sampling rate of the digital signal (e.g., sampled from an analog signal). If the resampling rate is greater than the original sampling rate, it is called upsampling. The essence of upsampling is interpolation or interpolation.

[0074] It should be understood that the downsampling process, also known as decimation, is one of the fundamental aspects of multi-rate signal processing. Downsampling can bring many corresponding benefits in different applications. Taking the most common digital receiver as an example, the final baseband signal sampling rate is equal to the symbol rate. This rate is relatively low, but the usual practice is not to directly sample the analog signal at this sampling rate, but to use a much higher sampling rate (tens or even hundreds of times higher). This improves the signal-to-noise ratio (SNR) of the sampled signal. Then, digital methods are used to perform multiple stages of filtering and decimation on the signal until the final signal sampling rate equals the symbol rate. The SNR gain obtained through this process is the ratio of the initial sampling rate to the final output signal sampling rate.

[0075] It should be understood that the cross-scale structure maintenance module consists of three parallel upsampling and downsampling units, where 2×, 4×, and 8× represent the upsampling and downsampling scales, respectively. The main purpose of the upsampling and downsampling units proposed in this invention is to extract rich multi-scale global features. It is worth noting that the upsampling and downsampling scales can be set by the user (in this invention, they are set to 2x, 4x, and 8x), with the aim of extracting global structural features at different scales without changing the convolution kernel size.

[0076] Specifically, the input features (i.e., the satellite feature map after the second feature extraction) are subjected to feature upsampling and downsampling at a scale of 2x, 4x, and 8x. The resulting three downsampled features of different sizes (i.e., the first downsampled satellite feature map, the second downsampled satellite feature map, and the third downsampled satellite feature map) are then further processed by a 3x3 convolutional layer (i.e., the third 3x3 convolutional layer) to extract more features. These features are then element-wise added to the input vector (i.e., the satellite feature map after the second feature extraction) to obtain the final output result (i.e., the fused satellite feature map). After obtaining three sets of output features at different scales (i.e., the first downsampled satellite feature map, the second downsampled satellite feature map, and the third downsampled satellite feature map), multi-scale feature fusion is performed through the connection layer.

[0077] In the above embodiments, the feature fusion layer performs feature fusion analysis on the satellite feature map after the second feature extraction to obtain the fused satellite feature map, which can extract rich multi-scale global features and extract global structural features at different scales without changing the size of the convolution kernel.

[0078] Optionally, as an embodiment of the present invention, the local texture attention module includes a second 1×1 convolutional layer, a RuLU activation function layer, a Sigmoid activation function layer, and a plurality of fourth 3×3 convolutional layers arranged in sequence;

[0079] The process of extracting attention features from each of the global structural feature maps using the first local texture attention module to obtain attention feature maps corresponding to each of the original satellite images includes:

[0080] The second 1×1 convolutional layer performs a fourth feature extraction on each of the global structural feature maps to obtain the feature-extracted global structural feature maps corresponding to each of the original satellite images.

[0081] The fourth 3×3 convolutional layer performs a fifth feature extraction on each of the global structural feature maps after feature extraction, thereby obtaining a global structural feature map to be mapped corresponding to each of the original satellite images.

[0082] The RuLU activation function layer is used to perform the first mapping process on each of the global structural feature maps to be mapped, so as to obtain the mapped global structural feature maps corresponding to each of the original satellite images.

[0083] The Sigmoid activation function layer performs a second mapping process on each of the mapped global structural feature maps to obtain attention feature maps corresponding to each of the original satellite images.

[0084] Preferably, the number of the fourth 3×3 convolutional layers is 2.

[0085] Specifically, the local texture attention module consists of convolutional layers with different kernel sizes and sigmoid layers. It starts with a 1×1 convolutional layer (i.e., the second 1×1 convolutional layer) to reduce the vector dimension, making the entire block very lightweight. Then, the invention uses a 3×3 convolutional layer with a stride of 2 (the fourth 3×3 convolutional layer) to expand the receptive field, rapidly reducing the spatial dimension at the start of the network. Subsequently, this invention employs a gating mechanism similar to channel attention to capture local texture details. This is because information in low-resolution space has rich low-frequency and high-frequency components. The low-frequency components are usually flat, while the high-frequency components are usually full of details, such as edges and textures. On the other hand, each filter in the convolutional layer operates using a local receptive field. Therefore, local texture detail information can be obtained in this way. Finally, a 1×1 convolutional layer (i.e., the second 1×1 convolutional layer) is used to recover the vector dimension. Then, an attention feature (i.e., the attention feature map) is generated through a Sigmoid layer (i.e., the Sigmoid activation function layer). Before dimensionality reduction, this invention also uses residual concatenation to directly forward the input features to the end of the block.

[0086] In the above embodiments, attention feature maps are obtained by extracting attention features from the global structure feature map through the first local texture attention module, which expands the receptive field, quickly reduces the spatial dimension at the beginning of the network, and captures local texture details.

[0087] Optionally, as an embodiment of the present invention, the reconstruction model includes a convolutional layer and a fourth upsampling layer.

[0088] The process of constructing a reconstruction model and reconstructing images of each of the target satellite feature maps using the reconstruction model to obtain super-resolution satellite images corresponding to each of the original satellite images includes:

[0089] The convolutional layer performs a sixth feature extraction on each of the target satellite feature maps to obtain the target satellite feature maps after feature extraction corresponding to each of the original satellite images.

[0090] The fourth upsampling layer performs upsampling processing on the feature maps of each target satellite after feature extraction to obtain super-resolution satellite images corresponding to each original satellite image.

[0091] It should be understood that the reconstruction module (i.e., the reconstruction model) is composed of a convolutional layer and an upsampling layer (i.e., the fourth upsampling layer) connected in series. The refined satellite image feature map (i.e., the target satellite feature map) passes through the convolutional layer and the upsampling layer (i.e., the fourth upsampling layer) in sequence to obtain the reconstructed high-resolution satellite image (i.e., the super-resolution satellite image).

[0092] It should be understood that the reconstruction module (i.e., the reconstruction model) is used to reconstruct the refined satellite image feature map (i.e., the target satellite feature map), generating a corresponding high-resolution satellite image (i.e., the super-resolution satellite image) from the low-resolution satellite image. By reconstructing the global structure and local texture of the image, global feature fusion can be achieved while maximizing the utilization of local texture information.

[0093] In the above embodiments, the sixth feature extraction of each target satellite feature map is performed by a convolutional layer to obtain the target satellite feature map after feature extraction. The fourth upsampling layer is used to upsample each target satellite feature map after feature extraction to obtain a super-resolution satellite image. This achieves the fusion of global features and maximizes the use of local texture information.

[0094] Optionally, as another embodiment of the present invention, the present invention achieves end-to-end mapping between low-resolution satellite images and high-resolution satellite images. The present invention can generate higher quality high-resolution satellite images. At the same time, the present invention aims to solve the technical problem that the image quality is not high when generating high-resolution satellite images from low-resolution satellite images.

[0095] Optionally, as another embodiment of the present invention, the experimental results obtained using the present invention are compared with those of other satellite image super-resolution algorithms under the same conditions, as shown in Table 1.

[0096] Table 1 shows examples of the experimental results of this invention compared with other image super-resolution algorithms.

[0097] algorithm Bicubic EDSR SRResNet RCAN MSRN MHAN EEGAN SeaNet Ours Parameter quantity / M --- 80.86 2.73 15.59 6.08 11.35 6.53 7.24 7.25 FLOPs / M --- 376.78 13.29 36.77 12.65 26.10 18.47 20.51 15.21 PSNR / dB 30.84 32.41 31.68 33.62 33.57 33.14 33.17 33.29 34.08 SSIM 0.8217 0.8599 0.8441 0.8919 0.8877 0.8819 0.8781 0.8907 0.8975 FISM 0.8481 0.8859 0.8721 0.9126 0.9078 0.9035 0.9021 0.9089 0.9153 VIF 0.4223 0.4736 0.4443 0.5195 0.5221 0.5002 0.4985 0.5164 0.5367 ERGAS 1.3906 1.1572 1.2586 0.9884 1.0061 1.0549 1.0543 1.0121 0.9431 LPIPS 0.3749 0.2756 0.3178 0.1874 0.1976 0.2160 0.2266 0.1929 0.1869

[0098] As can be seen from Table 1, the present invention achieved higher scores than the comparative algorithms in PSNR, SSIM, FISM, VIF, ERGAS, and LPIPS, and outperformed the comparative algorithms in all of them.

[0099] Alternatively, as another embodiment of the present invention, such as Figure 2As shown, the hardware structure of the present invention includes a processor 1001 (e.g., a Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to implement communication between these components; the user interface 1003 may include a display screen or an input unit such as a keyboard; the network interface 1004 may optionally include a standard wired interface or a wireless interface (e.g., Wireless Fidelity, Wi-Fi); the memory 1005 may be high-speed random access memory (RAM) or stable memory (non-volatile memory), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art will understand that… Figure 2 The hardware structure shown does not constitute a limitation of the invention and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0100] Figure 3 This is a block diagram of a satellite super-resolution image generation device provided in an embodiment of the present invention.

[0101] Alternatively, as another embodiment of the present invention, such as Figure 3 As shown, a satellite super-resolution image generation device includes:

[0102] The image preprocessing module is used to import multiple raw satellite images, perform image preprocessing on each of the raw satellite images respectively, and obtain a low-resolution satellite image corresponding to each of the raw satellite images;

[0103] The feature extraction module is used to perform initial feature extraction on each of the low-resolution satellite images to obtain a low-level satellite feature map corresponding to each of the original satellite images;

[0104] The feature extraction and analysis module is used to construct a training model, and to perform feature extraction and analysis on each of the low-level satellite feature maps through the training model to obtain target satellite feature maps corresponding to each of the original satellite images.

[0105] The image generation result acquisition module is used to construct a reconstruction model, and to reconstruct the feature maps of each target satellite using the reconstruction model to obtain super-resolution satellite images corresponding to each original satellite image, and to use all the super-resolution satellite images as the image generation result.

[0106] Optionally, another embodiment of the present invention provides a satellite super-resolution image generation apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the satellite super-resolution image generation method as described above. This apparatus may be a computer or similar device.

[0107] Optionally, another embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the satellite super-resolution image generation method as described above.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0109] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0112] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating satellite super-resolution images, characterized in that, Includes the following steps: Import multiple raw satellite images, and perform image preprocessing on each of the raw satellite images to obtain low-resolution satellite images corresponding to each of the raw satellite images; Each of the low-resolution satellite images is subjected to initial feature extraction to obtain a low-level satellite feature map corresponding to each of the original satellite images; A training model is constructed, and feature extraction and analysis are performed on each of the low-level satellite feature maps using the training model to obtain target satellite feature maps corresponding to each of the original satellite images. A reconstruction model is constructed, and image reconstruction is performed on the feature maps of each target satellite using the reconstruction model to obtain super-resolution satellite images corresponding to each original satellite image. All super-resolution satellite images are used as the image generation result. The training model includes multiple cross-scale structure maintenance blocks and multiple local texture attention modules, wherein the number of cross-scale structure maintenance blocks and local texture attention modules is the same and they are arranged alternately. The process of constructing a training model and performing feature extraction and analysis on each of the low-level satellite feature maps using the training model to obtain target satellite feature maps corresponding to each of the original satellite images includes: The global structural features of each of the low-level satellite feature maps are extracted by the first cross-scale structure maintenance block to obtain the global structural feature map corresponding to each of the original satellite images. The first local texture attention module extracts attention features from each of the global structure feature maps to obtain attention feature maps corresponding to each of the original satellite images. The attention feature maps are then input into the next cross-scale structure maintenance block until they pass through the last local texture attention module. All attention feature maps obtained after passing through the last local texture attention module are used as target satellite feature maps.

2. The satellite super-resolution image generation method according to claim 1, characterized in that, The process of performing image preprocessing on each of the original satellite images to obtain low-resolution satellite images corresponding to each of the original satellite images includes: Each of the original satellite images is cropped to obtain a cropped satellite image corresponding to each of the original satellite images; The cropped satellite images are downsampled for the first time using a bicubic interpolation algorithm to obtain low-resolution satellite images corresponding to the original satellite images.

3. The satellite super-resolution image generation method according to claim 1, characterized in that, The process of performing initial feature extraction on each of the low-resolution satellite images to obtain low-level satellite feature maps corresponding to each of the original satellite images includes: Based on the first 3×3 convolutional layer, the low-resolution satellite images are subjected to initial feature extraction to obtain low-level satellite feature maps corresponding to each of the original satellite images.

4. The satellite super-resolution image generation method according to claim 1, characterized in that, The cross-scale structure maintenance block includes a second 3×3 convolutional layer, a feature fusion layer, and a first 1×1 convolutional layer; The process of extracting global structural features from each of the low-level satellite feature maps using the first cross-scale structure maintenance block to obtain global structural feature maps corresponding to each of the original satellite images includes: The second 3×3 convolutional layer is used to perform a second feature extraction on each of the low-level satellite feature maps to obtain satellite feature maps after the second feature extraction corresponding to each of the original satellite images. The feature fusion layer performs feature fusion analysis on each of the satellite feature maps after the second feature extraction to obtain a fused satellite feature map corresponding to each of the original satellite images. The first 1×1 convolutional layer performs dimensionality reduction on each of the fused satellite feature maps to obtain a global structural feature map corresponding to each of the original satellite images.

5. The satellite super-resolution image generation method according to claim 4, characterized in that, The feature fusion layer includes a first upsampling layer, a first downsampling layer, a second upsampling layer, a second downsampling layer, a third upsampling layer, a third downsampling layer, a third 3×3 convolutional layer, and a connection layer; The process of performing feature fusion analysis on each of the satellite feature maps after the second feature extraction through the feature fusion layer to obtain the fused satellite feature map corresponding to each of the original satellite images includes: The satellite feature maps after the second feature extraction are upsampled by the first upsampling layer and the preset first scale respectively to obtain the first upsampled satellite feature map corresponding to each of the original satellite images; The first downsampling layer and the preset first scale are used to downsample each of the first upsampled satellite feature maps to obtain the first downsampled satellite feature map corresponding to each of the original satellite images. The satellite feature maps after the second feature extraction are upsampled by the second upsampling layer and the preset second scale respectively to obtain the second upsampled satellite feature maps corresponding to each of the original satellite images; The second downsampling layer and the preset second scale are used to downsample each of the second upsampled satellite feature maps to obtain the second downsampled satellite feature maps corresponding to each of the original satellite images. The satellite feature maps after the second feature extraction are upsampled by the third upsampling layer and the preset third scale to obtain the third upsampled satellite feature maps corresponding to the original satellite images. The satellite feature maps after third upsampling are downsampled by the third downsampling layer and the preset second scale respectively to obtain the satellite feature maps after third downsampling corresponding to each of the original satellite images; The third 3×3 convolutional layer performs a third feature extraction on each of the first downsampled satellite feature maps, each of the second downsampled satellite feature maps, and each of the third downsampled satellite feature maps, to obtain a first satellite feature map to be added corresponding to each of the first downsampled satellite feature maps, a second satellite feature map to be added corresponding to each of the second downsampled satellite feature maps, and a third satellite feature map to be added corresponding to each of the third downsampled satellite feature maps. The connection layer performs feature fusion on each of the satellite feature maps after the second feature extraction, the first satellite feature map to be added corresponding to each of the original satellite images, the second satellite feature map to be added corresponding to each of the original satellite images, and the third satellite feature map to be added corresponding to each of the original satellite images, to obtain the fused satellite feature map corresponding to each of the original satellite images.

6. The satellite super-resolution image generation method according to claim 1, characterized in that, The local texture attention module includes a second 1×1 convolutional layer, a RuLU activation function layer, a Sigmoid activation function layer, and multiple fourth 3×3 convolutional layers arranged in sequence; The process of extracting attention features from each of the global structural feature maps using the first local texture attention module to obtain attention feature maps corresponding to each of the original satellite images includes: The second 1×1 convolutional layer performs a fourth feature extraction on each of the global structural feature maps to obtain the feature-extracted global structural feature maps corresponding to each of the original satellite images. The fourth 3×3 convolutional layer performs a fifth feature extraction on each of the global structural feature maps after feature extraction, thereby obtaining a global structural feature map to be mapped corresponding to each of the original satellite images. The RuLU activation function layer is used to perform the first mapping process on each of the global structural feature maps to be mapped, so as to obtain the mapped global structural feature maps corresponding to each of the original satellite images. The Sigmoid activation function layer performs a second mapping process on each of the mapped global structural feature maps to obtain attention feature maps corresponding to each of the original satellite images.

7. The satellite super-resolution image generation method according to claim 1, characterized in that, The reconstruction model includes convolutional layers and a fourth upsampling layer. The process of constructing a reconstruction model and reconstructing images of each of the target satellite feature maps using the reconstruction model to obtain super-resolution satellite images corresponding to each of the original satellite images includes: The convolutional layer performs a sixth feature extraction on each of the target satellite feature maps to obtain the target satellite feature maps after feature extraction corresponding to each of the original satellite images. The fourth upsampling layer performs upsampling processing on the feature maps of each target satellite after feature extraction to obtain super-resolution satellite images corresponding to each original satellite image.

8. A satellite super-resolution image generation device, characterized in that, include: The image preprocessing module is used to import multiple raw satellite images, perform image preprocessing on each of the raw satellite images respectively, and obtain a low-resolution satellite image corresponding to each of the raw satellite images; The feature extraction module is used to perform initial feature extraction on each of the low-resolution satellite images to obtain a low-level satellite feature map corresponding to each of the original satellite images; The feature extraction and analysis module is used to construct a training model, and to perform feature extraction and analysis on each of the low-level satellite feature maps through the training model to obtain target satellite feature maps corresponding to each of the original satellite images. The image generation result acquisition module is used to construct a reconstruction model, and to perform image reconstruction on each of the target satellite feature maps through the reconstruction model to obtain super-resolution satellite images corresponding to each of the original satellite images, and to use all the super-resolution satellite images as the image generation result. The training model includes multiple cross-scale structure maintenance blocks and multiple local texture attention modules, wherein the number of cross-scale structure maintenance blocks and local texture attention modules is the same and they are arranged alternately. The feature extraction and analysis module is specifically used for: The global structural features of each of the low-level satellite feature maps are extracted by the first cross-scale structure maintenance block to obtain the global structural feature map corresponding to each of the original satellite images. The first local texture attention module extracts attention features from each of the global structure feature maps to obtain attention feature maps corresponding to each of the original satellite images. The attention feature maps are then input into the next cross-scale structure maintenance block until they pass through the last local texture attention module. All attention feature maps obtained after passing through the last local texture attention module are used as target satellite feature maps.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the satellite super-resolution image generation method as described in any one of claims 1 to 7.

Images