A method, system, device and storage medium for super-resolution reconstruction of remote sensing images

Abstract

The present invention relates to the technical field of image reconstruction, and specifically to a super-resolution reconstruction method, system, device and storage medium for remote sensing images. To solve the technical problem of poor effect of low-resolution remote sensing image reconstruction methods in the prior art, first, the remote sensing image to be processed is radiometrically normalized and mapped into a high-dimensional feature space, and then regional enhancement processing is performed. The radiometric high-dimensional enhanced feature map is dimensionally reshaped to obtain a neighborhood-related feature map and a long-distance dependence feature map. After fusion, feature mapping error correction is performed to obtain a spatial information reconstruction feature map. Then, different scale feature maps of the spatial information reconstruction feature map are respectively subjected to image enhancement processing and then cross-scale fusion to obtain a multi-scale fusion feature map. Finally, a static branch feature map and a dynamic branch feature map of the multi-scale fusion feature map are obtained. After fusion, a super-resolution reconstruction map is obtained, improving the quality and stability of super-resolution reconstruction of remote sensing images.

Description

Technical Field

[0001] The present invention relates to the technical field of image reconstruction, and particularly to a method, system, device and storage medium for super-resolution reconstruction of remote sensing images. Background Art

[0002] With the continuous development of remote sensing technology, remote sensing images have been widely used in fields such as geographic information systems, agriculture, and urban planning. However, due to limitations in acquisition conditions, such as sensor resolution, sampling distance, etc., the resolution of remote sensing images is low, which affects their application in precise analysis and detail extraction. To overcome this problem, super-resolution reconstruction technology has emerged, aiming to restore images with higher spatial resolution by processing low-resolution remote sensing images.

[0003] Existing methods for super-resolution reconstruction of remote sensing images are mostly based on interpolation techniques or deep learning methods, but these methods generally have the following deficiencies: interpolation methods are prone to causing image blurring and cannot effectively restore image details; deep learning-based methods usually rely on a large amount of labeled data for training and lack an effective multi-scale information fusion mechanism. Especially when dealing with complex ground objects, the image reconstruction effect is poor and unstable. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, system, device and storage medium for super-resolution reconstruction of remote sensing images.

[0005] The technical solution of the present invention is as follows:

[0006] A method for super-resolution reconstruction of remote sensing images includes the following operations:

[0007] S1. The remote sensing image to be processed is subjected to radiometric normalization processing to compensate for image radiometric distortion, obtaining a radiometrically normalized image; the radiometrically normalized image is subjected to channel space projection processing to map the radiometrically normalized image into a high-dimensional feature space, obtaining a radiometric high-dimensional feature image; the radiometric high-dimensional feature image is subjected to regional enhancement processing to obtain a radiometric high-dimensional enhanced feature image;

[0008] S2. After dimension reshaping of the radiometric high-dimensional enhanced feature image, a neighborhood correlation feature image and a long-distance dependence feature image are obtained. After fusion, a context-aware feature image is obtained; based on the pixel position confidence of the context-aware feature image, feature mapping error correction is performed on the context-aware feature image to obtain a spatial information reconstruction feature image;

[0009] S3. The spatially informative reconstructed feature map is processed by pixel-level image enhancement to obtain the first-scale feature map; the spatially informative reconstructed feature map is downsampled at the first scale and then processed by state-space image enhancement to obtain the second-scale feature map; the spatially informative reconstructed feature map is downsampled at the second scale and then processed by global and local image enhancement to obtain the third-scale feature map; the third-scale feature map is fused with the second-scale feature map and the first-scale feature map through cross-scale fusion to obtain the multi-scale fused feature map.

[0010] S4. The multi-scale fused feature map is processed by Fourier transform with a fixed convolution kernel to obtain the static branch feature map; after obtaining the degradation descriptor features of the multi-scale fused feature map, convolutional features and channel features are extracted and fused to obtain the dynamic branch feature map; the dynamic branch feature map is fused with the static branch feature map to obtain the super-resolution reconstructed map.

[0011] The operation of radiation normalization in S1 is specifically as follows: the remote sensing image to be processed is processed by feature convolution to obtain the remote sensing convolution image to be processed; based on the mean, or / and variance, or / and shadow ratio of the remote sensing convolution image to be processed, a light description feature map is obtained; the light description feature map is processed by a multi-layer perceptron and smoothed to obtain the smoothed remote sensing image to be processed; after the smoothed remote sensing image to be processed is processed by channel attention, it is added element-wise to the smoothed remote sensing image to be processed to obtain the radiation normalization map.

[0012] The operation of region enhancement in S1 is specifically as follows: the radiation high-dimensional feature map is processed by convolution with different convolution kernels and dilated convolution respectively to obtain a number of convolutional features, which are cross-fused and then fused to obtain the total fused feature; the total fused feature is processed by non-linear processing and a multi-layer perceptron to obtain the radiation high-dimensional enhanced feature map.

[0013] The operation of dimension reshaping in S2 is specifically as follows: based on the spatial importance map of the radiation high-dimensional enhanced feature map, a recombination matrix is obtained; based on the recombination matrix, the offset correction is performed on each corresponding position in the radiation high-dimensional enhanced feature map to obtain the reshaped feature map.

[0014] The operation of obtaining the spatially informative reconstructed feature map in S2 is specifically as follows: the confidence of each pixel position in the context-aware feature map is obtained, and the pixel positions with confidence less than the confidence threshold are used as the positions to be corrected; the mapping error compensation is performed on the corresponding positions of the positions to be corrected in the context-aware feature map to obtain the spatially informative reconstructed feature map.

[0015] The operation of state space image enhancement processing in S3 is specifically as follows: Based on the state transition matrix and channel feature map of the first downsampled map of spatial information, and a preset state space equation, perform state vector enhancement processing on each channel of the first downsampled map of spatial information to obtain a second-scale feature map; the first downsampled map of spatial information is obtained by performing first-scale downsampling processing on the reconstructed feature map of spatial information.

[0016] The operation of global and local image enhancement processing in S3 is specifically as follows: The second downsampled map of spatial information is processed by a one-dimensional convolutional gated unit to obtain a globally downsampled map of spatial information; the globally downsampled map of spatial information is divided into several feature map blocks, and within each small block, according to the state transition matrix and projection matrix of the feature map block, and a preset state space equation, several locally enhanced blocks of the feature map are obtained; all the locally enhanced blocks of the feature map are processed by fusion to obtain a third-scale feature map; the second downsampled map of spatial information is obtained by performing second-scale downsampling processing on the reconstructed feature map of spatial information.

[0017] A remote sensing image super-resolution reconstruction system for implementing the above remote sensing image super-resolution reconstruction method, including:

[0018] A radiation high-dimensional enhanced feature map generation module, which is used to perform radiation normalization processing on the remote sensing image to be processed, compensate for image radiation distortion, and obtain a radiation-normalized map; the radiation-normalized map is processed by channel space projection to map the radiation-normalized map into a high-dimensional feature space to obtain a radiation high-dimensional feature map; the radiation high-dimensional feature map is processed by regional enhancement to obtain a radiation high-dimensional enhanced feature map;

[0019] A spatial information reconstructed feature map generation module, which is used to reshape the dimensions of the radiation high-dimensional enhanced feature map, obtain a neighborhood-related feature map and a long-distance dependence feature map, and after fusion, obtain a context-aware feature map; based on the pixel position confidence of the context-aware feature map, perform feature mapping error correction on the context-aware feature map to obtain a spatial information reconstructed feature map;

[0020] A multi-scale fusion feature map generation module, which is used to perform pixel-level image enhancement processing on the spatial information reconstructed feature map to obtain a first-scale feature map; perform state space image enhancement processing on the spatial information reconstructed feature map after first-scale downsampling to obtain a second-scale feature map; perform global and local image enhancement processing on the spatial information reconstructed feature map after second-scale downsampling to obtain a third-scale feature map; the third-scale feature map is processed by cross-scale fusion with the second-scale feature map and the first-scale feature map to obtain a multi-scale fusion feature map;

[0021] The super-resolution reconstruction map generation module is used to perform Fourier transform processing on the multi-scale fusion feature map with a fixed convolution kernel to obtain a static branch feature map; after obtaining the degradation descriptor features of the multi-scale fusion feature map, convolutional features and channel features are extracted, and after fusion, a dynamic branch feature map is obtained; the dynamic branch feature map is fused with the static branch feature map to obtain a super-resolution reconstruction map.

[0022] A remote sensing image super-resolution reconstruction device includes a processor and a memory. Among them, when the processor executes the computer program stored in the memory, the above-mentioned remote sensing image super-resolution reconstruction method is implemented.

[0023] A computer-readable storage medium is used to store a computer program. Among them, when the computer program is executed by a processor, the above-mentioned remote sensing image super-resolution reconstruction method is implemented.

[0024] The beneficial effects of the present invention are as follows:

[0025] A remote sensing image super-resolution reconstruction method provided by the present invention. First, the remote sensing image to be processed is radiometrically normalized to identify the image content and compensate for the radiometric distortion of the sensor, obtaining a radiometrically normalized map; and the radiometrically normalized map is mapped to a high-dimensional feature space to enhance the local feature response of the image, thereby enhancing the modeling ability for the irregular object distribution in the remote sensing image, obtaining a radiometric high-dimensional feature map; the radiometric high-dimensional feature map is subjected to regional enhancement processing to obtain a radiometric high-dimensional enhanced feature map, highlighting important regions and retaining background information, providing a richer and more complete feature representation for subsequent global modeling; then, the radiometric high-dimensional enhanced feature map is dimensionally reshaped, and after retaining the spatial topological relationship between feature channels, a neighborhood-related feature map and a long-range dependence feature map are obtained. After fusion, a context-aware feature map is obtained; and based on the pixel position confidence of the context-aware feature map, the feature mapping error of the context-aware feature map is corrected to restore the image structure, obtaining a spatial information reconstruction feature map; then, different scale feature maps of the spatial information reconstruction feature map are respectively subjected to pixel-level image enhancement processing, state space image enhancement processing, and global and local image enhancement processing to obtain the original detail information of the image, global semantic information, and local context relationship information. After cross-scale fusion, the priority transmission of detail and semantic information is ensured, and finally the reconstruction quality of the image is improved, obtaining a multi-scale fusion feature map; finally, the static branch feature map and the dynamic branch feature map of the multi-scale fusion feature map are obtained, and after fusion, a super-resolution reconstruction map is obtained, improving the quality and reconstruction stability of the remote sensing image super-resolution reconstruction result. Description of the Drawings

[0026] The solutions and advantages of the present application will become clear to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention.

[0027] In the drawings:

[0028] Figure 1 is the super-resolution reconstruction effect diagram obtained by processing the airport remote sensing image using the method of this embodiment; in Figure 1 in, (a) is the original airport remote sensing image, and (b) is the reconstruction effect diagram.

[0029] Figure 2 is the super-resolution reconstruction effect diagram obtained by processing the urban remote sensing image using the method of this embodiment; in Figure 2 in, (a) is the original urban remote sensing image, and (b) is the reconstruction effect diagram. Specific Embodiments

[0030] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the drawings.

[0031] This embodiment provides a method for super-resolution reconstruction of remote sensing images, including the following operations:

[0032] S1. The remote sensing image to be processed is subjected to radiometric normalization processing to compensate for the radiometric distortion of the image, obtaining a radiometrically normalized image; the radiometrically normalized image is subjected to channel space projection processing to map the radiometrically normalized image into a high-dimensional feature space, obtaining a radiometric high-dimensional feature map; the radiometric high-dimensional feature map is subjected to regional enhancement processing to obtain a radiometric high-dimensional enhanced feature map;

[0033] S2. After dimension reshaping of the radiometric high-dimensional enhanced feature map, a neighborhood-related feature map and a long-range dependence feature map are obtained. After fusion, a context-aware feature map is obtained; based on the pixel position confidence of the context-aware feature map, feature mapping error correction is performed on the context-aware feature map to obtain a spatial information reconstruction feature map;

[0034] S3. The spatial information reconstruction feature map is subjected to pixel-level image enhancement processing to obtain a first-scale feature map; the spatial information reconstruction feature map is subjected to state space image enhancement processing after being downsampled at the first scale to obtain a second-scale feature map; the spatial information reconstruction feature map is subjected to global and local image enhancement processing after being downsampled at the second scale to obtain a third-scale feature map; the third-scale feature map is subjected to cross-scale fusion processing with the second-scale feature map and the first-scale feature map to obtain a multi-scale fusion feature map;

[0035] S4. The multi-scale fusion feature map is processed by Fourier transform with a fixed convolution kernel to obtain a static branch feature map. After obtaining the degradation descriptor features of the multi-scale fusion feature map, convolutional features and channel features are extracted and fused to obtain a dynamic branch feature map. The dynamic branch feature map is fused with the static branch feature map to obtain a super-resolution reconstruction map. The specific operation details are as follows.

[0036] S1. The remote sensing image to be processed is subjected to radiometric normalization to compensate for the radiometric distortion of the image, obtaining a radiometrically normalized map. The radiometrically normalized map is subjected to channel space projection to map the radiometrically normalized map into a high-dimensional feature space, obtaining a radiometric high-dimensional feature map. The radiometric high-dimensional feature map is subjected to regional enhancement to obtain a radiometric high-dimensional enhanced feature map.

[0037] The remote sensing image to be processed is radiometrically normalized to automatically identify the image content and compensate for the radiometric distortion of the sensor, obtaining a radiometrically normalized map. Then, the radiometrically normalized map is mapped into a high-dimensional feature space to enhance the local feature response of the image, thereby enhancing the modeling ability for the irregular object distribution in the remote sensing image, obtaining a radiometric high-dimensional feature map. The radiometric high-dimensional feature map is subjected to regional enhancement to obtain a radiometric high-dimensional enhanced feature map, highlighting important regions and retaining background information, providing a richer and more complete feature representation for subsequent global modeling.

[0038] First, the low-resolution remote sensing image to be processed is subjected to radiometric normalization instead of simple global normalization to compensate for the image radiometric distortion of the remote sensing image obtained by the sensor, obtaining a radiometrically normalized map.

[0039] The operation of radiometric normalization is specifically as follows: The remote sensing image to be processed is subjected to feature convolution processing by a CNN convolution network composed of several stacked convolutional blocks (preferably 3 convolutional blocks with a convolution kernel of 3×3) to obtain a remote sensing convolutional image to be processed. Based on the band mean, or / and band variance, or / and shadow ratio of the remote sensing convolutional image to be processed, illumination description features are obtained. Based on the illumination description features, the remote sensing image to be processed is subjected to multi-layer perceptron processing and smoothing processing to achieve sensor characteristic compensation, obtaining a smoothed remote sensing image to be processed. After the smoothed remote sensing image to be processed is subjected to channel attention processing, it is added element-wise to the smoothed remote sensing image to further eliminate non-linear distortion, obtaining a radiometrically normalized map.

[0040] Among them, the operation of obtaining the illumination description features is realized by the following formula:

[0041] ,

[0042] ,

[0043] ,

[0044] ,

[0045] is the feature description of illumination, is the band mean value, is the pixel mean value of the image corresponding to the m th band of the remote sensing convolutional image to be processed, is the band variance of the remote sensing convolutional image to be processed, is the pixel variance of the image corresponding to the m th band of the remote sensing convolutional image to be processed, and is obtained based on ; is the shadow ratio, is the indicator function, which is 1 when the condition holds and 0 when it does not hold; is the image corresponding to the m th band of the remote sensing convolutional image to be processed, is the shadow threshold of the image corresponding to the m th band of the remote sensing convolutional image to be processed, , is the balance coefficient, which is an empirical value, , where H and W are the length and width of the remote sensing convolutional image to be processed respectively.

[0046] Then, the radiometrically normalized map is processed by channel spatial projection to map the radiometrically normalized map into a high-dimensional feature space, obtaining a radiometric high-dimensional feature map.

[0047] The operation of channel spatial projection processing can be implemented by the following formula:

[0048] ,

[0049] is the radiometric high-dimensional feature map, is the convolutional weight, is the GELU activation function, is the radiometrically normalized map, is the bias term of the radiometrically normalized map, is the learnable balance parameter, is the lightweight spatial attention operator, preferably the SimAM network.

[0050] Among them, the bias term of the radiation normalization map is a matrix formed by the bias amounts at each position of the radiation normalization map. The specific method for obtaining the bias amount at each position is as follows: perform grouped convolution (preferably with a convolution kernel of 3 and a grouping number of 4) and ordinary convolution (convolution kernel of 1) on the radiation normalization map to obtain a radiation normalization feature convolution map; obtain the texture complexity and radiation uniformity of the radiation normalization feature convolution map. After normalization processing, multiply them with their respective preset weight matrices respectively, and then perform matrix addition with the Gaussian smoothing matrix of the radiation normalization map to obtain a bias matrix, which is used as the bias term of the radiation normalization map. This method effectively adapts to different images, dynamically generates biases according to the characteristics of image regions, and effectively adjusts the intensity of image feature responses. The texture complexity is obtained by calculating the gray variance of the 3×3 neighborhood of the radiation normalization feature convolution map and is used to quantify the richness of details in the region. The radiation uniformity (illumination uniformity) is the ratio of the minimum illuminance to the average illuminance in the region of interest (which can be set according to actual needs) and reflects the stability of the lighting conditions.

[0051] Finally, the radiation high-dimensional feature map is processed by region enhancement to highlight important regions and retain background information, obtaining a radiation high-dimensional enhanced feature map, which provides a more abundant and complete feature representation for subsequent global modeling.

[0052] The specific operation of the region enhancement processing is as follows: the radiation high-dimensional feature map is respectively processed by convolution with different convolution kernels (preferably using convolutions with convolution kernels of 3×3 and 5×5), and atrous convolution processing to obtain several convolution features. After cross-fusion (fusion processing between different convolution features), fusion processing is performed to generate a total fusion feature; the total fusion feature is processed by non-linear processing (preferably the Softmax function) and a multi-layer perceptron to ensure that while highlighting important regions, the context information of the background environment is retained, obtaining a radiation high-dimensional enhanced feature map, which provides a discriminative and complete feature representation for subsequent processing.

[0053] S2. After reshaping the dimensions of the radiation high-dimensional enhanced feature map, obtain a neighborhood-related feature map and a long-range dependence feature map. After fusion, obtain a context-aware feature map; based on the pixel position confidence of the context-aware feature map, perform feature mapping error correction on the context-aware feature map to obtain a spatial information reconstruction feature map.

[0054] Reshape the dimensions of the radiation high-dimensional enhanced feature map, retain the spatial topological relationship between feature channels, then obtain a neighborhood-related feature map and a long-range dependence feature map. After fusion, obtain a context-aware feature map; and based on the pixel position confidence of the context-aware feature map, perform feature mapping error correction on the context-aware feature map to restore the image structure and obtain a spatial information reconstruction feature map.

[0055] First, reshape the radiometric high-dimensional enhanced feature map. While converting the feature map dimensions from [B, C, H, W] to [B, H×W, C], preserve the spatial topological relationship between the feature channels to obtain the reshaped feature map. The specific operation of dimension reshaping is as follows: Based on the spatial importance map of the radiometric high-dimensional enhanced feature map (which can be obtained by processing the radiometric high-dimensional enhanced feature map through a lightweight CNN network), obtain a learnable recombination matrix (which can be obtained by converting the spatial importance map into a matrix form according to pixel information); Based on the recombination matrix, correct the offset for each corresponding position in the radiometric high-dimensional enhanced feature map to obtain the reshaped feature map.

[0056] Then, obtain the neighborhood-related feature map and the long-range dependence feature map of the reshaped feature map to capture the neighborhood feature correlation at each position in the reshaped feature map and the long-range dependence feature of the reshaped feature map.

[0057] Among them, the neighborhood-related feature map is obtained by processing the reshaped feature map through a gated linear unit (preferably a GLU network) and depthwise separable convolution. The long-range dependence feature can be obtained by processing the reshaped feature map through a Transformer network, or by processing the neighborhood-related feature map through a selective state mechanism.

[0058] The specific operation of the selective state mechanism processing is as follows: The neighborhood-related feature map is processed by a multi-layer perceptron to obtain state parameters; The neighborhood-related feature map is processed by a state space model (SSM). During the processing, according to the comparison result of the state parameters and the corresponding parameter thresholds, corresponding multi-granularity calculation processing is performed to obtain the long-range dependence feature. This dynamic regulation through content-aware means, rather than fixed attenuation, thus improves connectivity when processing long-range structures such as roads in remote sensing images, and saves computational effort in smooth areas such as farmland in remote sensing images compared to traditional methods, and also enhances the robustness to sensor noise.

[0059] Next, fuse the neighborhood-related feature map and the long-range dependence feature map (which can be achieved through weighted summation processing) to obtain the context-aware feature map.

[0060] Finally, based on the pixel position confidence of the context-aware feature map, correct the feature mapping error of the context-aware feature map to ensure the complete reconstruction of spatial information and obtain the spatial information reconstruction feature map.

[0061] The operation of obtaining the spatial information reconstruction feature map is specifically as follows: Obtain the confidence of each pixel position in the context-aware feature map (which can be obtained by processing the context-aware feature map through a lightweight UNet network), and use the pixel positions with confidence less than the confidence threshold as the positions to be corrected; perform mapping error compensation on the corresponding positions of the positions to be corrected in the context-aware feature map to obtain the spatial information reconstruction feature map.

[0062] The mapping error compensation can be achieved through the following formula: F_corrected = F + (1 - conf)×Conv([F,F_rec]), where F_corrected is the spatial information reconstruction feature map, F is the context-aware feature map, conf is the confidence. Low confidence (conf < 0.5) positions indicate possible errors (such as building edges), and high confidence (conf ≥ 0.5) positions indicate reliable regions (such as smooth ground). Conv( ) is the convolution process, and F_rec is the inverse transform map (obtained by performing inverse wavelet transform on the context-aware feature map).

[0063] S3. The spatial information reconstruction feature map is processed by pixel-level image enhancement to obtain the first-scale feature map; the spatial information reconstruction feature map is downsampled at the first scale and then processed by state-space image enhancement to obtain the second-scale feature map; the spatial information reconstruction feature map is downsampled at the second scale and then processed by global and local image enhancement to obtain the third-scale feature map; the third-scale feature map is fused with the second-scale feature map and the first-scale feature map through cross-scale fusion to obtain the multi-scale fusion feature map.

[0064] The different-scale feature maps of the spatial information reconstruction feature map are respectively processed by pixel-level image enhancement, state-space image enhancement, and global and local image enhancement to obtain the original image detail information, global semantic information, and local context relationship information. After cross-scale fusion, the priority transmission of detail and semantic information is ensured, and finally the reconstruction quality of the image is improved to obtain the multi-scale fusion feature map.

[0065] First, the spatial information reconstruction feature map is processed by pixel-level image enhancement to keep the resolution unchanged and capture pixel details, such as road markings in a remote sensing image, to obtain the first-scale feature map. Among them, the operation of pixel-level image enhancement can be achieved through Laplacian sharpening.

[0066] At the same time, the spatial information reconstruction feature map is downsampled at the first scale and then processed by state-space image enhancement to model medium-scale features and capture the regional features between the detail level and the global level, such as the morphological and layout features in a larger range of the image, to obtain the second-scale feature map. The spatial information reconstruction feature map is downsampled at the first scale as follows: The spatial information reconstruction feature map is downsampled by 1 / 2 to obtain the first downsampled map of spatial information.

[0067] The operations of the above state space image enhancement processing are specifically as follows: Based on the state transition matrix and channel feature map of the first downsampled map of spatial information, and a preset state space equation, perform state vector enhancement processing on each channel of the first downsampled map of spatial information (input the state transition matrix and channel feature map into the state space equation for solution) to obtain a second-scale feature map. The state transition matrix of the first downsampled map of spatial information can be obtained by processing the first downsampled map of spatial information with a state space model (SSM), and the channel feature map can be obtained by processing the first downsampled map of spatial information with a channel attention mechanism.

[0068] Next, the spatially reconstructed feature map is subjected to global and local image enhancement processing after being downsampled at the second scale, modeling deep-scale features and extracting semantic information such as ground object distribution and scene categories to obtain a third-scale feature map. The spatially reconstructed feature map is downsampled at the second scale as follows: The spatially reconstructed feature map is downsampled by 1 / 4 to obtain a second downsampled map of spatial information.

[0069] The operations of the above global and local image enhancement processing are specifically as follows: The second downsampled map of spatial information is processed by a one-dimensional convolutional gated unit, and the pixel values of each column (or each row) of the second downsampled map of spatial information are convolved one-dimensionally to capture the local context relationship in that direction, and the convolution result is dynamically adjusted through the gated unit to emphasize the correlation between pixels, obtaining a globally downsampled map of spatial information; The globally downsampled map of spatial information is divided into several feature map blocks, and within each small block, according to the state transition matrix and projection matrix of the feature map block, and a preset state space equation, the pixel information of the feature block is dynamically adjusted to obtain several locally enhanced blocks of feature maps; All the locally enhanced blocks of feature maps are fused to obtain a third-scale feature map.

[0070] Finally, the third-scale feature map is fused with the second-scale feature map and the first-scale feature map through cross-scale fusion processing to obtain a multi-scale fusion feature map. Specifically, the third-scale feature map is upsampled by 2 times and then element-wise added to the second-scale feature map to obtain a first fusion feature map; The first fusion feature map is upsampled by 2 times and then element-wise added to the first-scale feature map to obtain a multi-scale fusion feature map.

[0071] S4. The multi-scale fusion feature map is processed by Fourier transform with a fixed convolution kernel to obtain a static branch feature map; After obtaining the degradation descriptor features of the multi-scale fusion feature map, convolutional features and channel features are extracted, and after fusion, a dynamic branch feature map is obtained; The dynamic branch feature map is fused with the static branch feature map to obtain a super-resolution reconstruction map.

[0072] Obtain static branch feature maps and dynamic branch feature maps that respectively reflect global semantic features and generate position detail features. After fusion, the consistency of ground object classification is retained and the local target accuracy is improved to obtain a super-resolution reconstruction map.

[0073] First, the multi-scale fusion feature map is processed by Fourier transform with a fixed convolution kernel to obtain a static branch feature map, ensuring the basic stability during the image reconstruction process.

[0074] Then, after obtaining the degradation descriptor features of the multi-scale fusion feature map, convolutional features and channel features are extracted. After fusion, a dynamic branch feature map is obtained. The operation of obtaining the degradation descriptor features of the multi-scale fusion feature map is specifically as follows: The multi-scale fusion feature map is processed by parallel convolution to extract the degradation feature expression capabilities of high-frequency residuals and band distortion patterns to obtain a multi-scale fusion convolutional feature map; the multi-scale fusion convolutional feature map; is processed by a generative adversarial network (preferably the CycleGAN network), implicitly learns the inter-domain degradation mapping, and through attention pooling, generates global degradation descriptor features.

[0075] Finally, the dynamic branch feature map is fused with the static branch feature map to obtain a super-resolution reconstruction map.

[0076] To verify the image reconstruction effect of this embodiment, remote sensing images in the AID dataset are reconstructed. Refer to Figure 1 and Figure 2 It can be clearly seen that the reconstruction method of this embodiment has good effects and high-quality reconstructed images.

[0077] This embodiment also provides a remote sensing image super-resolution reconstruction system for implementing the above-mentioned remote sensing image super-resolution reconstruction method, including:

[0078] A radiation high-dimensional enhanced feature map generation module, which is used for the remote sensing image to be processed to undergo radiation normalization processing to compensate for image radiation distortion to obtain a radiation-normalized map; the radiation-normalized map is processed by channel space projection to map the radiation-normalized map into a high-dimensional feature space to obtain a radiation high-dimensional feature map; the radiation high-dimensional feature map is processed by regional enhancement to obtain a radiation high-dimensional enhanced feature map;

[0079] A spatial information reconstruction feature map generation module, which is used for after reshaping the dimensions of the radiation high-dimensional enhanced feature map, obtaining a neighborhood correlation feature map and a long-distance dependence feature map. After fusion, a context-aware feature map is obtained; based on the pixel position confidence of the context-aware feature map, the context-aware feature map is corrected for feature mapping error to obtain a spatial information reconstruction feature map;

[0080] The multi-scale fusion feature map generation module is used to perform pixel-level image enhancement processing on the feature map for spatial information reconstruction to obtain the first-scale feature map; perform state-space image enhancement processing on the feature map for spatial information reconstruction after downsampling at the first scale to obtain the second-scale feature map; perform global and local image enhancement processing on the feature map for spatial information reconstruction after downsampling at the second scale to obtain the third-scale feature map; perform cross-scale fusion processing on the third-scale feature map with the second-scale feature map and the first-scale feature map to obtain the multi-scale fusion feature map;

[0081] The super-resolution reconstruction map generation module is used to perform Fourier transform processing on the multi-scale fusion feature map with a fixed convolution kernel to obtain the static branch feature map; after obtaining the degradation descriptor features of the multi-scale fusion feature map, extract convolution features and channel features, and fuse them to obtain the dynamic branch feature map; fuse the dynamic branch feature map with the static branch feature map to obtain the super-resolution reconstruction map.

[0082] This embodiment also provides a remote sensing image super-resolution reconstruction device, including a processor and a memory. Among them, when the processor executes the computer program stored in the memory, the above-mentioned remote sensing image super-resolution reconstruction method is implemented.

[0083] This embodiment also provides a computer-readable storage medium for storing a computer program. Among them, when the computer program is executed by a processor, the above-mentioned remote sensing image super-resolution reconstruction method is implemented.

[0084] A super-resolution reconstruction method for remote sensing images provided by this embodiment. First, perform radiometric normalization on the remote sensing image to be processed, identify the image content and compensate for the radiometric distortion of the sensor to obtain a radiometrically normalized image; map the radiometrically normalized image to a high-dimensional feature space to enhance the local feature response of the image, thereby enhancing the modeling ability for the distribution of irregular objects in the remote sensing image to obtain a radiometric high-dimensional feature map; the radiometric high-dimensional feature map is processed by regional enhancement to obtain a radiometric high-dimensional enhanced feature map, highlighting important regions and retaining background information, providing a richer and more complete feature representation for subsequent global modeling; then, reshape the dimensions of the radiometric high-dimensional enhanced feature map, retain the spatial topological relationship between feature channels, and obtain a neighborhood-related feature map and a long-distance dependence feature map. After fusion, a context-aware feature map is obtained; and based on the pixel position confidence of the context-aware feature map, correct the feature mapping error of the context-aware feature map to restore the image structure and obtain a spatially information-reconstructed feature map; then, perform pixel-level image enhancement processing, state-space image enhancement processing, and global and local image enhancement processing on different scale feature maps of the spatially information-reconstructed feature map respectively, obtain the original detail information of the image, global semantic information, and local context relationship information. After cross-scale fusion, ensure the preferential transmission of detail and semantic information, and finally improve the reconstruction quality of the image to obtain a multi-scale fusion feature map; finally, obtain the static branch feature map and the dynamic branch feature map of the multi-scale fusion feature map. After fusion, a super-resolution reconstruction map is obtained, improving the super-resolution reconstruction quality and stability of the remote sensing image.

Claims

1. A super-resolution reconstruction method for remote sensing images, characterized in that, Including the following operations: S1. The remote sensing image to be processed is subjected to radiometric normalization processing to compensate for the radiometric distortion of the image, obtaining a radiometrically normalized image; the radiometrically normalized image is subjected to channel space projection processing to map the radiometrically normalized image into a high-dimensional feature space, obtaining a radiometric high-dimensional feature image; The radiometric high-dimensional feature image is subjected to regional enhancement processing to obtain a radiometric high-dimensional enhanced feature image; S2. After dimension reshaping the radiometric high-dimensional enhanced feature image, a neighborhood correlation feature image and a long-range dependence feature image are obtained. After fusion, a context-aware feature image is obtained; Based on the pixel position confidence of the context-aware feature image, the context-aware feature image is corrected for feature mapping error, obtaining a spatial information reconstruction feature image; S3. The spatial information reconstruction feature image is subjected to pixel-level image enhancement processing to obtain a first-scale feature image; the spatial information reconstruction feature image is downsampled at the first scale and then subjected to state-space image enhancement processing to obtain a second-scale feature image; the spatial information reconstruction feature image is downsampled at the second scale and then subjected to global and local image enhancement processing to obtain a third-scale feature image; The third-scale feature image is subjected to cross-scale fusion processing with the second-scale feature image and the first-scale feature image to obtain a multi-scale fusion feature image; S4. The multi-scale fusion feature image is subjected to Fourier transform processing with a fixed convolution kernel to obtain a static branch feature image; After obtaining the degradation descriptor features of the multi-scale fusion feature image, convolutional features and channel features are extracted. After fusion, a dynamic branch feature image is obtained; the dynamic branch feature image is fused with the static branch feature image to obtain a super-resolution reconstruction image.

2. The remote sensing image super-resolution reconstruction method according to claim 1, wherein The operation of the radiometric normalization processing in S1 is specifically as follows: The remote sensing image to be processed is subjected to feature convolution processing to obtain a remote sensing convolution image to be processed; based on the mean, or / and variance, or / and shadow ratio of the remote sensing convolution image to be processed, a lighting description feature image is obtained; the lighting description feature image is subjected to multi-layer perceptron processing and smoothing processing to obtain a remote sensing smoothed image to be processed; after the remote sensing smoothed image to be processed is subjected to channel attention processing, it is added element-wise to the remote sensing smoothed image to be processed to obtain a radiometrically normalized image.

3. The remote sensing image super-resolution reconstruction method according to claim 1, characterized in that The operation of the regional enhancement processing in S1 is specifically as follows: The radiometric high-dimensional feature image is respectively subjected to convolution processing with different convolution kernels and dilated convolution processing to obtain a number of convolution features. After cross-fusion, fusion processing is performed to obtain a total fusion feature; The total fusion feature is subjected to non-linear processing and multi-layer perceptron processing to obtain a radiometric high-dimensional enhanced feature image.

4. The super-resolution reconstruction method of remote sensing images according to claim 1, wherein The operation of the dimension reshaping in S2 is specifically as follows: Based on the spatial importance map of the radiometric high-dimensional enhanced feature image, a recombination matrix is obtained; based on the recombination matrix, the offset of each corresponding position in the radiometric high-dimensional enhanced feature image is corrected to obtain a reshaped feature image.

5. The remote sensing image super-resolution reconstruction method according to claim 1, characterized in that The operation of obtaining the spatial information reconstruction feature image in S2 is specifically as follows: The confidence of each pixel position in the context-aware feature image is obtained, and the pixel positions with confidence less than the confidence threshold are used as the positions to be corrected; the corresponding positions of the positions to be corrected in the context-aware feature image are compensated for mapping error to obtain a spatial information reconstruction feature image.

6. The super-resolution reconstruction method of remote sensing images according to claim 1, characterized in that, The operation of the state-space image enhancement processing in S3 is specifically as follows: Based on the state transition matrix and channel feature map of the first downsampled map of spatial information, and the preset state space equation, perform state vector enhancement processing on each channel of the first downsampled map of spatial information to obtain a second-scale feature map; the first downsampled map of spatial information is obtained by performing first-scale downsampling processing on the reconstructed feature map of spatial information.

7. The super-resolution reconstruction method of remote sensing images according to claim 1, wherein The operations of global and local image enhancement processing in S3 are specifically as follows: The second downsampled map of spatial information is processed by a one-dimensional convolutional gated unit to obtain a globally downsampled map of spatial information; the globally downsampled map of spatial information is divided into several feature map blocks, and within each small block, according to the state transition matrix and projection matrix of the feature map block, and the preset state space equation, several locally enhanced blocks of the feature map are obtained; all the locally enhanced blocks of the feature map are processed by fusion to obtain a third-scale feature map; the second downsampled map of spatial information is obtained by performing second-scale downsampling processing on the reconstructed feature map of spatial information.

8. A remote sensing image super-resolution reconstruction system for implementing the remote sensing image super-resolution reconstruction method according to claim 1, characterized in that, Including: A radiation high-dimensional enhanced feature map generation module, configured to perform radiation normalization processing on the remote sensing image to be processed, compensate for image radiation distortion, and obtain a radiation-normalized map; The radiation-normalized map is processed by channel space projection to map the radiation-normalized map to a high-dimensional feature space to obtain a radiation high-dimensional feature map; The radiation high-dimensional feature map is processed by region enhancement to obtain a radiation high-dimensional enhanced feature map; A reconstructed feature map generation module of spatial information, configured to, after reshaping the dimensions of the radiation high-dimensional enhanced feature map, obtain a neighborhood-related feature map and a long-distance dependence feature map, and after fusion, obtain a context-aware feature map; Based on the pixel position confidence of the context-aware feature map, perform feature mapping error correction on the context-aware feature map to obtain a reconstructed feature map of spatial information; A multi-scale fusion feature map generation module, configured to perform pixel-level image enhancement processing on the reconstructed feature map of spatial information to obtain a first-scale feature map; perform state space image enhancement processing on the reconstructed feature map of spatial information after first-scale downsampling to obtain a second-scale feature map; perform global and local image enhancement processing on the reconstructed feature map of spatial information after second-scale downsampling to obtain a third-scale feature map; The third-scale feature map, the second-scale feature map, and the first-scale feature map are processed by cross-scale fusion to obtain a multi-scale fusion feature map; A super-resolution reconstruction map generation module, configured to perform Fourier transform processing on the multi-scale fusion feature map with a fixed convolution kernel to obtain a static branch feature map; after obtaining the degradation descriptor features of the multi-scale fusion feature map, extract convolution features and channel features, and after fusion, obtain a dynamic branch feature map; fuse the dynamic branch feature map with the static branch feature map to obtain a super-resolution reconstruction map.

9. A remote sensing image super-resolution reconstruction device, characterized in that Including a processor and a memory, wherein, when the processor executes the computer program stored in the memory, the remote sensing image super-resolution reconstruction method according to any one of claims 1-7 is implemented.

10. A computer-readable storage medium, characterized in that, For storing a computer program, wherein, when the computer program is executed by a processor, the remote sensing image super-resolution reconstruction method according to any one of claims 1-7 is implemented.

Images