Remote sensing image super-resolution reconstruction method, system and device and storage medium

Abstract

The invention relates to the technical field of image reconstruction, in particular to a remote sensing image super-resolution reconstruction method, system and device and a storage medium. In order to solve the technical problem that in the prior art, a low-resolution remote sensing image reconstruction method is poor in effect, firstly, radiation normalization is carried out on a remote sensing image to be processed, the remote sensing image is mapped to a high-dimensional feature space, then region enhancement processing is carried out, and dimension remodeling is carried out on a radiation high-dimensional enhanced feature map; obtaining a neighborhood correlation feature map and a long-distance dependence feature map, and after fusion, carrying out feature mapping error correction to obtain a spatial information reconstruction feature map; then, feature maps of different scales of the spatial information reconstruction feature map are subjected to image enhancement processing and cross-scale fusion respectively, and a multi-scale fusion feature map is obtained; and finally, obtaining a static branch feature graph and a dynamic branch feature graph of the multi-scale fusion feature graph, and obtaining a super-resolution reconstruction graph after fusion, thereby improving the super-resolution reconstruction quality and stability of the remote sensing image.

Description

Technical Field

[0001] The present invention relates to the technical field of image reconstruction, and specifically to a super-resolution reconstruction method, system, device, and storage medium for remote sensing images. Background Art

[0002] With the continuous development of remote sensing technology, remote sensing images have been widely used in fields such as geographic information systems, agriculture, and urban planning. However, due to limitations in acquisition conditions, such as sensor resolution, sampling distance, etc., the resolution of remote sensing images is low, which affects their application in precise analysis and detail extraction. To overcome this problem, super-resolution reconstruction technology has emerged, aiming to restore images with higher spatial resolution by processing low-resolution remote sensing images.

[0003] Existing super-resolution reconstruction methods for remote sensing images are mostly based on interpolation techniques or deep learning methods, but these methods generally have the following deficiencies: Interpolation methods are prone to causing image blurring and cannot effectively restore image details; Deep learning-based methods usually rely on a large amount of labeled data for training and lack an effective multi-scale information fusion mechanism. Especially when dealing with complex ground objects, the image reconstruction effect is poor and unstable. Summary of the Invention

[0004] The purpose of the present invention is to provide a super-resolution reconstruction method, system, device, and storage medium for remote sensing images.

[0005] The technical solution of the present invention is as follows: A super-resolution reconstruction method for remote sensing images, including the following operations: S1. The remote sensing image to be processed is subjected to radiometric normalization processing to compensate for image radiometric distortion, obtaining a radiometrically normalized image; the radiometrically normalized image is subjected to channel space projection processing to map the radiometrically normalized image into a high-dimensional feature space, obtaining a radiometric high-dimensional feature image; the radiometric high-dimensional feature image is subjected to regional enhancement processing, obtaining a radiometric high-dimensional enhanced feature image; S2. After dimension reshaping of the radiometric high-dimensional enhanced feature image, a neighborhood-related feature image and a long-distance dependence feature image are obtained. After fusion, a context-aware feature image is obtained; based on the pixel position confidence of the context-aware feature image, feature mapping error correction is performed on the context-aware feature image, obtaining a spatial information reconstruction feature image; S3. The spatial information reconstruction feature image is subjected to pixel-level image enhancement processing, obtaining a first-scale feature image; the spatial information reconstruction feature image is subjected to downsampling at the first scale and then state-space image enhancement processing, obtaining a second-scale feature image; the spatial information reconstruction feature image is subjected to downsampling at the second scale and then global and local image enhancement processing, obtaining a third-scale feature image; the third-scale feature image is subjected to cross-scale fusion processing with the second-scale feature image and the first-scale feature image, obtaining a multi-scale fusion feature image; S4. The multi-scale fusion feature map is processed by Fourier transform with a fixed convolution kernel to obtain a static branch feature map. After obtaining the degradation descriptor features of the multi-scale fusion feature map, convolutional features and channel features are extracted, and after fusion, a dynamic branch feature map is obtained. The dynamic branch feature map is fused with the static branch feature map to obtain a super-resolution reconstruction map.

[0006] The operation of radiation normalization processing in S1 is specifically as follows: The remote sensing image to be processed is subjected to feature convolution processing to obtain a remote sensing convolution image to be processed. Based on the mean, or / and variance, or / and shadow ratio of the remote sensing convolution image to be processed, an illumination description feature map is obtained. The illumination description feature map is processed by a multi-layer perceptron and smoothed to obtain a remote sensing smoothed image to be processed. After the remote sensing smoothed image to be processed is subjected to channel attention processing, it is added element-wise to the remote sensing smoothed image to be processed to obtain a radiation normalization map.

[0007] The operation of region enhancement processing in S1 is specifically as follows: The high-dimensional radiation feature map is respectively subjected to convolution processing with different convolution kernels and dilated convolution processing to obtain a number of convolutional features. After cross-fusion, fusion processing is performed to obtain a total fusion feature. The total fusion feature is subjected to non-linear processing and multi-layer perceptron processing to obtain a high-dimensional radiation enhanced feature map.

[0008] The operation of dimension reshaping in S2 is specifically as follows: Based on the spatial importance map of the high-dimensional radiation enhanced feature map, a recombination matrix is obtained. Based on the recombination matrix, the offset of each corresponding position in the high-dimensional radiation enhanced feature map is corrected to obtain a reshaped feature map.

[0009] The operation of obtaining the spatial information reconstruction feature map in S2 is specifically as follows: The confidence of each pixel position in the context-aware feature map is obtained, and the pixel positions with a confidence less than the confidence threshold are used as the positions to be corrected. Mapping error compensation is performed on the corresponding positions of the positions to be corrected in the context-aware feature map to obtain a spatial information reconstruction feature map.

[0010] The operation of state space image enhancement processing in S3 is specifically as follows: Based on the state transition matrix and channel feature map of the first downsampled spatial information map, and a preset state space equation, state vector enhancement processing is performed on each channel of the first downsampled spatial information map to obtain a second-scale feature map. The first downsampled spatial information map is obtained by performing first-scale downsampling processing on the spatial information reconstruction feature map.

[0011] The operations of global and local image enhancement processing in S3 are specifically as follows: The second downsampled map of spatial information is processed by a one-dimensional convolutional gated unit to obtain a globally downsampled map of spatial information; the globally downsampled map of spatial information is segmented into several feature map blocks, and within each small block, several locally enhanced feature map blocks are obtained according to the state transition matrix and projection matrix of the feature map block, as well as a preset state space equation; all the locally enhanced feature map blocks are processed by fusion to obtain a third-scale feature map; the second downsampled map of spatial information is obtained by performing second-scale downsampling on the reconstructed feature map of spatial information.

[0012] A remote sensing image super-resolution reconstruction system for implementing the above-mentioned remote sensing image super-resolution reconstruction method includes: A radiation high-dimensional enhanced feature map generation module, which is used to perform radiation normalization processing on a remote sensing image to be processed, compensate for image radiation distortion, and obtain a radiation-normalized map; the radiation-normalized map is processed by channel space projection to map the radiation-normalized map into a high-dimensional feature space to obtain a radiation high-dimensional feature map; the radiation high-dimensional feature map is processed by regional enhancement to obtain a radiation high-dimensional enhanced feature map; A spatial information reconstructed feature map generation module, which is used to reshape the dimensions of the radiation high-dimensional enhanced feature map, obtain a neighborhood-related feature map and a long-distance dependence feature map, and after fusion, obtain a context-aware feature map; based on the pixel position confidence of the context-aware feature map, perform feature mapping error correction on the context-aware feature map to obtain a spatial information reconstructed feature map; A multi-scale fusion feature map generation module, which is used to perform pixel-level image enhancement processing on the spatial information reconstructed feature map to obtain a first-scale feature map; perform state space image enhancement processing on the spatially reconstructed feature map after first-scale downsampling to obtain a second-scale feature map; perform global and local image enhancement processing on the spatially reconstructed feature map after second-scale downsampling to obtain a third-scale feature map; the third-scale feature map is fused with the second-scale feature map and the first-scale feature map through cross-scale fusion processing to obtain a multi-scale fusion feature map; A super-resolution reconstruction map generation module, which is used to perform Fourier transform processing on the multi-scale fusion feature map with a fixed convolution kernel to obtain a static branch feature map; after obtaining the degradation descriptor features of the multi-scale fusion feature map, extract convolution features and channel features, and after fusion, obtain a dynamic branch feature map; fuse the dynamic branch feature map with the static branch feature map to obtain a super-resolution reconstruction map.

[0013] A remote sensing image super-resolution reconstruction device includes a processor and a memory. Among them, when the processor executes the computer program stored in the memory, the above-mentioned remote sensing image super-resolution reconstruction method is implemented.

[0014] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned remote sensing image super-resolution reconstruction method.

[0015] The beneficial effects of the present invention are as follows: A remote sensing image super-resolution reconstruction method provided by the present invention. First, the remote sensing image to be processed is radiometrically normalized to identify the image content and compensate for the radiometric distortion of the sensor, obtaining a radiometrically normalized image; and the radiometrically normalized image is mapped to a high-dimensional feature space to enhance the local feature response of the image, thereby enhancing the modeling ability for the irregular object distribution in the remote sensing image, obtaining a radiometric high-dimensional feature map; the radiometric high-dimensional feature map is processed by regional enhancement to obtain a radiometric high-dimensional enhanced feature map, highlighting important regions and retaining background information, providing a more abundant and complete feature representation for subsequent global modeling; then, the radiometric high-dimensional enhanced feature map is dimensionally reshaped. After retaining the spatial topological relationship between the feature channels, a neighborhood-related feature map and a long-range dependence feature map are obtained. After fusion, a context-aware feature map is obtained; and based on the pixel position confidence of the context-aware feature map, the feature mapping error of the context-aware feature map is corrected to restore the image structure, obtaining a spatial information reconstruction feature map; then, the different-scale feature maps of the spatial information reconstruction feature map are respectively subjected to pixel-level image enhancement processing, state-space image enhancement processing, and global and local image enhancement processing to obtain the original detail information of the image, global semantic information, and local context relationship information. After cross-scale fusion, the priority transmission of detail and semantic information is ensured, and finally the reconstruction quality of the image is improved, obtaining a multi-scale fusion feature map; finally, a static branch feature map and a dynamic branch feature map of the multi-scale fusion feature map are obtained. After fusion, a super-resolution reconstruction map is obtained, improving the quality and reconstruction stability of the remote sensing image super-resolution reconstruction result. Description of the Drawings

[0016] By reading the detailed description of the preferred embodiments below, the solutions and advantages of the present application will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention.

[0017] In the drawings: Figure 1 For the super-resolution reconstruction effect diagram obtained by applying the method of this embodiment to process the airport remote sensing image in the embodiment; in Figure 1 , (a) is the original airport remote sensing image, and (b) is the reconstruction effect diagram, Figure 2 For the super-resolution reconstruction effect diagram obtained by applying the method of this embodiment to process the urban remote sensing image in the embodiment; in Figure 2 , (a) is the original urban remote sensing image, and (b) is the reconstruction effect diagram. Detailed Embodiments

[0018] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0019] This embodiment provides a remote sensing image super-resolution reconstruction method, including the following operations: S1. The remote sensing image to be processed is subjected to radiometric normalization processing to compensate for the radiometric distortion of the image, obtaining a radiometrically normalized image; the radiometrically normalized image is subjected to channel space projection processing to map the radiometrically normalized image into a high-dimensional feature space, obtaining a radiometric high-dimensional feature image; the radiometric high-dimensional feature image is subjected to regional enhancement processing, obtaining a radiometric high-dimensional enhanced feature image; S2. After reshaping the dimensions of the radiometric high-dimensional enhanced feature image, a neighborhood correlation feature image and a long-range dependence feature image are obtained. After fusion, a context-aware feature image is obtained; based on the pixel position confidence of the context-aware feature image, the context-aware feature image is corrected for feature mapping error, obtaining a spatial information reconstruction feature image; S3. The spatial information reconstruction feature image is subjected to pixel-level image enhancement processing, obtaining a first-scale feature image; the spatial information reconstruction feature image is downsampled at the first scale and then subjected to state-space image enhancement processing, obtaining a second-scale feature image; the spatial information reconstruction feature image is downsampled at the second scale and then subjected to global and local image enhancement processing, obtaining a third-scale feature image; the third-scale feature image is subjected to cross-scale fusion processing with the second-scale feature image and the first-scale feature image, obtaining a multi-scale fusion feature image; S4. The multi-scale fusion feature image is subjected to Fourier transform processing with a fixed convolution kernel, obtaining a static branch feature image; after obtaining the degradation descriptor features of the multi-scale fusion feature image, convolutional features and channel features are extracted. After fusion, a dynamic branch feature image is obtained; the dynamic branch feature image is fused with the static branch feature image, obtaining a super-resolution reconstruction image. The specific operation details are as follows.

[0020] S1. The remote sensing image to be processed is subjected to radiometric normalization processing to compensate for the radiometric distortion of the image, obtaining a radiometrically normalized image; the radiometrically normalized image is subjected to channel space projection processing to map the radiometrically normalized image into a high-dimensional feature space, obtaining a radiometric high-dimensional feature image; the radiometric high-dimensional feature image is subjected to regional enhancement processing, obtaining a radiometric high-dimensional enhanced feature image.

[0021] The remote sensing image to be processed is radiometrically normalized to automatically identify the image content and compensate for the radiometric distortion of the sensor, obtaining a radiometrically normalized image; then, the radiometrically normalized image is mapped into a high-dimensional feature space to enhance the local feature response of the image, thereby enhancing the modeling ability for the irregular object distribution in the remote sensing image, obtaining a radiometric high-dimensional feature image; the radiometric high-dimensional feature image is subjected to regional enhancement processing, obtaining a radiometric high-dimensional enhanced feature image, highlighting the important regions and retaining the background information, providing a more abundant and complete feature representation for subsequent global modeling.

[0022] First, the low-resolution remote sensing image to be processed is subjected to radiometric normalization instead of simple global normalization to compensate for the image radiometric distortion of the remote sensing image obtained by the sensor, and a radiometric normalization map is obtained.

[0023] The operation of radiometric normalization is specifically as follows: The remote sensing image to be processed is subjected to feature convolution processing by a CNN convolution network composed of a superposition of several convolutional blocks (preferably 3 convolutional blocks with a convolutional kernel of 3×3) to obtain a remote sensing convolution image to be processed; based on the band mean, and / or band variance, and / or shadow ratio of the remote sensing convolution image to be processed, a lighting description feature is obtained; based on the lighting description feature, the remote sensing image to be processed is subjected to multi-layer perceptron processing and smoothing processing to achieve sensor characteristic compensation, and a remote sensing smoothed image to be processed is obtained; after the remote sensing smoothed image to be processed is subjected to channel attention processing, it is added element by element to the remote sensing smoothed image to be processed to further eliminate non-linear distortion, and a radiometric normalization map is obtained.

[0024] Among them, the operation of obtaining the lighting description feature is achieved through the following formula: , , , , is the lighting description feature, is the band mean, is the pixel mean of the image corresponding to the m th band of the remote sensing convolution image to be processed, is the band variance of the remote sensing convolution image to be processed, is the pixel variance of the image corresponding to the m th band of the remote sensing convolution image to be processed, which is based on obtained; is the shadow ratio, is an indicator function, which is 1 when the condition is satisfied and 0 when it is not; is the image corresponding to the m th band of the remote sensing convolution image to be processed, is the image corresponding to the m th band of the remote sensing convolution image to be processed, , is a balance coefficient, which is an empirical value, , where H and W are the length and width of the remote sensing convolution image to be processed respectively.

[0025] Then, the radiometric normalization map is subjected to channel space projection processing to map the radiometric normalization map to a high-dimensional feature space, and a radiometric high-dimensional feature map is obtained.

[0026] The operation of channel space projection processing can be implemented by the following formula: , is the radiated high-dimensional feature map, is the convolution weight, is the GELU activation function, is the radiated normalization map, is the bias term of the radiated normalization map, is the learnable balance parameter, is the lightweight spatial attention operator, preferably the SimAM network.

[0027] Among them, the bias term of the radiated normalization map is a matrix formed by the bias amounts at each position of the radiated normalization map. The specific method for obtaining the bias amount at each position is as follows: perform grouped convolution (preferably with a convolution kernel of 3 and a number of groups of 4) processing and ordinary convolution (convolution kernel of 1) processing on the radiated normalization map to obtain the radiated normalization feature convolution map; obtain the texture complexity and radiated uniformity of the radiated normalization feature convolution map. After normalization processing, multiply them by their respective preset weight matrices respectively, and then perform matrix addition with the Gaussian smoothing matrix of the radiated normalization map to obtain the bias matrix, which is used as the bias term of the radiated normalization map. This method effectively adapts to different images, dynamically generates biases according to the characteristics of the image regions, and effectively adjusts the intensity of the image feature response. The texture complexity is obtained by calculating the gray variance of the 3×3 neighborhood of the radiated normalization feature convolution map, and is used to quantify the richness of details in the region. The radiated uniformity (illumination uniformity) is the ratio of the minimum illuminance to the average illuminance in the region of interest (which can be set according to actual needs), and reflects the stability of the illumination conditions.

[0028] Finally, the radiated high-dimensional feature map is subjected to region enhancement processing to highlight important regions and retain background information, obtaining the radiated high-dimensional enhanced feature map, which provides a richer and more complete feature representation for subsequent global modeling.

[0029] The operation of the region enhancement processing is specifically as follows: the radiated high-dimensional feature map is respectively subjected to convolution processing with different convolution kernels (preferably using convolutions with convolution kernels of 3×3 and 5×5), and atrous convolution processing to obtain several convolution features. After cross-fusion (fusion processing between different convolution features), fusion processing is performed to generate the total fusion feature; the total fusion feature is subjected to non-linear processing (preferably the Softmax function) and multi-layer perceptron processing to ensure that while highlighting important regions, the context information of the background environment is retained, obtaining the radiated high-dimensional enhanced feature map, which provides a discriminative and complete feature representation for subsequent processing.

[0030] S2. After reshaping the radiated high-dimensional enhanced feature map, obtain the neighborhood-related feature map and the long-distance dependence feature map. After fusion, obtain the context-aware feature map; based on the pixel position confidence of the context-aware feature map, correct the feature mapping error of the context-aware feature map to obtain the spatial information reconstruction feature map.

[0031] Reshape the radiated high-dimensional enhanced feature map. After retaining the spatial topological relationship between feature channels, obtain the neighborhood-related feature map and the long-distance dependence feature map. After fusion, obtain the context-aware feature map; and based on the pixel position confidence of the context-aware feature map, correct the feature mapping error of the context-aware feature map to restore the image structure and obtain the spatial information reconstruction feature map.

[0032] First, reshape the radiated high-dimensional enhanced feature map. While converting the feature map dimension from [B, C, H, W] to [B, H×W, C], retain the spatial topological relationship between feature channels to obtain the reshaped feature map. The operation of dimension reshaping is specifically as follows: based on the spatial importance map of the radiated high-dimensional enhanced feature map (which can be obtained by processing the radiated high-dimensional enhanced feature map through a lightweight CNN network), obtain a learnable recombination matrix (which can be obtained by converting the spatial importance map into a matrix form according to pixel information); based on the recombination matrix, correct the offset of each corresponding position in the radiated high-dimensional enhanced feature map to obtain the reshaped feature map.

[0033] Then, obtain the neighborhood-related feature map and the long-distance dependence feature map of the reshaped feature map, and capture the neighborhood feature correlation of each position in the reshaped feature map and the long-distance dependence feature of the reshaped feature map.

[0034] Among them, the neighborhood-related feature map is obtained by processing the reshaped feature map through a gated linear unit (preferably a GLU network) and depthwise separable convolution. The long-distance dependence feature can be obtained by processing the reshaped feature map through a Transformer network, or by processing the neighborhood-related feature map through a selective state mechanism.

[0035] The operation of the selective state mechanism processing is specifically as follows: the neighborhood-related feature map is processed through a multi-layer perceptron to obtain state parameters; the neighborhood-related feature map is processed through a state space model (SSM). During the processing, according to the comparison result of the state parameters and the corresponding parameter thresholds, corresponding multi-granularity calculation processing is performed to obtain the long-distance dependence feature. This dynamic regulation through content awareness, rather than fixed attenuation, thus improves connectivity when processing long-range structures such as roads in remote sensing images, and saves computational effort in smooth areas such as farmland in remote sensing images compared with traditional methods, and also enhances the robustness to sensor noise.

[0036] Next, the neighborhood-related feature map and the long-range dependence feature map are fused (which can be achieved through weighted summation) to obtain a context-aware feature map.

[0037] Finally, based on the pixel position confidence of the context-aware feature map, the context-aware feature map is corrected for feature mapping error to ensure the complete reconstruction of spatial information, and a spatial information reconstruction feature map is obtained.

[0038] The operation of obtaining the spatial information reconstruction feature map is specifically as follows: Obtain the confidence of each pixel position in the context-aware feature map (which can be obtained by processing the context-aware feature map through a lightweight UNet network), and use the pixel positions with confidence less than the confidence threshold as the positions to be corrected; perform mapping error compensation on the corresponding positions of the positions to be corrected in the context-aware feature map to obtain the spatial information reconstruction feature map.

[0039] The mapping error compensation can be achieved through the following formula: F_corrected = F + (1 - conf) × Conv([F, F_rec]), where F_corrected is the spatial information reconstruction feature map, F is the context-aware feature map, conf is the confidence, positions with low confidence (conf < 0.5) indicate possible errors (such as building edges), positions with high confidence (conf ≥ 0.5) indicate reliable regions (such as smooth ground), Conv( ) is the convolution process, and F_rec is the inverse transform map (obtained by performing inverse wavelet transform on the context-aware feature map).

[0040] S3. The spatial information reconstruction feature map is processed by pixel-level image enhancement to obtain a first-scale feature map; the spatial information reconstruction feature map is downsampled at the first scale and then processed by state-space image enhancement to obtain a second-scale feature map; the spatial information reconstruction feature map is downsampled at the second scale and then processed by global and local image enhancement to obtain a third-scale feature map; the third-scale feature map is fused with the second-scale feature map and the first-scale feature map through cross-scale fusion to obtain a multi-scale fusion feature map.

[0041] The different-scale feature maps of the spatial information reconstruction feature map are respectively processed by pixel-level image enhancement, state-space image enhancement, and global and local image enhancement to obtain the original image detail information, global semantic information, and local context relationship information. After cross-scale fusion, the priority transmission of detail and semantic information is ensured, and finally the reconstruction quality of the image is improved to obtain a multi-scale fusion feature map.

[0042] First, the spatial information reconstruction feature map is processed by pixel-level image enhancement to keep the resolution unchanged and capture pixel details, such as road markings in a remote sensing image, to obtain a first-scale feature map. Among them, the operation of pixel-level image enhancement can be achieved through Laplacian sharpening.

[0043] Meanwhile, the spatially informative reconstruction feature map is subjected to state-space image enhancement processing after being downsampled at the first scale to model medium-scale features and capture regional features between the detailed level and the global level, such as the morphological and layout features in a large area of the image, resulting in a second-scale feature map. The spatially informative reconstruction feature map is downsampled at the first scale by: downsampling the spatially informative reconstruction feature map by a factor of 1 / 2 to obtain the first downsampled map of spatial information.

[0044] The specific operation of the above state-space image enhancement processing is as follows: based on the state transition matrix and channel feature map of the first downsampled map of spatial information, and a preset state-space equation, perform state vector enhancement processing on each channel of the first downsampled map of spatial information (input the state transition matrix and channel feature map into the state-space equation for solution) to obtain the second-scale feature map. The state transition matrix of the first downsampled map of spatial information can be obtained by processing the first downsampled map of spatial information with a state space model (SSM), and the channel feature map can be obtained by processing the first downsampled map of spatial information with a channel attention mechanism.

[0045] Next, the spatially informative reconstruction feature map is subjected to global and local image enhancement processing after being downsampled at the second scale to model deep-scale features and extract semantic information, such as the distribution of ground objects and scene categories, resulting in a third-scale feature map. The spatially informative reconstruction feature map is downsampled at the second scale by: downsampling the spatially informative reconstruction feature map by a factor of 1 / 4 to obtain the second downsampled map of spatial information.

[0046] The specific operation of the above global and local image enhancement processing is as follows: the second downsampled map of spatial information is processed by a one-dimensional convolutional gated unit, and the pixel values of each column (or each row) of the second downsampled map of spatial information are convolved one-dimensionally to capture the local context relationship in that direction, and the convolution result is dynamically adjusted by the gated unit to emphasize the correlation between pixels, obtaining the globally downsampled map of spatial information; the globally downsampled map of spatial information is segmented into several feature map blocks, and within each small block, according to the state transition matrix and projection matrix of the feature map block, and a preset state-space equation, the pixel information of the feature block is dynamically adjusted to obtain several locally enhanced blocks of the feature map; all the locally enhanced blocks of the feature map are subjected to fusion processing to obtain the third-scale feature map.

[0047] Finally, the third-scale feature map is subjected to cross-scale fusion processing with the second-scale feature map and the first-scale feature map to obtain a multi-scale fusion feature map. Specifically, the third-scale feature map is upsampled by a factor of 2 and then added element-wise to the second-scale feature map to obtain the first fusion feature map; the first fusion feature map is upsampled by a factor of 2 and then added element-wise to the first-scale feature map to obtain the multi-scale fusion feature map.

[0048] S4. The multi-scale fusion feature map is processed by Fourier transform with a fixed convolution kernel to obtain a static branch feature map. After obtaining the degradation descriptor features of the multi-scale fusion feature map, convolutional features and channel features are extracted, and after fusion, a dynamic branch feature map is obtained. The dynamic branch feature map is fused with the static branch feature map to obtain a super-resolution reconstruction map.

[0049] The static branch feature map and the dynamic branch feature map that respectively reflect the global semantic features and the generated position detail features are obtained. After fusion, the consistency of ground object classification is retained and the local target accuracy is improved to obtain a super-resolution reconstruction map.

[0050] First, the multi-scale fusion feature map is processed by Fourier transform with a fixed convolution kernel to obtain a static branch feature map, ensuring the basic stability in the image reconstruction process.

[0051] Then, after obtaining the degradation descriptor features of the multi-scale fusion feature map, convolutional features and channel features are extracted, and after fusion, a dynamic branch feature map is obtained. The operation of obtaining the degradation descriptor features of the multi-scale fusion feature map is specifically as follows: the multi-scale fusion feature map is processed by parallel convolution to extract the degradation feature expression capabilities of high-frequency residuals and band distortion patterns to obtain a multi-scale fusion convolutional feature map; the multi-scale fusion convolutional feature map is processed by a generative adversarial network (preferably the CycleGAN network), implicitly learns the inter-domain degradation mapping, and through attention pooling, generates global degradation descriptor features.

[0052] Finally, the dynamic branch feature map is fused with the static branch feature map to obtain a super-resolution reconstruction map.

[0053] To verify the image reconstruction effect of this embodiment, remote sensing images in the AID dataset are reconstructed. Refer to Figure 1 and Figure 2 , it can be clearly seen that the reconstruction method of this embodiment has a good effect and the quality of the reconstructed image is high.

[0054] This embodiment also provides a remote sensing image super-resolution reconstruction system for implementing the above-mentioned remote sensing image super-resolution reconstruction method, including: A radiation high-dimensional enhanced feature map generation module, which is used to perform radiation normalization processing on the to-be-processed remote sensing image to compensate for image radiation distortion and obtain a radiation-normalized map; the radiation-normalized map is processed by channel space projection to map the radiation-normalized map to a high-dimensional feature space to obtain a radiation high-dimensional feature map; the radiation high-dimensional feature map is processed by regional enhancement to obtain a radiation high-dimensional enhanced feature map; The spatial information reconstruction feature map generation module is used to reshape the dimension of the radiation high-dimensional enhanced feature map to obtain a neighborhood-related feature map and a long-distance dependence feature map. After fusion, a context-aware feature map is obtained. Based on the pixel position confidence of the context-aware feature map, the context-aware feature map is corrected for feature mapping error to obtain a spatial information reconstruction feature map. The multi-scale fusion feature map generation module is used to perform pixel-level image enhancement processing on the spatial information reconstruction feature map to obtain a first-scale feature map; perform state-space image enhancement processing on the spatial information reconstruction feature map after downsampling at the first scale to obtain a second-scale feature map; perform global and local image enhancement processing on the spatial information reconstruction feature map after downsampling at the second scale to obtain a third-scale feature map; perform cross-scale fusion processing on the third-scale feature map with the second-scale feature map and the first-scale feature map to obtain a multi-scale fusion feature map. The super-resolution reconstruction map generation module is used to perform Fourier transform processing on the multi-scale fusion feature map with a fixed convolution kernel to obtain a static branch feature map; after obtaining the degradation descriptor features of the multi-scale fusion feature map, extract convolution features and channel features, and after fusion, obtain a dynamic branch feature map; fuse the dynamic branch feature map with the static branch feature map to obtain a super-resolution reconstruction map.

[0055] This embodiment also provides a remote sensing image super-resolution reconstruction device, including a processor and a memory. Among them, when the processor executes the computer program stored in the memory, the above-mentioned remote sensing image super-resolution reconstruction method is implemented.

[0056] This embodiment also provides a computer-readable storage medium for storing a computer program. Among them, when the computer program is executed by a processor, the above-mentioned remote sensing image super-resolution reconstruction method is implemented.

[0057] A super-resolution reconstruction method for remote sensing images provided in this embodiment. First, the remote sensing image to be processed is radiometrically normalized to identify the image content and compensate for the radiometric distortion of the sensor, obtaining a radiometrically normalized image; and the radiometrically normalized image is mapped to a high-dimensional feature space to enhance the local feature response of the image, thereby enhancing the modeling ability for the distribution of irregular objects in the remote sensing image, obtaining a radiometric high-dimensional feature map; the radiometric high-dimensional feature map is processed by regional enhancement to obtain a radiometric high-dimensional enhanced feature map, highlighting important regions and retaining background information, providing a more abundant and complete feature representation for subsequent global modeling; then, the radiometric high-dimensional enhanced feature map is dimensionally reshaped, and after retaining the spatial topological relationship between feature channels, a neighborhood-related feature map and a long-range dependence feature map are obtained. After fusion, a context-aware feature map is obtained; and based on the pixel position confidence of the context-aware feature map, the feature mapping error of the context-aware feature map is corrected to restore the image structure, obtaining a spatial information reconstruction feature map; next, different scale feature maps of the spatial information reconstruction feature map are respectively subjected to pixel-level image enhancement processing, state-space image enhancement processing, and global and local image enhancement processing to obtain the original detail information of the image, global semantic information, and local context relationship information. After cross-scale fusion, the priority transmission of detail and semantic information is ensured, and finally the reconstruction quality of the image is improved, obtaining a multi-scale fusion feature map; finally, the static branch feature map and the dynamic branch feature map of the multi-scale fusion feature map are obtained. After fusion, a super-resolution reconstruction map is obtained, improving the super-resolution reconstruction quality and stability of the remote sensing image.

Claims

1. A super-resolution reconstruction method for remote sensing images, characterized in that, Including the following operations: S1. The remote sensing image to be processed is subjected to radiometric normalization processing to compensate for the radiometric distortion of the image, obtaining a radiometrically normalized image; the radiometrically normalized image is subjected to channel space projection processing to map the radiometrically normalized image into a high-dimensional feature space, obtaining a radiometric high-dimensional feature image; The radiometric high-dimensional feature image is subjected to regional enhancement processing to obtain a radiometric high-dimensional enhanced feature image; S2. After dimension reshaping the radiometric high-dimensional enhanced feature image, a neighborhood correlation feature image and a long-range dependence feature image are obtained. After fusion, a context-aware feature image is obtained; Based on the pixel position confidence of the context-aware feature image, the context-aware feature image is corrected for feature mapping error, obtaining a spatial information reconstruction feature image; S3. The spatial information reconstruction feature image is subjected to pixel-level image enhancement processing to obtain a first-scale feature image; the spatial information reconstruction feature image is subjected to downsampling at the first scale and then subjected to state-space image enhancement processing to obtain a second-scale feature image; the spatial information reconstruction feature image is subjected to downsampling at the second scale and then subjected to global and local image enhancement processing to obtain a third-scale feature image; The third-scale feature image, the second-scale feature image, and the first-scale feature image are subjected to cross-scale fusion processing to obtain a multi-scale fusion feature image; S4. The multi-scale fusion feature image is subjected to Fourier transform processing with a fixed convolution kernel to obtain a static branch feature image; After obtaining the degradation descriptor features of the multi-scale fusion feature image, convolutional features and channel features are extracted. After fusion, a dynamic branch feature image is obtained; the dynamic branch feature image is fused with the static branch feature image to obtain a super-resolution reconstruction image.

2. The super-resolution reconstruction method of remote sensing images according to claim 1, characterized in that The operation of the radiometric normalization processing in S1 is specifically as follows: The remote sensing image to be processed is subjected to feature convolution processing to obtain a remote sensing convolution image to be processed; based on the mean, or / and variance, or / and shadow ratio of the remote sensing convolution image to be processed, a lighting description feature image is obtained; the lighting description feature image is subjected to multi-layer perceptron processing and smoothing processing to obtain a remote sensing smoothed image to be processed; after the remote sensing smoothed image to be processed is subjected to channel attention processing, it is added element-wise to the remote sensing smoothed image to be processed to obtain a radiometrically normalized image.

3. The super-resolution reconstruction method of remote sensing images according to claim 1, characterized in that The operation of the regional enhancement processing in S1 is specifically as follows: The radiometric high-dimensional feature image is respectively subjected to convolution processing with different convolution kernels and dilated convolution processing to obtain a number of convolution features. After cross-fusion, fusion processing is performed to obtain a total fusion feature; The total fusion feature is subjected to non-linear processing and multi-layer perceptron processing to obtain a radiometric high-dimensional enhanced feature image.

4. The super-resolution reconstruction method of remote sensing images according to claim 1, wherein The operation of the dimension reshaping in S2 is specifically as follows: Based on the spatial importance map of the radiometric high-dimensional enhanced feature image, a recombination matrix is obtained; based on the recombination matrix, the offset of each corresponding position in the radiometric high-dimensional enhanced feature image is corrected to obtain a reshaped feature image.

5. The remote sensing image super-resolution reconstruction method according to claim 1, characterized in that The operation of obtaining the spatial information reconstruction feature image in S2 is specifically as follows: The confidence of each pixel position in the context-aware feature image is obtained, and the pixel positions with confidence less than the confidence threshold are used as the positions to be corrected; the corresponding positions of the positions to be corrected in the context-aware feature image are compensated for mapping error to obtain a spatial information reconstruction feature image.

6. The super-resolution reconstruction method of remote sensing images according to claim 1, characterized in that The operation of the state-space image enhancement processing in S3 is specifically as follows: Based on the state transition matrix and channel feature map of the first downsampled map of spatial information, and a preset state space equation, perform state vector enhancement processing on each channel of the first downsampled map of spatial information to obtain a second-scale feature map; the first downsampled map of spatial information is obtained by performing first-scale downsampling processing on the reconstructed feature map of spatial information.

7. The super-resolution reconstruction method of remote sensing images according to claim 1, wherein The operations of global and local image enhancement processing in S3 are specifically as follows: The second downsampled map of spatial information is processed by a one-dimensional convolutional gated unit to obtain a globally downsampled map of spatial information; the globally downsampled map of spatial information is segmented into several feature map blocks, and within each small block, several locally enhanced blocks of feature maps are obtained according to the state transition matrix and projection matrix of the feature map block, and a preset state space equation; all the locally enhanced blocks of feature maps are processed by fusion to obtain a third-scale feature map; the second downsampled map of spatial information is obtained by performing second-scale downsampling processing on the reconstructed feature map of spatial information.

8. A remote sensing image super-resolution reconstruction system for implementing the remote sensing image super-resolution reconstruction method according to claim 1, characterized in that, Including: A radiation high-dimensional enhanced feature map generation module, configured to perform radiation normalization processing on a remote sensing image to be processed, compensate for image radiation distortion, and obtain a radiation-normalized map; The radiation-normalized map is processed by channel space projection to map the radiation-normalized map into a high-dimensional feature space to obtain a radiation high-dimensional feature map; The radiation high-dimensional feature map is processed by regional enhancement to obtain a radiation high-dimensional enhanced feature map; A reconstructed feature map generation module of spatial information, configured to, after reshaping the dimensions of the radiation high-dimensional enhanced feature map, obtain a neighborhood-related feature map and a long-range dependence feature map, and after fusion, obtain a context-aware feature map; Based on the pixel position confidence of the context-aware feature map, perform feature mapping error correction on the context-aware feature map to obtain a reconstructed feature map of spatial information; A multi-scale fusion feature map generation module, configured to perform pixel-level image enhancement processing on the reconstructed feature map of spatial information to obtain a first-scale feature map; perform state space image enhancement processing on the reconstructed feature map of spatial information after first-scale downsampling to obtain a second-scale feature map; perform global and local image enhancement processing on the reconstructed feature map of spatial information after second-scale downsampling to obtain a third-scale feature map; The third-scale feature map, the second-scale feature map, and the first-scale feature map are processed by cross-scale fusion to obtain a multi-scale fusion feature map; A super-resolution reconstruction map generation module, configured to perform Fourier transform processing on the multi-scale fusion feature map with a fixed convolution kernel to obtain a static branch feature map; after obtaining the degradation descriptor features of the multi-scale fusion feature map, extract convolution features and channel features, and after fusion, obtain a dynamic branch feature map; fuse the dynamic branch feature map with the static branch feature map to obtain a super-resolution reconstruction map.

9. A remote sensing image super-resolution reconstruction device, characterized in that, Including a processor and a memory, wherein when the processor executes the computer program stored in the memory, the remote sensing image super-resolution reconstruction method according to any one of claims 1-7 is implemented.

10. A computer-readable storage medium, characterized in that, For storing a computer program, wherein when the computer program is executed by a processor, the remote sensing image super-resolution reconstruction method according to any one of claims 1-7 is implemented.

Images