Remote sensing image super-resolution reconstruction and detail enhancement deep learning system

Abstract

The invention relates to the field of remote sensing image processing, in particular to a remote sensing image super-resolution reconstruction and detail enhancement deep learning system which comprises a super-resolution reconstruction network module, a texture synthesis detail enhancement network module based on manifold learning, a training management module and a preprocessing module. Wherein the super-resolution reconstruction network module adopts an architecture based on a generative adversarial network, and provides progressive 1 / 4, 1 / 2 and 1-time resolution reconstruction; the manifold learning-based texture synthesis detail enhancement network module extracts texture features through multi-level high-pass filtering, maps the features to a manifold space, generates a texture residual error by adopting a geodesic interpolation principle, and applies manifold constraint optimization for a specific ground feature category so as to obtain a manifold learning-based texture synthesis detail enhancement network. According to the system, two tasks of super-resolution reconstruction and detail enhancement are innovatively combined, more accurate texture feature capture and reconstruction are realized, and the adaptability of the system to different surface feature categories is improved.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing, specifically to a deep learning system for super-resolution reconstruction and detail enhancement of remote sensing images. This system is mainly applied to resolution enhancement and detail enhancement processing of low-resolution remote sensing images acquired through satellite remote sensing and aerial remote sensing, providing high-quality data support for subsequent remote sensing image analysis and applications. Background Technology

[0002] With the rapid development of remote sensing technology, remote sensing imagery has been widely used in many fields such as urban planning, environmental monitoring, and disaster assessment. However, due to limitations in imaging equipment, transmission bandwidth, and storage capacity, acquired remote sensing imagery often suffers from insufficient resolution and loss of detail, which seriously affects the effectiveness of remote sensing imagery in refined applications.

[0003] Traditional super-resolution reconstruction methods mainly include interpolation methods, reconstruction-based methods, and example-based learning methods. Interpolation methods are computationally simple but can easily cause edge blurring; reconstruction-based methods can preserve some edge information, but their effectiveness is limited for complex textures; example-based learning methods rely on high-quality prior samples, and their performance degrades significantly when there are insufficient or mismatched samples.

[0004] In recent years, deep learning technology has made significant progress in the field of image processing, and various deep learning-based super-resolution reconstruction methods have been proposed. However, existing technologies still have the following problems:

[0005] 1. A single network architecture cannot simultaneously handle both super-resolution reconstruction and detail enhancement tasks;

[0006] 2. Insufficient capture of complex ground texture features in remote sensing imagery;

[0007] 3. Lack of adaptive processing mechanisms for different land cover categories;

[0008] 4. The generated high-resolution images often have problems such as unrealistic textures and loss of detail.

[0009] Therefore, there is an urgent need for a system that can effectively combine super-resolution reconstruction and detail enhancement to improve the resolution and detail of remote sensing images. Summary of the Invention

[0010] The purpose of this invention is to provide a deep learning system for super-resolution reconstruction and detail enhancement of remote sensing images. This system achieves high-quality super-resolution reconstruction and detail enhancement of remote sensing images by innovatively combining generative adversarial networks and texture synthesis networks based on manifold learning, overcoming problems such as unrealistic textures and loss of details in existing technologies.

[0011] This invention proposes a deep learning system for super-resolution reconstruction and detail enhancement of remote sensing images, comprising:

[0012] The super-resolution reconstruction network module is used for:

[0013] Acquire low-resolution remote sensing images;

[0014] Features of low-resolution remote sensing images are extracted using a super-resolution reconstruction network based on generative adversarial networks;

[0015] Generate progressive reconstruction results at 1 / 4, 1 / 2, and 1x resolution;

[0016] Output high-resolution reconstructed images;

[0017] A texture synthesis detail enhancement network module based on manifold learning, connected to the super-resolution reconstruction network module, is used for:

[0018] Receive the high-resolution reconstructed image output by the super-resolution reconstruction network module;

[0019] The high-resolution reconstructed image is subjected to multi-level high-pass filtering to generate multi-level texture features;

[0020] The multi-level texture features are mapped to a manifold space to construct a texture feature manifold representation.

[0021] Generating texture residuals based on geodesic interpolation principles;

[0022] For specific land cover categories, manifold constraints are applied to optimize the texture residuals;

[0023] The optimized texture residual is fused with the high-resolution reconstructed image to generate a high-resolution remote sensing image with enhanced details;

[0024] The training management module, connected to both the super-resolution reconstruction network module and the manifold-based texture synthesis detail enhancement network module, is used for:

[0025] Construct a high-resolution training sample set;

[0026] Provide network training parameters and optimization strategies;

[0027] Perform model training and optimization;

[0028] The preprocessing module, connected to the super-resolution reconstruction network module, is used for:

[0029] Preprocess the input remote sensing images;

[0030] Perform data augmentation operations;

[0031] The processed data is then passed to the super-resolution reconstruction network module.

[0032] Preferably, the super-resolution reconstruction network module includes:

[0033] The low-resolution feature extraction unit is used to extract low-resolution image features and generate residual features. The low-resolution feature extraction unit contains 5 U-NET encoder-decoder structures, including 1 convolutional layer with 16 convolutional kernels, 4 downsampling convolution-batch normalization-linear activation-upsampling convolutional structures with residual network structures, and 1 upsampling convolutional layer.

[0034] High-resolution reconstruction architecture unit, used to achieve feature alignment and reconstruction based on U-NET architecture, including low-resolution to high-resolution feature alignment network and residual feature mapping network;

[0035] A high-resolution feature extraction unit is used to extract high-resolution features and fuse them with the residual features;

[0036] The output unit is used to superimpose and fuse the residual features and the output corresponding to the high-resolution image through a residual network to generate the final high-resolution remote sensing image output.

[0037] Preferably, the low-resolution to high-resolution feature alignment network includes five U-NET encoder-decoder structures: a forward low-resolution network, a backward low-resolution network, a forward high-resolution network, a backward high-resolution network, and a mapping network for low-resolution feature to high-resolution feature mapping, wherein the mapping network consists of a single convolutional layer.

[0038] Preferably, the texture synthesis detail enhancement network module based on manifold learning includes:

[0039] Multi-stage high-pass filter unit, used for:

[0040] The high-resolution reconstructed image is subjected to n layers of high-pass filtering to obtain n high-pass filtered images;

[0041] The original high-resolution reconstructed image is retained for subsequent fusion;

[0042] The manifold embedding feature mapping unit, connected to the multi-level high-pass filter unit, is used for:

[0043] Receive the n high-pass filtered images;

[0044] Texture features are mapped to a manifold space using K feature extraction modules;

[0045] Construct a manifold representation of texture features;

[0046] The geodesic residual synthesis unit, connected to the manifold embedding feature mapping unit, is used for:

[0047] Calculate the geodesic distances between feature points in the manifold space;

[0048] Generate a weighted fusion matrix based on geodesic distance;

[0049] Synthesize the initial texture residual map;

[0050] The ground feature sensing manifold constraint unit, connected to the geodesic residual synthesis unit, is used for:

[0051] Identify different land cover categories in an image;

[0052] Specific manifold constraints are constructed for each type of land cover;

[0053] Optimize texture residuals based on a sample library;

[0054] The multi-scale feature fusion unit, connected to the ground feature perception manifold constraint unit and the multi-level high-pass filter unit, is used for:

[0055] Construct a multi-level feature pyramid;

[0056] Adaptive fusion of features at different scales;

[0057] The fusion result is combined with the high-resolution reconstructed image to generate the final detail-enhanced output image.

[0058] Preferably, the high-pass filter in the multi-level high-pass filter unit is designed with learnable parameters, and the size of the high-pass filter is 3×3 or 5×5, and it includes a nonlinear activation function to enhance feature representation.

[0059] Preferably, each of the K feature extraction modules in the manifold embedding feature mapping unit contains a multi-layer convolutional network and a feature transformation layer. Each module outputs K feature channels, each channel representing a dimension on the manifold, and the value of K ranges from 4 to 8.

[0060] 7. The deep learning system for super-resolution reconstruction and detail enhancement of remote sensing images according to claim 4, characterized in that the geodesic residual synthesis unit realizes the approximate calculation of geodesic distance through multi-layer cascaded neighboring point connections, and uses an exponential decay function to calculate the weights, with the weights being larger the closer the distance; the residual synthesis is achieved by combining weighted averaging and nonlinear transformation.

[0061] Preferably, the ground feature sensing manifold constraint unit includes:

[0062] The feature segmentation module is used to identify different feature categories in the image;

[0063] The manifold constraint module is used to construct a specific manifold constraint matrix for each type of land cover.

[0064] The sample optimization module is used to retrieve the most similar texture samples from the sample library and optimize the texture residuals based on L1 constraints.

[0065] Preferably, the training management module adopts a phased training strategy, including:

[0066] In the first stage, the super-resolution reconstruction network module and the manifold learning-based texture synthesis detail enhancement network module are trained independently, while other components are frozen.

[0067] The second phase involves end-to-end fine-tuning of the entire system to optimize the synergy between modules.

[0068] The loss function of the training management module includes texture fidelity loss, manifold consistency loss, and ground feature specificity loss.

[0069] Preferably, the preprocessing module includes:

[0070] Image acquisition unit, used to acquire raw remote sensing images;

[0071] The data augmentation unit is used to perform multi-scale cropping, rotation and flipping, and noise addition operations to increase the diversity of training samples;

[0072] An image rearrangement unit is used to perform sample augmentation using a pyramid pattern and a block pattern, wherein the pyramid pattern includes multiple sets of images with different resolutions, and the block pattern includes multiple sub-images randomly combined into an image with the same number of image blocks.

[0073] The beneficial effects of this invention include:

[0074] 1. By adopting a dual-network collaborative architecture, the two tasks of super-resolution reconstruction and detail enhancement are effectively combined, thereby improving the overall performance of the system;

[0075] 2. Employing a texture synthesis network based on manifold learning can more accurately capture and reconstruct complex ground texture features, making the reconstructed images more realistic and natural;

[0076] 3. By introducing a manifold constraint mechanism based on ground cover perception, adaptive texture optimization for different ground cover categories is achieved, improving the adaptability and accuracy of the reconstruction results;

[0077] 4. The progressive multi-resolution reconstruction mechanism provides 1 / 4, 1 / 2, and 1x reconstruction resolutions to meet the needs of different application scenarios;

[0078] 5. The application of geodesic interpolation principle solves the problems of pseudo textures and unnatural transitions in traditional methods, thus improving the quality of texture synthesis;

[0079] 6. The overall system has strong generalization ability and adaptability, and can effectively process various complex remote sensing images. Attached Figure Description

[0080] Figure 1 This is a schematic diagram of the overall architecture of the deep learning system for remote sensing image super-resolution reconstruction and detail enhancement of the present invention;

[0081] Figure 2 This is a schematic diagram of the super-resolution reconstruction network module of the present invention;

[0082] Figure 3 This is a schematic diagram of the texture synthesis detail enhancement network module based on manifold learning of the present invention;

[0083] Figure 4 This is a schematic diagram of the working process of the multi-level high-pass filter unit of the present invention;

[0084] Figure 5 This is a schematic diagram of the structure of the manifold embedding feature mapping unit of the present invention;

[0085] Figure 6 This is a schematic diagram illustrating the working principle of the geodesic residual synthesis unit of the present invention;

[0086] Figure 7 This is a schematic diagram of the structure of the ground feature sensing manifold constraint unit of the present invention;

[0087] Figure 8 This is a schematic diagram of the workflow of the multi-scale feature fusion unit of the present invention;

[0088] Figure 9 This is a schematic diagram of the training strategy for the training management module of the present invention;

[0089] Figure 10 This is a schematic diagram of the data processing flow of the preprocessing module of the present invention. Detailed Implementation

[0090] Please refer to Figures 1-10 The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0091] Reference Figure 1 The remote sensing image super-resolution reconstruction and detail enhancement deep learning system provided by the present invention mainly includes a super-resolution reconstruction network module 1, a texture synthesis detail enhancement network module 2 based on manifold learning, a training management module 3, and a preprocessing module 4.

[0092] The super-resolution reconstruction network module 1 is used to acquire low-resolution remote sensing images. It extracts features of low-resolution remote sensing images through a super-resolution reconstruction network based on generative adversarial networks, generates progressive reconstruction results at 1 / 4, 1 / 2 and 1x resolution, and outputs high-resolution reconstructed images.

[0093] The texture synthesis detail enhancement network module 2, based on manifold learning, is connected to the super-resolution reconstruction network module 1. It receives the high-resolution reconstructed image output from the super-resolution reconstruction network module 1, performs multi-level high-pass filtering on the high-resolution reconstructed image to generate multi-level texture features, and maps these features to the manifold space to construct a texture feature manifold representation. Furthermore, this module generates texture residuals based on the geodesic interpolation principle, applies manifold constraints to optimize the texture residuals for specific land cover categories, and finally fuses the optimized texture residuals with the high-resolution reconstructed image to generate a high-resolution remote sensing image with enhanced details.

[0094] The training management module 3 is connected to the super-resolution reconstruction network module 1 and the texture synthesis detail enhancement network module 2 based on manifold learning, respectively. It is responsible for building a high-resolution training sample set, providing network training parameters and optimization strategies, and performing model training and optimization.

[0095] The preprocessing module 4 is connected to the super-resolution reconstruction network module 1 and is used to preprocess the input remote sensing images, perform data augmentation operations, and transmit the processed data to the super-resolution reconstruction network module 1.

[0096] Reference Figure 2 The super-resolution reconstruction network module 1 mainly includes a low-resolution feature extraction unit 11, a high-resolution reconstruction architecture unit 12, a high-resolution feature extraction unit 13, and an output unit 14.

[0097] The low-resolution feature extraction unit 11 is used to extract low-resolution image features and generate residual features. In a preferred embodiment of the present invention, this unit includes five U-NET encoder-decoder structures, specifically including one convolutional layer with 16 convolutional kernels, four downsampling convolution-batch normalization-linear activation-upsampling convolutional structures with residual network structures, and one upsampling convolutional layer. This multi-level feature extraction mechanism can effectively capture image features at different scales, providing rich feature representations for subsequent super-resolution reconstruction.

[0098] Preferably, the convolutional kernel size is 3×3, batch normalization uses standard normalization with a mean of 0 and a variance of 1, and the linear activation function is ReLU (Rectified Linear Unit). Practice shows that this configuration performs well in remote sensing image feature extraction. The stride of the convolutional layer is 1 to retain more spatial information.

[0099] The high-resolution reconstruction architecture unit 12 implements feature alignment and reconstruction based on the U-NET architecture, including a low-resolution to high-resolution feature alignment network and a residual feature mapping network. The low-resolution to high-resolution feature alignment network comprises five U-NET encoder-decoder structures: a forward low-resolution network, a backward low-resolution network, a forward high-resolution network, a backward high-resolution network, and a mapping network for low-resolution to high-resolution feature mapping. The mapping network consists of a single convolutional layer, achieving feature space mapping transformation in a simple and efficient manner.

[0100] This bidirectional feature alignment network design can more comprehensively capture the correspondence between low-resolution and high-resolution features, improving the accuracy of feature alignment. The forward network is mainly responsible for feature extraction and transformation from input to output, while the backward network focuses on feature mapping from output to input. The combination of the two forms a complete feature alignment mechanism.

[0101] The high-resolution feature extraction unit 13 is used to extract high-resolution features and fuse them with residual features. This unit extracts rich feature representations from high-resolution images through a multi-layer convolutional network and effectively fuses them with the residual features generated by the low-resolution feature extraction unit, providing comprehensive feature support for the final image reconstruction.

[0102] Output unit 14 uses a residual network to superimpose and fuse the residual features and the output corresponding to the high-resolution image, generating the final high-resolution remote sensing image output. The design of the residual connection effectively alleviates the gradient vanishing problem in deep network training, while preserving the original feature information and improving the quality of the reconstructed image.

[0103] Reference Figure 3 The texture synthesis detail enhancement network module 2 based on manifold learning includes a multi-level high-pass filtering unit 21, a manifold embedding feature mapping unit 22, a geodesic residual synthesis unit 23, a ground feature perception manifold constraint unit 24, and a multi-scale feature fusion unit 25. These units together constitute a complete texture detail enhancement processing flow.

[0104] The multi-level high-pass filtering unit 21 is used to perform n-level high-pass filtering on the high-resolution reconstructed image to obtain n high-pass filtered images, while retaining the original high-resolution reconstructed image for subsequent fusion.

[0105] In one embodiment of the present invention, the high-pass filter employs a learnable parameter design, unlike traditional fixed filter kernels. The filter size can be 3×3 or 5×5, and it includes a nonlinear activation function to enhance feature representation. Practice has shown that when n takes values ​​from 3 to 5, it can effectively capture texture features across different frequency ranges, making it suitable for most remote sensing image scenarios.

[0106] The mathematical principle of multi-stage high-pass filtering can be expressed as:

[0107] ,

[0108] in: Let represent the high-pass filtering result of the i-th layer, and denote the output feature map; Let be the i-th layer high-pass filter, and represent the learnable filter parameters; The input is a high-resolution reconstructed image with dimensions H×W×C, where H represents the image height, W represents the image width, and C represents the number of channels; * indicates a convolution operation.

[0109] To enhance the expressive power of the features, a nonlinear activation function is applied to the filtering results:

[0110] ,

[0111] in: This is the high-pass feature map of the i-th layer after processing by the activation function; ReLU is the modified linear unit activation function, defined as ReLU(x) = max(0, x), used to introduce nonlinearity and eliminate negative values.

[0112] The manifold embedding feature mapping unit 22 is connected to the multi-level high-pass filtering unit 21. It is used to receive n high-pass filtered images and map the texture features to the manifold space through K feature extraction modules to construct the manifold representation of the texture features.

[0113] In a preferred embodiment of the present invention, each of the K feature extraction modules includes a multi-layer convolutional network and a feature transformation layer. Each module outputs K feature channels, each channel representing one dimension on the manifold. The value of K is preferably in the range of 4 to 8, which achieves a good balance between expressive power and computational complexity.

[0114] The core idea of ​​manifold embedding is to view texture features as low-dimensional manifold structures in a high-dimensional space. Each feature extraction module corresponds to a local tangent space on the manifold. By combining these modules, the geometric properties of texture features can be fully captured.

[0115] The feature mapping process can be represented as:

[0116] ,

[0117] in: For the kth feature channel, there is a dimension on the manifold with size H×W; The mapping function for the k-th feature extraction module includes convolution, activation, and normalization operations; Let be an n-layer high-pass feature map, with each layer having a size of H×W×C_i, where C_i represents the number of channels in the i-th layer.

[0118] Simultaneously, the unit also calculates the correlation matrix between the feature map location and the filtering location, which is used to adaptively adjust the feature weights:

[0119] ,

[0120] in: The correlation between positions p and q is given, with a value range of [-1, 1]. and Let p and q be the feature vectors at positions p and q, respectively, both with dimension K. This is a vector dot product operation that calculates the sum of the element-wise products of two vectors. The L2 norm of a vector, i.e. .

[0121] The geodesic residual synthesis unit 23 is connected to the manifold embedding feature mapping unit 22. It is used to calculate the geodesic distance between feature points in the manifold space, generate a weight fusion matrix based on the geodesic distance, and synthesize a preliminary texture residual map.

[0122] In one embodiment of the invention, the unit achieves approximate calculation of geodesic distances through multi-layered cascaded neighboring point connections, and uses an exponential decay function to calculate weights, with closer distances resulting in larger weights. Residual synthesis is achieved by combining weighted averaging with nonlinear transformation.

[0123] The approximate calculation of geodesic distance can be expressed as:

[0124] ,

[0125] in: The geodesic distance between points p and q; Adjacent points and The Euclidean distance between them is calculated as follows: ; Let p be a path from p to q, containing m+1 points; This indicates selecting the path with the smallest sum of distances among all possible paths.

[0126] The weighting calculation based on geodesic distance uses an exponential decay function:

[0127] ,

[0128] in: Let be the weight between points p and q, with a value range of (0,1]. The parameter used to control the decay rate is usually set to the average value of the geodesic distance; It is a natural exponential function.

[0129] The process of synthesizing texture residuals can be represented as:

[0130] ,

[0131] in: Let K be the texture residual at position p, with dimension K. The feature space contains all pixels in the image; Let q be the feature at position q, with dimension K; This represents a weighted summation of all points in the feature space.

[0132] The ground feature perception manifold constraint unit 24 is connected to the geodesic residual synthesis unit 23. It is used to identify different ground feature categories in the image, construct specific manifold constraints for each type of ground feature, and optimize texture residuals based on the sample library.

[0133] This unit comprises a land cover segmentation module, a manifold constraint module, and a sample optimization module. The land cover segmentation module identifies different land cover categories in an image and can be implemented using a pre-trained semantic segmentation network. The manifold constraint module constructs a specific manifold constraint matrix for each land cover category, ensuring that texture features conform to the characteristics of the land cover. The sample optimization module retrieves the most similar texture samples from the sample library and optimizes the texture residuals based on L1 constraints.

[0134] The process of land feature segmentation can be represented as:

[0135] ,

[0136] in: The result is a land cover segmentation diagram with dimensions H×W. The value of each pixel represents the corresponding land cover category number. Seg is the segmentation function, which is usually a pre-trained semantic segmentation network. The input image is the original high-resolution reconstructed image.

[0137] The manifold constraint matrix constructed for each type of land cover can be expressed as:

[0138] ,

[0139] in: Let N_t be the manifold constraint matrix for land cover category t, with size N_t×N_t, where N_t is the number of pixels in category t; The function is designed based on the characteristics of the land cover category; This is the result of feature segmentation; This is a manifold characteristic.

[0140] Sample-based texture residual optimization employs L1 constraints:

[0141] ,

[0142] in: The texture residual for the optimized land cover category t; This is the initial texture residual; The reference texture residual is from the sample library; This is the regularization coefficient, typically set to 0.01 to 0.1; This is a regularization term used to control the smoothness of texture residuals, typically employing Total Variation regularization. This indicates finding the parameter values ​​that minimize the objective function; This represents the L1 norm, which is the sum of the absolute values ​​of all elements.

[0143] The multi-scale feature fusion unit 25 is connected to the ground object perception manifold constraint unit 24 and the multi-level high-pass filter unit 21 to construct a multi-level feature pyramid, adaptively fuse features of different scales, and combine the fusion result with the high-resolution reconstructed image to generate the final detail-enhanced output image.

[0144] In one embodiment of the invention, the feature pyramid comprises 3 to 5 levels, covering different spatial scales. The fusion weights are adaptively calculated using an attention mechanism, and the residual connections employ skip connections to preserve the original information.

[0145] The construction of a multi-level feature pyramid can be represented as:

[0146] ,

[0147] in: This is the i-th level feature pyramid, with size . × × ; This is a downsampling function, typically implemented using average pooling or max pooling. The optimized texture residual has a size of × × The adaptive fusion weights are calculated using an attention mechanism.

[0148] ,

[0149] in: The attention weights for the i-th level feature, with dimensions equal to... same; For attention calculation functions, a combination of channel attention and spatial attention is typically used. This is the i-th level feature pyramid; This is the original high-resolution reconstructed image.

[0150] The final feature fusion process can be represented as:

[0151] ,

[0152] in: The enhanced output image has a size of [size missing]. × × ; The original high-resolution reconstructed image, with a size of × × ; Let be the attention weight for the i-th level feature; This is the i-th level feature pyramid; This is an upsampling function used to adjust features of different scales to the same size. This indicates a weighted summation of features at all levels.

[0153] The training management module 3 is connected to the super-resolution reconstruction network module 1 and the manifold learning-based texture synthesis detail enhancement network module 2, respectively. It is used to build a high-resolution training sample set, provide network training parameters and optimization strategies, and perform model training and optimization.

[0154] This module employs a phased training strategy, including the first phase of independently training the super-resolution reconstruction network module and the manifold learning-based texture synthesis detail enhancement network module, while freezing other components; the second phase involves end-to-end fine-tuning of the entire system to optimize the synergistic effect between modules.

[0155] The loss function design includes texture fidelity loss, manifold consistency loss, and feature-specific loss. Texture fidelity loss measures the similarity between the reconstructed texture and the real texture, manifold consistency loss ensures that the features are reasonably distributed on the manifold, and feature-specific loss targets specific optimization objectives for different feature categories.

[0156] Texture fidelity loss can be expressed as:

[0157] ,

[0158] in: For texture fidelity loss, is a non-negative scalar; The enhanced image output by the system has dimensions of H×W×C; It is a true high-resolution image with dimensions of H×W×C; It is the squared L2 norm, which is the sum of the squares of the differences between the elements.

[0159] The manifold consistency loss can be expressed as:

[0160] ,

[0161] in: The loss is the manifold consistency loss, and it is a non-negative scalar. It is a set of neighborhood point pairs, which usually contains spatially adjacent pixel pairs; The geodesic distance between points p and q; The Euclidean distance between the corresponding feature vectors; This represents the summation of all neighboring point pairs.

[0162] Specific losses of land features can be expressed as:

[0163] ,

[0164] in: For specific losses of ground features, it is a non-negative scalar quantity; This is an index for land cover categories, ranging from 1 to T, where T is the total number of land cover categories; This is a weighting coefficient used to adjust the relative importance of different land cover categories; The texture residual for the optimized land cover category t; The reference texture residual is from the sample library; It is an L1 norm; This represents the summation over all land cover categories.

[0165] The total loss function is:

[0166] ,

[0167] in This is the total loss function, used for network optimization; , , These are weighting coefficients, which control the relative importance of each loss term. In practical applications, these coefficients can be adjusted according to the specific dataset and task requirements. Preferably, The values ​​were set to 1.0, 0.1, and 0.5 respectively.

[0168] The preprocessing module 4 is connected to the super-resolution reconstruction network module 1 and is used to preprocess the input remote sensing images, perform data augmentation operations, and transmit the processed data to the super-resolution reconstruction network module 1.

[0169] This module includes an image acquisition unit, a data augmentation unit, and an image rearrangement unit. The image acquisition unit is used to acquire raw remote sensing images. The data augmentation unit is used to perform multi-scale cropping, rotation and flipping, and noise addition operations to increase the diversity of training samples. The image rearrangement unit is used to augment samples using pyramid and block patterns.

[0170] The pyramid mode comprises multiple sets of images at different resolutions, used to train the system to learn its ability to reconstruct details. The block mode comprises multiple sub-images randomly combined into an image with the same number of blocks, used to eliminate information redundancy between image blocks and improve sample utilization efficiency.

[0171] Multi-scale cropping typically involves randomly cropping regions of different sizes from the original image, such as 64×64, 128×128, and 256×256 pixel image patches, to increase the diversity of scale variations. Rotation and flipping include rotations of 0°, 90°, 180°, and 270°, as well as horizontal and vertical flips, for a total of eight transformations. Noise addition adds Gaussian noise (mean 0, standard deviation 0.01 to 0.05) or salt-and-pepper noise (probability 0.001 to 0.01) to the original image to improve the robustness of the model.

[0172] These preprocessing and data augmentation operations can significantly increase the quantity and diversity of training samples, thereby improving the model's generalization ability and adaptability.

[0173] In practical applications, the remote sensing image super-resolution reconstruction and detail enhancement deep learning system of the present invention can process various types of remote sensing images, including but not limited to optical remote sensing images, multispectral remote sensing images and hyperspectral remote sensing images.

[0174] Taking optical remote sensing image processing as an example, the input low-resolution image (e.g., 2-meter resolution) is first normalized and data-enhanced by a preprocessing module, and then input into a super-resolution reconstruction network module. This module, based on a generative adversarial network architecture, generates a preliminary high-resolution reconstructed image (e.g., 0.5-meter resolution) through multi-level feature extraction and mapping. Next, a texture synthesis detail enhancement network module based on manifold learning processes the reconstructed image, extracts texture features through multi-level high-pass filtering, generates natural texture residuals using manifold embedding and geodesic interpolation principles, and performs adaptive optimization for different land cover categories (e.g., buildings, roads, vegetation), ultimately generating a high-quality remote sensing image with rich details.

[0175] Experimental results show that, compared with existing technologies, the present invention significantly improves objective indicators such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), with an average PSNR improvement of 1.5-2.5 dB and an SSIM improvement of 0.02-0.04. More importantly, the present invention excels in maintaining the realism of detailed textures, especially in the reconstruction of complex features such as building outlines, road networks, and vegetation textures, where the results are significantly better than existing methods.

[0176] This invention provides a deep learning system for super-resolution reconstruction and detail enhancement of remote sensing images. By innovatively combining generative adversarial networks and manifold-based texture synthesis networks, it achieves high-quality super-resolution reconstruction and detail enhancement of remote sensing images. This system is not only theoretically innovative but also demonstrates significant performance advantages and broad applicability in practical applications, providing a new technical approach for super-resolution reconstruction and detail enhancement of remote sensing images.

[0177] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, various changes and modifications can be made without departing from the spirit and scope of the invention, and these changes and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A deep learning system for super-resolution reconstruction and detail enhancement of remote sensing images, characterized in that, The method comprises the following steps: An ultra-resolution reconstruction network module is used for: Obtaining a low-resolution remote sensing image; Extracting low-resolution remote sensing image features through an ultra-resolution reconstruction network based on a generative adversarial network; Generating progressive 1 / 4, 1 / 2 and 1 times resolution reconstruction results; Outputting a high-resolution reconstructed image; A texture synthesis detail enhancement network module based on manifold learning is connected with the ultra-resolution reconstruction network module and used for: Receiving the high-resolution reconstructed image output by the ultra-resolution reconstruction network module; Performing multi-level high-pass filtering processing on the high-resolution reconstructed image to generate multi-level texture features; Mapping the multi-level texture features to a manifold space to construct a texture feature manifold representation; Generating texture residuals based on the principle of geodesic interpolation; Optimizing the texture residuals for specific ground object categories by applying manifold constraints; Fusing the optimized texture residuals with the high-resolution reconstructed image to generate a high-resolution remote sensing image with enhanced details; A training management module is connected with the ultra-resolution reconstruction network module and the texture synthesis detail enhancement network module based on manifold learning and used for: Constructing a high-resolution training sample set; Providing network training parameters and optimization strategies; Performing model training and optimization; A preprocessing module is connected with the ultra-resolution reconstruction network module and used for: Preprocessing the input remote sensing image; Performing data enhancement operations; Passing the processed data to the ultra-resolution reconstruction network module. 2.The remote sensing image super-resolution reconstruction and detail enhancement deep learning system of claim 1, wherein, The ultra-resolution reconstruction network module comprises: A low-resolution feature extraction unit for extracting low-resolution image features and generating residual features, which contains five U-NET encoder-decoder structures, including one convolutional layer with 16 convolutional kernels, four down-sampling convolution-batch normalization-linear activation-up-sampling convolution structures with residual network structures, and one up-sampling convolutional layer; A high-resolution reconstruction architecture unit for realizing feature alignment and reconstruction based on a U-NET architecture, including a low-resolution to high-resolution feature alignment network and a residual feature mapping network; A high-resolution feature extraction unit for extracting high-resolution features and fusing them with the residual features; An output unit for superimposing and fusing the residual features and the output corresponding to the high-resolution image through a residual network to generate a final high-resolution remote sensing image output. 3.The remote sensing image super-resolution reconstruction and detail enhancement deep learning system of claim 2, wherein, The low-resolution to high-resolution feature alignment network contains five U-NET encoder-decoder structures, which are a forward low-resolution network, a backward low-resolution network, a forward high-resolution network, a backward high-resolution network, and a mapping network for low-resolution feature to high-resolution feature mapping, and the mapping network is composed of a single convolutional layer. 4.The remote sensing image super-resolution reconstruction and detail enhancement deep learning system of claim 1, wherein, The texture synthesis detail enhancement network module based on manifold learning comprises: A multi-level high-pass filtering unit for: Performing n-layer high-pass filtering on the high-resolution reconstructed image to obtain n high-pass filtered images; Reserving the original high-resolution reconstructed image for subsequent fusion; A manifold embedding feature mapping unit connected with the multi-level high-pass filtering unit and used for: Receiving the n high-pass filtered images; mapping texture features to manifold space by K feature extraction modules; constructing manifold representation of texture features; a geodesic residual synthesis unit connected to the manifold embedding feature mapping unit, configured to: calculate geodesic distance between feature points in manifold space; generate weight fusion matrix based on geodesic distance; synthesize preliminary texture residual map; a feature-aware manifold constraint unit connected to the geodesic residual synthesis unit, configured to: identify different feature categories in the image; construct specific manifold constraint for each feature category; optimize texture residual based on sample library; a multi-scale feature fusion unit connected to the feature-aware manifold constraint unit and the multi-level high-pass filtering unit, configured to: construct multi-level feature pyramid; adaptively fuse features of different scales; combine the fusion result with the high-resolution reconstructed image to generate final detail-enhanced output image. 5.The remote sensing image super-resolution reconstruction and detail enhancement deep learning system of claim 4, wherein, The high-pass filter in the multi-level high-pass filtering unit is designed with learnable parameters, and the size of the high-pass filter is 3x3 or 5x5, and a nonlinear activation function is included to enhance feature expression. 6.The remote sensing image super-resolution reconstruction and detail enhancement deep learning system of claim 4, wherein, Each of the K feature extraction modules in the manifold embedding feature mapping unit includes a multi-layer convolutional network and a feature transformation layer, and each module outputs K feature channels, each channel representing a dimension on the manifold, and the value of K ranges from 4 to 8. 7.The remote sensing image super-resolution reconstruction and detail enhancement deep learning system of claim 4, wherein, The geodesic residual synthesis unit approximates the calculation of geodesic distance by multi-level cascading of adjacent point connections, and uses an exponential decay function to calculate the weight, with the weight increasing as the distance decreases, and the residual synthesis is realized by combining weighted average with nonlinear transformation. 8.The remote sensing image super-resolution reconstruction and detail enhancement deep learning system of claim 4, wherein, The feature-aware manifold constraint unit includes: a feature segmentation module for identifying different feature categories in the image; a manifold constraint module for constructing a specific manifold constraint matrix for each feature category; a sample optimization module for retrieving the most similar texture samples from the sample library and optimizing the texture residual based on L1 constraint. 9.The remote sensing image super-resolution reconstruction and detail enhancement deep learning system of claim 1, wherein, The training management module adopts a phased training strategy, including: In the first phase, the super-resolution reconstruction network module and the texture synthesis detail enhancement network module based on manifold learning are trained independently, and other components are frozen; In the second phase, the entire system is fine-tuned end-to-end to optimize the collaborative effect between modules; The loss function of the training management module includes texture fidelity loss, manifold consistency loss, and feature-specific loss. 10.The remote sensing image super-resolution reconstruction and detail enhancement deep learning system of claim 1, wherein, The preprocessing module includes: an image acquisition unit for obtaining original remote sensing images; a data enhancement unit for performing multi-scale cropping, rotation, flipping, and noise addition operations to increase the diversity of training samples; an image rearrangement unit for sample enhancement using pyramid mode and block mode, wherein the pyramid mode includes multiple groups of images with different resolutions, and the block mode includes multiple sub-images randomly combined into images consistent with the number of image blocks.

Images