Super-resolution high-definition image generation method based on optimization generative neural network

Abstract

The application discloses a kind of super-resolution high-definition image generation methods based on optimization generative neural network, constructs super network model: based on hierarchical encoding-decoding structure, utilize Mamba residual block to construct an end-to-end super network model;Shallow feature extraction module is converted into high-dimensional feature space by single-layer convolution to input, then multi-scale feature processing is carried out by hierarchical Mamba processing module, finally, the final super-resolution image is obtained by upsampling module, multi-scale feature fusion module and image reconstruction module.The application is used for the processing of river channel satellite remote sensing image, and high-quality image super-resolution processing can be realized efficiently.Can the existing 10m resolution river remote sensing image be converted into 2.5m resolution (4 times super-resolution), effectively improve the river detail recognition ability.

Description

Technical Field

[0001] This invention pertains to image processing technology for satellite remote sensing images of rivers and waterways, specifically a method for generating super-resolution high-definition images based on optimized generative neural networks, belonging to the category of image super-resolution methods based on the Mamba state-space model. Background Technology

[0002] In the information age, images have become a crucial means for humans to acquire and transmit information. However, due to factors such as inclement weather, limitations in imaging equipment resolution, and image compression, acquired images often have low resolution. Low-resolution images not only affect human visual perception but also severely impact the performance of various image-based intelligent information processing systems. Therefore, image super-resolution processing is essential. This is especially true for remote sensing satellite imagery of rivers, some of which have complex and varied morphologies. Currently, the widely used Sentinel-2 satellite remote sensing imagery has a resolution of 10m, which is insufficient to clearly depict the detailed features of rivers, such as the precise location of river boundaries, the spatial distribution of sandbars and shoals, and the dynamic processes of river erosion and deposition. These detailed features are crucial for levee management, river sand mining supervision, water conservancy project planning, and river ecological protection. Furthermore, traditional high-resolution satellite imagery is costly, time-consuming, and lacks continuous coverage.

[0003] Currently, image super-resolution algorithms are mainly divided into three categories: interpolation-based super-resolution methods, image prior-based super-resolution methods, and deep learning-based super-resolution methods. The first category, interpolation-based super-resolution methods (such as bilinear interpolation and bicubic interpolation), is computationally simple but has limited effectiveness, failing to effectively recover lost high-frequency details in the image. The second category, image prior-based methods, perform super-resolution by learning the statistical properties of the image; these methods are simple but have weak generalization ability. The third category, deep learning-based super-resolution methods, learn the features of high-resolution images from data by constructing neural network models; this method is the most widely used.

[0004] Early super-resolution methods based on deep learning employed convolutional neural networks (CNNs), such as SRCNN and SRResNet. While these methods achieved significant improvements over traditional approaches, the receptive field of convolutional operations was limited, making it difficult to effectively model global dependencies in images. To address this issue, the Transformer model was introduced in recent years, utilizing a self-attention mechanism to establish global dependencies. However, the computational complexity of Transformer's self-attention mechanism is O(n²), where n is the sequence length. For high-resolution image processing, the computational cost of the self-attention matrix is ​​enormous, resulting in extremely high GPU memory consumption, which is detrimental to practical applications.

[0005] Existing methods present a trade-off between computational efficiency and super-resolution performance, making it difficult to achieve both high-quality super-resolution and fast inference simultaneously.

[0006] Therefore, there is an urgent need for an image super-resolution method that can effectively model long-range dependencies while maintaining computational efficiency for processing satellite remote sensing images of rivers and waterways. Summary of the Invention

[0007] To address the above problems, the objective of this invention is to provide a method for generating super-resolution high-definition images based on optimized generative neural networks, achieving efficient and high-quality image super-resolution processing. This objective is achieved through the following technical solution:

[0008] A method for generating super-resolution high-definition images based on optimized generative neural networks specifically includes the following steps:

[0009] S1: Construct the training dataset for the model: Obtain the high-resolution image dataset of remote sensing satellites, and obtain the corresponding low-resolution images of remote sensing satellites through downsampling. Create the low-resolution-high-resolution image pairs required for training, and process the image pair samples to obtain the training dataset for the model.

[0010] S2: Constructing the Super-Resolution Network Model: Based on a hierarchical encoder-decoder structure, an end-to-end super-resolution network model is constructed using Mamba residual blocks. The architecture of the super-resolution network model includes: a shallow feature extraction module, Mamba processing modules at layers 1-3, a multi-scale feature fusion module, an upsampling module, and an image reconstruction module. The shallow feature extraction module converts the input into a high-dimensional feature space through a single-layer convolution, then performs multi-scale feature processing through the hierarchical Mamba processing module, and finally obtains the final super-resolution image through the upsampling module, the multi-scale feature fusion module, and the image reconstruction module.

[0011] S3: Model Training: Input the training dataset obtained in S1 into the super-resolution network model built in S2. During the training process, calculate the loss through multiple loss functions, continuously iterate and update the model parameters, and finally obtain the trained super-resolution network model.

[0012] S4: Use the super-resolution network model trained in S3 to input the low-resolution image of the remote sensing satellite to be tested for image super-resolution.

[0013] Further optimization, step S1 specifically includes the following steps:

[0014] S11: Obtain a high-resolution image dataset, denoted as... , where H and W represent the height and width of the image respectively, and 3 represents the RGB three channels;

[0015] S12: Downsample the high-resolution image to obtain the corresponding low-resolution image; the downsampling uses bicubic interpolation, with a downsampling factor of 4. ;

[0016] S13: Randomly crop a patch from the low-resolution and high-resolution image pairs; set the length and width of the high-resolution patch to be 4 times the length and width of the low-resolution patch, and crop the image at a random position in each Epoch to generate multiple training samples.

[0017] S14: Perform data augmentation on the cropped image; the data augmentation includes random horizontal flipping, random vertical flipping, random rotation, and adding Gaussian noise.

[0018] S15: Divide the enhanced low-resolution to high-resolution image samples into training, validation, and test sets, and use them as input to the super-resolution network model.

[0019] Further optimization involves constructing a super-resolution network model in step S2, such as... Figure 1 As shown, the specific steps include:

[0020] S21: Shallow Feature Extraction: Feature extraction from low-resolution images using a single-layer 3×3 convolution.

[0021]

[0022] in, For convolution kernel, For bias terms, The ReLU activation function is used, and * represents the convolution operation; the output dimension is F. shallow ;

[0023] The purpose of this layer is to transform the image from RGB three-channel to a higher-dimensional feature space, facilitating subsequent processing. Simultaneously, the output of this layer is saved as a baseline for residual connections.

[0024] S22: Hierarchical Mamba Feature Processing

[0025] The network adopts a three-layer hierarchical structure, forming an encoder-decoder framework:

[0026] Encoding path, downsampling process:

[0027] Layer 1: At full resolution (H×W), six consecutive Mamba residual blocks (RMAB) are applied for feature processing.

[0028] In this embodiment of the invention, the first layer is processed using Mamba. The shallow layer features F... shallowFThe sequence is divided into 8×8 patches and flattened into a one-dimensional sequence. The input is then processed into six concatenated Mamba residual blocks (RMABs). Each RMAb contains components such as a Mamba module, LayerNorm, and MLP. The Mamba module performs linear-time sequence modeling based on a state-space model. The features processed in the first layer are restored to a two-dimensional form with dimensions H / 4×W / 4×64.

[0029] Layer 2: The features of layer 1 are downsampled by convolution with stride of 2 to obtain a 1 / 2 resolution feature map (H / 2×W / 2), and then processed by 6 RMAB blocks.

[0030] In this embodiment of the invention, downsampling is performed to the second layer (1 / 2 resolution). Downsampling is achieved through convolutions with a stride of 2. This convolutional layer uses a 3×3 convolutional kernel with a stride of 2. After downsampling, a feature map with a resolution of H / 8×W / 8 is obtained. The second layer also contains 6 RMAB blocks, and the processing flow is the same as the first layer. After processing, F_level2 is obtained, with dimensions of H / 8×W / 8×64.

[0031] Layer 3: The features of layer 2 are further downsampled by convolution with stride of 2 to obtain a 1 / 4 resolution feature map (H / 4×W / 4), and then processed by 6 RMAB blocks.

[0032] In this embodiment of the invention, downsampling is performed up to layer 3 (1 / 4 resolution). Further downsampling using convolutions with a stride of 2 yields a feature map with a resolution of H / 16×W / 16. Layer 3 contains 6 RMAB blocks; this is the deepest layer in the network and has the largest receptive field. After processing, F_level3 is obtained, with dimensions H / 16×W / 16×64.

[0033] Decoding path, upsampling process:

[0034] Upsampling begins from layer 3: A 2x upsampling is performed through pixel rearrangement to restore half the resolution. The principle is: convolution expands the number of channels by a factor of four (from 64 to 256), then rearranges these channels to double the spatial resolution. After upsampling, the resolution is restored to H / 8 × W / 8.

[0035] Feature fusion: The upsampled features are concatenated with the second-layer skip connection features stored in the encoding path, and then fused using three RMAB blocks. In this embodiment, the upsampled features are concatenated with the second-layer features stored in the encoding path, resulting in a feature map with dimensions H / 8×W / 8×128 (64+64=128). After concatenation, the feature map is fused using three RMAB blocks, and then a 1×1 convolution is used to reduce the number of channels from 128 back to 64, resulting in F_level2_fuse with dimensions H / 8×W / 8×64.

[0036] Continue upsampling: Upsample back to layer 1. Continue upsampling by 2x through pixel rearrangement to restore the resolution to H / 4×W / 4.

[0037] The second feature fusion involves concatenating the upsampled features with the skip connection features from layer 1 to obtain a feature map with dimensions H / 4×W / 4×128. This is then fused using three RMAB blocks. After fusion, a 1×1 convolution reduces the number of channels from 128 back to 64, resulting in F_level1_fuse with dimensions H / 4×W / 4×64.

[0038] S23: Feature Fusion and Upsampling

[0039] The features after multi-layer fusion are summarized through dense connections and further processed by convolution. Pixel rearrangement is used to achieve 4x upsampling (through two cascaded 2x upsampling steps).

[0040] S24: Image Reconstruction and Residual Learning

[0041] The low-resolution image is upsampled to the target resolution using bicubic interpolation, fused with the network output features, and finally reconstructed using two convolutional layers to generate the final super-resolution image.

[0042] Furthermore, the Mamba residual block structure is as follows: Figure 2 As shown, the processing flow for a single RMAB is as follows:

[0043] Layer 1 Mamba processing (full resolution). Shallow features are segmented into 8×8 patches, and each patch is flattened into a one-dimensional sequence with a length of (H×W) / 64. Each token has a dimension of 4096 (calculated from 8×8×64). The sequence is then fed into six concatenated Mamba residual blocks (RMABs) for processing.

[0044] The processing flow for each RMAB (Mamba residual block) includes the following steps:

[0045] Step 1: Input x is normalized using LayerNorm to obtain x_norm1. LayerNorm standardizes the features, making network training more stable. The calculation method is as follows: calculate the mean and standard deviation of the features for each sample, then standardize them, multiply by a learnable scaling factor, and add a learnable offset term.

[0046] Step 2: x_norm1 is input into the Mamba module to obtain x_mamba. The Mamba module is the core innovation of this invention, which performs linear-time sequence modeling based on the State-Space Model (SSM).

[0047] Step 3: Obtain x_res1 = x + α1 × x_mamba through residual connections, where α1 is a learnable residual weight. This design allows the network to dynamically adjust the contribution of the Mamba module.

[0048] Step 4: x_res1 is normalized through the second layer to obtain x_norm2.

[0049] Step 5: x_norm2 is input into an MLP (Multilayer Perceptron), which consists of two linear layers and a GELU activation function. The first layer expands the dimension, and after passing through the GELU activation function, the second layer reduces the dimension back to x_mlp. The GELU activation function preserves gradient information better than ReLU.

[0050] Step 6: Obtain x_out = x_res1 + α2 × x_mlp through the second residual connection, where α2 is another learnable residual weight.

[0051] The formula is expressed as:

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] Where α1 and α2 are the learnable residual weights, and LayerNorm is calculated as follows:

[0059]

[0060] Where μ is the mean, Let ε be the variance, γ and β be learnable parameters, and ϵ = 1e−6 be the numerical stability parameter.

[0061] The GELU activation function is defined as follows:

[0062]

[0063] Where Φ(x) is the cumulative distribution function of the standard normal distribution.

[0064] The core of the Mamba module is the state space recursive formula:

[0065]

[0066] Where h(t) represents the hidden state at a certain time, h(t-1) represents the hidden state at the previous time, x(t) is the input, y(t) is the output, and A, B, and C are the learned parameter matrices. Mamba has a time complexity of O(n⋅d), which is significantly better than the Transformer self-attention's O(n²⋅d), where n is the sequence length and d is the feature dimension.

[0067] A detailed explanation of the working mechanism of the Mamba module. Mamba is a novel sequence modeling method with a structure as follows: Figure 3 As shown, it is based on the state-space model (SSM). Unlike the self-attention mechanism of Transformer, Mamba uses a recursive state transition.

[0068] For an input sequence x = [x0, x1, ..., x_T], where T is the sequence length, Mamba performs the following steps:

[0069] Step 1: Input Projection. Perform linear projection on the input features: x_proj = W_in × x + b_in, where W_in is the projection matrix and b_in is the bias.

[0070] Step 2: Gating separation. The projected features are divided into two parts: x_proj and gate, both with a dimension of d_inner / 2.

[0071] Step 3: 1D convolution processing. Apply 1D convolution to x_proj with a kernel size of 4 for local feature extraction: x_conv=Conv1D(x_proj,kernel_size=4).

[0072] Step 4: SSM recursion. Perform recursive calculations for each position t in the sequence:

[0073] Hidden state update: h_t = A_bar⊗h_(t-1) + B_bar⊗x_conv_t

[0074] Where h_t is the hidden state vector, and A_bar and B_bar are the discretized coefficients.

[0075] Output mapping: y_t = C_bar⊗h_t

[0076] Step 5: Gating Application. Perform element-wise multiplication of the output with the gate: output = y ⊗ SiLU(gate), where SiLU is the SigmoidLinearUnit activation function.

[0077] Step 6: Output Projection. Project the output linearly back to the original dimension: x_mamba = W_out × output + b_out.

[0078] Furthermore, in step S3, the loss function consists of three parts: L1 loss L_L1, perceptual loss L_perc, and total variation loss L_TV; the L1 loss (MAE - Mean Absolute Error) is the mean absolute error between the predicted value and the true value, and is calculated as follows:

[0079]

[0080] Where N is the total number of pixels in the image. This represents the value of the i-th pixel in the super-resolution image. This represents the value of the i-th pixel in a real high-resolution image.

[0081] Compared to L2 loss (mean squared error), L1 loss is less sensitive to outliers and produces clearer results in super-resolution tasks.

[0082] Perceptual loss utilizes a pre-trained VGG19 network to extract features, making the super-resolution result as similar as possible to the features of the real image. The calculation method is as follows:

[0083]

[0084] in Let represent the feature extraction function of the j-th layer of the VGG19 network. These represent the number of channels, height, and width of the feature map for that layer, respectively.

[0085] The advantage of perceptual loss is that it compares in the feature space rather than the pixel space, which can better capture the semantic and texture information of the image, making the super-resolution results more in line with human visual perception.

[0086] Total Variation Loss (TVLoss) encourages smoothness between adjacent pixels and suppresses high-frequency noise and artifacts. The calculation method is as follows:

[0087]

[0088] in Represents the gradient in the horizontal direction. This represents the gradient in the vertical direction.

[0089] The purpose of TV loss is to minimize the total variation of an image, making the image change smoothly between adjacent pixels and reducing over-smoothing or noise.

[0090] The final total loss function is a weighted combination:

[0091]

[0092] Where λ1=1.0, λ perc =0.1, λ TV =0.01.

[0093] These weights are set according to the following principles: L L1 As the primary loss function, it has the largest weight (1.0), ensuring basic pixel-level reconstruction accuracy. L perc As an auxiliary loss function, with a weight of 0.1, it guides the model to learn higher-level features, making the output more natural. TV As a regularization term, it has a small weight (0.01) and is used for smoothing and noise suppression.

[0094] During training, each loss component is recorded separately to facilitate monitoring of training progress. If a loss component is too large or too small, its weight can be adjusted accordingly.

[0095] The beneficial effects of this invention are as follows:

[0096] Highly efficient sequence modeling capabilities: Compared to the O(n²) complexity of Transformer, the time complexity of the Mamba module is O(n), which can handle long sequences and high-resolution images more efficiently.

[0097] Memory friendly: Since it does not require the computation and storage of the complete attention matrix, the Mamba module has a significantly lower memory footprint than the Transformer, making it easier to deploy on devices with limited hardware resources.

[0098] Multi-scale feature fusion: The three-layer hierarchical structure can capture multi-scale features from details to the global picture. The encoder-decoder framework combined with skip connections effectively fuses information from different levels, significantly improving super-resolution quality. It can convert existing 10m resolution river remote sensing images into 2.5m resolution (4x super-resolution), effectively improving the ability to identify river details.

[0099] Flexible loss function design: The combination of L1, perceptual and TV loss ensures both pixel-level accuracy and visual quality and smoothness, making the super-resolution results clear and natural.

[0100] Residual learning framework: By directly learning the difference between low and high scores rather than the absolute value, it accelerates convergence and improves final performance.

[0101] End-to-end trainable: The entire network is an end-to-end differentiable architecture, supporting gradient backpropagation and adaptive learning. Attached Figure Description

[0102] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0103] Figure 1 This describes the overall network structure of the super-resolution model.

[0104] Figure 2 For Mamba residual blocks;

[0105] Figure 3 This is a display of the super-resolution results. Detailed Implementation

[0106] Example 1

[0107] The Yangtze River, China's largest inland river, plays a vital role in flood control, water resource management, and environmental protection. Jingzhou City is located in the middle reaches of the Yangtze River, where the river channel is approximately 100 km long and 1000-3000 m wide. The river's morphology is complex and varied, including numerous sandbars, tributary confluences, and meanders. Currently, the widely used Sentinel-2 satellite remote sensing imagery has a resolution of 10 m, which is insufficient to clearly depict the river's detailed features, such as the precise location of the riverbed boundaries, the spatial distribution of sandbars and shoals, and the dynamic processes of erosion and deposition. These detailed features are crucial for the management of Yangtze River flood control dikes, the supervision of river sand mining, water conservancy project planning, and river ecological protection. Traditional high-resolution satellite imagery is costly, time-consuming, and lacks continuous coverage. However, deep learning-based image super-resolution technology can convert existing 10 m resolution river remote sensing images to 2.5 m resolution (4x super-resolution), effectively improving the ability to identify river details and providing high-quality remote sensing data support for river management, flood monitoring, and ecological assessment in the Jingzhou section of the Yangtze River.

[0108] A method for generating super-resolution high-definition images based on optimized generative neural networks.

[0109] The steps include the following:

[0110] S1: Construct the training dataset for the model: Obtain a high-resolution image dataset and obtain the corresponding low-resolution images through downsampling. Create low-resolution-high-resolution image pairs required for training and process the image pairs to obtain the dataset.

[0111] S11. This embodiment uses the Jingzhou section of the Yangtze River as the study area, selecting the river corridor zone between 111°40' and 112°30' east longitude and 29°50' and 30°20' north latitude. GF-7 remote sensing satellite imagery data was acquired through the China Resources Satellite Application Center. The GF-7 satellite has a high spatial resolution of 0.8m-3.2m, providing more detailed river detail capture compared to Sentinel-2's 10m resolution. This embodiment acquired 15 GF-7 panchromatic images (2.1m resolution) and corresponding multispectral images (5.8m resolution), which were then fused to obtain a 2.5m resolution false-color remote sensing image. These images were collected in three time periods: May (high water season), July (normal water season), and September (low water season) of 2024, covering river morphological characteristics under different water level conditions, making the sample highly representative of the seasons. The images include various river morphologies such as river channel meandering, sandbar distribution, and tributary inflow. All images underwent preprocessing including orthorectification, atmospheric correction, image matching, and fusion, ultimately yielding 15 high-quality river remote sensing images with a resolution of 2.5m. A high-resolution image dataset was acquired, denoted as I_HR∈R^(H×W×3), where H and W represent the image height and width, respectively, and 3 represents the RGB three channels.

[0112] S12. To create the low-resolution-high-resolution image pairs required for training, each 2.5m resolution image in the reference image set is downsampled by a factor of 4, and a bicubic interpolation method is used to simulate a 10m resolution Sentinel-2 remote sensing image, denoted as . ;

[0113] S13. Obtain a low-resolution river image with a resolution of 1024×1024. This downsampling method can simulate the information loss during the actual Sentinel-2 image acquisition process. Then, randomly crop from the paired high-resolution reference image and low-resolution image, setting the high-resolution size to 192×192 pixels and the low-resolution size to 48×48 pixels. Through multiple random cropping and position shifting, approximately 1000 training sample pairs are generated from each original image, resulting in a total of 15000 river remote sensing samples.

[0114] S14. Perform data augmentation on all samples to improve the model's generalization ability. Augmentation operations include: random horizontal flipping with a probability of 0.5 to simulate river morphology observed from different directions; random vertical flipping with a probability of 0.5; random rotation, randomly selected from {0°, 90°, 180°, 270°} to adapt to different river directions; random Gaussian noise addition with a noise standard deviation σ=0.01 to simulate remote sensing sensor noise and atmospheric interference; brightness adjustment with a probability of 0.3, multiplying pixel values ​​by a random coefficient in the range [0.7, 1.3] to adapt to changes in river brightness under different seasons and lighting conditions. The augmented image is then converted into tensor form and regularized to the range [0, 1].

[0115] S15. Divide the processed multispectral satellite remote sensing image dataset into training set, validation set and test set according to the ratio of 8:1:1 to obtain the processed multispectral satellite remote sensing image super-resolution dataset.

[0116] S2. Model Construction: This embodiment constructs an end-to-end convolutional neural network specifically for super-resolution of river remote sensing images based on the Mamba state-space model. Compared with traditional convolutional neural networks and Transformer architectures, the Mamba module adopts a sequence modeling method with linear time complexity O(n), which has a significant computational efficiency advantage compared to the Transformer's O(n²) complexity. For high-resolution, spatially complex image super-resolution tasks such as river remote sensing images, Mamba's efficient sequence modeling capability can effectively capture the spatial structural features of rivers, such as the continuity of river channel boundaries and the topological relationships of river islands. This super-resolution network adopts a hierarchical encoder-decoder structure (similar to the U-Net architecture), which preserves the detailed information of river images through multi-scale feature fusion, while utilizing Mamba residual blocks for feature extraction and processing. The entire network consists of five main modules: a shallow feature extraction module, a hierarchical Mamba processing module (layers 1-3), a multi-scale feature fusion module, an upsampling module, and a final image reconstruction module. The overall process of the network is as follows: low-resolution river imagery is first converted into a high-dimensional feature space through shallow feature extraction. Then, it undergoes progressive downsampling and feature extraction through hierarchical Mamba blocks, achieving global feature perception at the deepest layer of the network. Finally, spatial resolution is restored through progressive upsampling and feature fusion, ultimately generating super-resolution river remote sensing imagery. This design can preserve the global direction and macroscopic morphological features of the river while restoring the microscopic details of the river through multi-scale fusion, including the fine contours of the riverbank, the morphology of sandbars and shoals, etc., thereby obtaining more accurate river information for river flood control monitoring and river management.

[0117] The overall data flow of the network is as follows:

[0118] S21: Shallow Feature Extraction

[0119] The low-resolution river image (48×48×3) first enters the shallow feature extraction module. This module processes the image using a single-layer 3×3 convolutional kernel and the ReLU activation function, converting the original RGB three-channel image into a 64-dimensional feature space. After convolution processing, the feature dimension changes from 48×48×3 to 48×48×64.

[0120] The shallow feature layer (48×48×64) enters the first layer of the Mamba processing module. This layer operates at the original 48×48 resolution and is responsible for extracting detailed features of the river image at full spatial resolution. The first layer contains six cascaded Mamba residual blocks (RMABs). Before entering these blocks, the feature map is divided into 8×8 patches, resulting in 36 patches (48÷8=6, so 6×6=36). Each patch has a length of 4096 tokens (8×8×64). These patch sequences are processed sequentially through the six RMAB blocks. After processing, the output feature dimension remains 48×48×64.

[0121] The specific data flow for steps S22~S24 is shown in Table 1:

[0122] Table 1

[0123] step Handling method Input Dimensions Output Dimension illustrate Network input - 48×48×3 48×48×3 Low-resolution river remote sensing image patch Shallow feature extraction 3×3 convolution + ReLU 48×48×3 48×48×64 RGB three channels converted to 64-dimensional feature space Layer 1 RMAB block 16×RMAB processing 48×48×64 48×48×64 Detail feature extraction at full resolution Layer 1 output - 48×48×64 48×48×64 Detailed river features (riverbank outline, river width variation) Downsampling of layers 1-2 stride 2 convolution 48×48×64 24×24×64 The resolution is reduced to half to expand the receptive field. 2nd layer RMAB block 6×RMAB processing 24×24×64 24×24×64 River feature extraction at a medium scale Layer 2 output - 24×24×64 24×24×64 Medium-scale characteristics (channel orientation, sandbar organization) Downsampling of layers 2-3 stride 2 convolution 24×24×64 12×12×64 Resolution reduced to 1 / 4 3rd layer RMAB block 6×RMAB processing 12×12×64 12×12×64 Global semantic feature extraction Layer 3 output - 12×12×64 12×12×64 Global-scale characteristics (river segment orientation, geographical relationships) First 2x upsampling PixelShuffle 12×12×64 24×24×64 Feature resolution restored to 1 / 2 First feature fusion splicing + 3×RMAB 24×24×128 24×24×64 Fusion with features from layer 2 (deep global + shallow detail) Second 2x upsampling PixelShuffle 24×24×64 48×48×64 Feature resolution restored to full resolution Second feature fusion splicing + 3×RMAB 48×48×128 48×48×64 Fusion with Layer 1 features First 4x upsampling PixelShuffle 48×48×64 96×96×64 Feature resolution increased by 2 times Second 4x upsampling PixelShuffle 96×96×64 192×192×64 Feature resolution increased to target resolution Low resolution baseline Bicubic interpolation 48×48×3 192×192×3 Provide a baseline for residual learning Image Reconstruction Layer 1 3×3 convolution 192×192×64 192×192×32 Preliminary feature processing Image Reconstruction Layer 2 3×3 convolution 192×192×32 192×192×3 Convert to RGB three channels Residual learning fusion addition operation 192×192×3+192×192×3 192×192×3 The reconstruction result is added to the interpolation baseline. Network output - - 192×192×3 Final super-resolution river remote sensing image

[0124] In this embodiment:

[0125] The output of layer 1 is downsampled using a convolution with a stride of 2 and then fed into the second layer's Mamba processing module. After downsampling, the feature map resolution is 24×24, while the number of channels remains 64. Layer 2 also contains 6 cascaded Mamba residual blocks.

[0126] The output features from layer 2 enter a 4x upsampling module. This module achieves this through two consecutive 2x upsampling operations. First, a pixel rearrangement technique is used for the first 2x upsampling, increasing the feature resolution from 24×24 to 48×48. Then, a second 2x upsampling is performed, increasing the feature resolution from 48×48 to 192×192. The principle of pixel rearrangement is to expand the number of output channels to four times the original number through convolution operations, and then rearrange the element positions of these channels to double the spatial resolution. After two cascaded upsampling operations, the feature map transforms from 24×24×64 to 192×192×64.

[0127] The features upsampled by 4x are fed into the image reconstruction module. This module first performs bicubic interpolation upsampling on the original low-resolution input to the target resolution of 192×192, which serves as the baseline. Then, the output features of the network are processed through the reconstruction layer. The reconstruction layer contains two 3×3 convolutional operations: the first layer converts the 64-channel features into a 3-channel RGB image, and the second layer performs further feature fusion and optimization. Finally, through a residual learning mechanism, the reconstruction result is added to the interpolated baseline to obtain the final super-resolution river image (192×192×3).

[0128] The Mamba residual block is the core computational unit of this network. Each RMAB block contains a combination of Mamba modules, layer normalization, multilayer perceptron (MLP), and residual connections. In this embodiment, the processing flow of each RMAB includes the following steps:

[0129] Step 1: Layer normalization of the input sequence

[0130] When a token sequence of length 4096 enters the RMAB block, it first undergoes layer normalization. Layer normalization is a feature standardization technique that calculates the mean and standard deviation of all features in the current sample and then standardizes all features to give them a distribution with zero mean and unit variance.

[0131] Step 2: Sequence modeling using the Mamba module

[0132] The normalized sequence is input into the Mamba module. The Mamba module is the core innovation of this network, performing efficient sequence modeling based on a state-space model. The state-space model processes the sequence recursively: at time t, the hidden state is generated by a linear combination of the hidden state of the previous time step and the input token of the current time step, and the output is obtained by a linear transformation of the current hidden state. This recursive approach gives the Mamba module linear time complexity, a significant advantage over the quadratic complexity of the Transformer self-attention mechanism. For long sequences of high-resolution images such as river remote sensing images, the Mamba module can efficiently capture long-range spatial dependencies.

[0133] Step 3: First Residual Connection

[0134] The output of the Mamba module is added to the original input sequence through a learnable residual weight factor. The residual weight factor is initialized to 0.1, allowing the network to adaptively adjust the contribution of the Mamba module to features during training. This residual connection design enables gradients to propagate back more directly, which is beneficial for training deep networks.

[0135] Step 4: Second layer normalization

[0136] The output of the first residual connection is then subjected to layer normalization. This re-normalization further stabilizes the input of subsequent MLP modules.

[0137] Step 5: Feature Transformation of Multilayer Perceptron (MLP)

[0138] The normalized features are input into a Multilayer Perceptron (MLP). The MLP consists of two fully connected layers and a GELU activation function. The first fully connected layer expands the feature dimension from 4096 to 16384 (a 4-fold expansion), increasing the module's non-linear expressive power. Then, a non-linear transformation is performed using the GELU activation function. Compared to the commonly used ReLU activation function, the GELU activation function better preserves gradient information, which is beneficial for network training. After the activation function, the second fully connected layer reduces the dimension from 16384 back to 4096, restoring the original dimension.

[0139] Step 6: Second residual connection

[0140] The output of the MLP is added to the output of the first residual connection by another learnable residual weight factor (initial value 0.1). This design further enhances the network's expressive power and training stability. After this step, a complete RMAB block is processed, and the length of the output sequence remains 4096.

[0141] The concatenated processing of six RMAB blocks means that the output of each block serves as the input to the next, progressively optimizing and extracting features. Each block learns feature representations at different levels of abstraction; earlier blocks learn more specific local features, while later blocks learn more abstract global features. After processing the six RMAB blocks in the first layer, the 4096-bit sequence is reshaped back into a two-dimensional spatial form, resulting in a 48×48×64 feature map. This feature map contains rich information about the river at detailed scales, such as the contours of the riverbanks, variations in river width, and the boundary between sandbars and the main channel—microscopic features.

[0142] Image Reconstruction Module

[0143] The image reconstruction module is the last key module of the network, responsible for converting the feature representations learned by the network into the final super-resolution river image.

[0144] Step 1: Generation of low-resolution baselines

[0145] First, the original low-resolution input river image (48×48×3) is upsampled to the target resolution of 192×192 using bicubic interpolation. This upsampled result serves as the baseline (192×192×3), providing initial estimates for subsequent residual learning.

[0146] Step 2: Processing of 4x upsampled features

[0147] The feature map (192×192×64) output by the 4x upsampling module is fed into the reconstruction layer. The first convolutional layer of the reconstruction layer uses a 3×3 convolutional kernel to convolve the 64-channel feature map, and then performs a non-linear transformation through an activation function.

[0148] Step 3: Feature to RGB Image Conversion

[0149] The two convolutional layers in the reconstruction layer ultimately convert the 64-channel features into a 3-channel RGB image, achieving a mapping from the feature space back to the image space.

[0150] Step 4: Application of Residual Learning

[0151] The residual learning mechanism adds the RGB image generated by the reconstruction layer to the previously generated interpolation baseline.

[0152] Step 5: Output

[0153] After residual learning, the final super-resolution river remote sensing image (192×192×3) was obtained. This super-resolution image has the following characteristics: the river boundary is clear and sharp, and the contour of the riverbank is accurately depicted by making full use of the detailed information learned by the network; the morphology of the sandbars is identifiable, the boundary lines between different sandbars are clear, and the spatial distribution relationship is clear; the river direction is coherent and natural, and the overall direction and meandering pattern of the river are well preserved by the network modeling long-distance dependencies through the Mamba module; the image is free of artifacts and noise, and the super-resolution result has good visual quality and naturalness through optimization of multiple loss functions.

[0154] S3: Model Parameter Setting and Training Process

[0155] S31. Training Hyperparameter Configuration

[0156] This embodiment optimizes the model based on the training set. The batch size is set to 32. The initial learning rate is set to 2×10⁻⁻⁻⁶. 4 The Adam optimizer was selected for parameter optimization. The exponential decay rate of the first moment estimate of the Adam optimizer was set to 0.9, and the exponential decay rate of the second moment estimate was set to 0.999.

[0157] The training epochs were set to 1000, and a cosine annealing learning rate scheduling strategy was adopted to keep the learning rate at a relatively high value in the early stages of training for rapid initial parameter adjustments. As training progressed, the learning rate gradually decreased according to the law of the cosine function. In the middle of training, the learning rate dropped to half of its initial value. In the later stages of training, the learning rate continued to decrease and approached the minimum value of 1×10⁻⁻⁶. 7 .

[0158] Mixed-precision training is employed to accelerate the training process and conserve GPU memory. During forward propagation, features and intermediate activation values ​​are calculated using float16. For gradient calculation, the float16 gradients are scaled to an appropriate range to avoid underflow. In the parameter update phase, float32 is used for optimizer calculation and parameter updates, ensuring numerical accuracy in parameter updates. Mixed-precision training typically reduces GPU memory usage by approximately 50% and increases training speed by 2x.

[0159] S32, Loss Function Design

[0160] To achieve high-quality super-resolution of river remote sensing images, this embodiment employs a weighted combination of multiple loss functions, guiding the network learning through multi-objective optimization.

[0161] The first term is the L1 reconstruction loss, with a weight set to 1.0, which is the main loss term. The L1 loss calculates the mean absolute error between the predicted super-resolution river image and the true high-resolution river image. Compared to the squared error, the L1 loss penalizes outliers less, thus producing clearer results in super-resolution tasks, especially for terrain features with distinct boundaries such as rivers, where the L1 loss better preserves the sharpness of the boundaries.

[0162]

[0163] Where N is the total number of pixels in the image. This represents the value of the i-th pixel in the super-resolution image. This represents the value of the i-th pixel in a real high-resolution image.

[0164] The second term is the perceptual loss, with a weight set to 0.1. The perceptual loss uses a pre-trained VGG19 neural network to extract features. Specifically, the predicted super-resolution image and the true high-resolution image are respectively processed through multiple convolutional layers of the VGG19 network for feature extraction, and then the error between the two is calculated in the feature space.

[0165]

[0166] in Let represent the feature extraction function of the j-th layer of the VGG19 network. These represent the number of channels, height, and width of the feature map for that layer, respectively.

[0167] The third term is the total variation loss, with a weight set to 0.01. The total variation loss measures the degree of variation between adjacent pixels in an image.

[0168]

[0169] in Represents the gradient in the horizontal direction. This represents the gradient in the vertical direction.

[0170] The weighted combination of these three loss functions constitutes the final total loss function. The L1 reconstruction loss has the largest weight (1.0), ensuring basic pixel-level reconstruction accuracy. The perceptual loss has a weight of 0.1, used to guide the model to learn high-level features. The total variation loss has the smallest weight (0.01), used as a regularization term for smoothing and noise suppression.

[0171]

[0172] S33. Validation and Model Selection Strategy

[0173] During training, the model's performance was evaluated on a validation set every 10 epochs. The validation set contained 6000 river remote sensing image patch samples. During the validation phase, the model ran in inference mode, performing super-resolution processing on all samples in the validation set. The super-resolution results were compared with the original high-resolution reference image, and two evaluation metrics, PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity), were calculated.

[0174] PSNR measures the pixel-level difference between a predicted image and the real image. A higher PSNR value indicates that the prediction is closer to the actual result. Generally, a PSNR greater than 30dB indicates a good result. SSIM measures the similarity between two images in multiple aspects such as brightness, contrast, and structure, ranging from 0 to 1. The closer the SSIM value is to 1, the more similar the two images are. Compared to PSNR, SSIM is more in line with human visual perception.

[0175] During validation, when the average PSNR on the validation set exceeds the previous best performance, the current model's weight parameters are saved. Simultaneously, if no improvement in the validation set metrics is observed for 50 consecutive epochs, an early stopping mechanism is triggered, immediately terminating the training process. The purpose of the early stopping mechanism is to prevent the model from overfitting in subsequent training.

[0176] S34. Training Hardware Environment and Expected Training Time

[0177] The model training in this embodiment was performed on a high-performance computing platform. The platform was configured with four NVIDIA A100 graphics processors, each with 80GB of video memory, for a total of 320GB. The CPU was an Intel Xeon Platinum 8380 processor running at 2.30GHz, and the system memory was 256GB. Storage consisted of 4TB NVIDIA NVMe solid-state drives. The operating system was Ubuntu 20.04 LTS, and the deep learning framework was PyTorch 2.0.0 with Python 3.9. CUDA version 12.1 was used to ensure compatibility between the GPU and the framework.

[0178] With this hardware configuration, a distributed data-parallel training approach is used, distributing the training task across four GPUs for parallel processing. Each GPU independently processes a different data batch and periodically performs gradient synchronization.

[0179] S4: Model Evaluation Results

[0180] Quantitative results on the S41 test set

[0181] On a test set of remote sensing images of the Yangtze River in Jingzhou (containing 6000 patch samples), the super-resolution model in this embodiment achieved the following results: PSNR reached 33.2dB, an improvement of 5.0dB compared to the bicubic interpolation baseline (28.2dB), an improvement of 2.2dB compared to SRResNet (31.0dB), and an improvement of 1.1dB compared to RCAN (32.1dB). SSIM reached 0.92, an improvement of 0.12 compared to bicubic interpolation (0.80), an improvement of 0.05 compared to SRResNet (0.87), and an improvement of 0.03 compared to RCAN (0.89). IoU reached 0.88, indicating high accuracy in river boundary extraction.

[0182] S42. Qualitative Assessment Results

[0183] In terms of visual effect evaluation, the river image obtained by super-resolution in this embodiment exhibits the following characteristics: First, the river boundary is clear and sharp. By fully utilizing the detailed information learned by the network, the contour of the riverbank is accurately depicted, showing a significant improvement compared to the interpolated baseline. Second, the morphology of the sandbars is clearly identifiable, with distinct boundary lines between different sandbars and a clear spatial distribution relationship, allowing the geomorphic features within the river to be displayed. Third, the river's direction is continuous and natural. The network's modeling of long-distance dependencies through the Mamba module ensures that the overall direction and meandering pattern of the river are well preserved. Fourth, the image is free of artifacts and noise. Through optimization of multiple loss functions, the super-resolution result has good visual quality and a natural feel, avoiding common super-resolution artifacts such as ringing effects and oversharpening.

[0184] Because the training data included remote sensing images of rivers during three different seasons—high water (May), normal water (July), and low water (September)—the model demonstrated excellent adaptability under various water level conditions. During the high water season, when river levels were highest and river width was greatest, the super-resolution model was able to clearly identify river boundaries, achieving a PSNR of 32.8 dB. During the normal water season, when river levels were at normal, the PSNR reached 33.4 dB, the best among the three periods. During the low water season, when water levels were lowest, sandbars and shoals were most exposed, and the model could still accurately identify these small features, achieving a PSNR of 33.1 dB. This indicates that the model has good seasonal adaptability and robustness.

[0185] This embodiment evaluates the model's performance under different river morphology conditions. In the straight sections of the river, where the channel direction is relatively simple, the model achieves the best super-resolution performance, with a PSNR of 34.1 dB. In the meandering sections, due to the complex channel morphology and diverse boundary variations, the model's super-resolution performance is relatively weaker but still good, with a PSNR of 32.8 dB. Near the confluence of tributaries, the complex flow and sedimentation patterns resulting from the confluence of two rivers lead to a decrease in the accuracy of the super-resolution results, with a PSNR of 32.1 dB. In areas with dense sandbars, the boundaries between numerous small sandbars are finely defined, and the model can identify them well, achieving a PSNR of 33.0 dB. Overall, the model achieves satisfactory super-resolution performance under various river morphology conditions. Table 2 shows the quantitative comparison results of this embodiment's super-resolution model with other mainstream super-resolution methods:

[0186] Table 2

[0187]

[0188] The super-resolution model in this embodiment outperforms other methods in both PSNR and SSIM metrics. Compared to the improved version of U²-Net, which also uses an encoder-decoder structure, this method improves PSNR by 1.4 dB and SSIM by 0.03. It also boasts 3.5 times faster inference speed and reduces GPU memory usage by more than half, fully demonstrating the advantages of the Mamba module compared to other architectures.

[0189] Evaluation of practical application effects

[0190] In practical applications, this embodiment applies the super-resolution model to river management and flood control monitoring in the Jingzhou section of the Yangtze River. By super-resolutioning the 10m resolution Sentinel-2 remote sensing imagery to 2.5m resolution by a factor of 4, key information such as individual sandbars, detailed morphology of river sandbars, and the location of river embankments can be clearly identified. Compared with traditional low-resolution imagery, the super-resolution results improve the accuracy of identifying key river features by over 90%. This high-quality river imagery provides crucial data support for the Jingzhou Municipal Water Resources Bureau's work on river sand mining supervision, flood control embankment maintenance, and river ecological assessment, significantly improving the accuracy and efficiency of river management.

[0191] Finally, it should be noted that the above description is only used to illustrate the technical solutions of the present invention and is not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention.

Claims

1. A method for generating super-resolution high-definition images based on optimized generative neural networks, characterized in that, Specifically, the following steps are included: S1: Constructing the model training dataset: Obtain the high-resolution image dataset of remote sensing satellites, and obtain the corresponding low-resolution images of remote sensing satellites through downsampling. Create the low-resolution-high-resolution image pairs required for training, and process the image pair samples to obtain the model training dataset. S2: Constructing the Super-Resolution Network Model: Based on a hierarchical encoder-decoder structure, an end-to-end super-resolution network model is constructed using Mamba residual blocks. The architecture of the super-resolution network model includes: a shallow feature extraction module, Mamba processing modules at layers 1-3, a multi-scale feature fusion module, an upsampling module, and an image reconstruction module. The shallow feature extraction module converts the input into a high-dimensional feature space through a single-layer convolution, then performs multi-scale feature processing through the hierarchical Mamba processing module, and finally obtains the final super-resolution image through the upsampling module, the multi-scale feature fusion module, and the image reconstruction module. Constructing a super-resolution network model includes the following steps: S21: Shallow feature extraction: Feature extraction of low-resolution images through a single-layer 3×3 convolution; S22: Hierarchical Mamba Feature Processing The network adopts a three-layer hierarchical structure, forming an encoder-decoder framework: Encoding path, downsampling process: Layer 1: At full resolution, feature processing is performed by applying six consecutive Mamba residual blocks (RMAB). Layer 2: The features of layer 1 are downsampled by convolution with stride of 2 to obtain a 1 / 2 resolution feature map, and then processed by 6 RMAB blocks; Layer 3: The features of layer 2 are further downsampled by convolution with stride of 2 to obtain a 1 / 4 resolution feature map, and then processed by 6 RMAB blocks; Decoding path, upsampling process: Upsampling starts from layer 3: 2x upsampling is performed through pixel rearrangement to restore half the resolution; Feature fusion: The upsampled features are concatenated with the second-layer skip connection features stored in the encoding path, and then fused through 3 RMAB blocks; Continue upsampling: Upsample the fused features by a factor of 2 to restore them to full resolution; Second fusion: The upsampled features are concatenated with the first-layer skip connection features stored in the encoding path, and then fused using three RMAB blocks; S23: Feature Fusion and Upsampling: The features after multi-layer fusion are summarized through dense connections and further processed by convolution. Pixel rearrangement is used to achieve 4x upsampling. S24: Image Reconstruction and Residual Learning: Image reconstruction and residual learning are fused through two convolutional layers to generate the final super-resolution image; The processing flow for a single RMAB is as follows: Step 1: Input x is normalized to obtain x_norm1; Step 2: x_norm1 is input into the Mamba module to obtain x_mamba; the Mamba module is for linear-time sequence modeling based on the state-space model SSM. Step 3: Obtain x_res1 = x + α1 × x_mamba through residual connections, where α1 is a learnable residual weight. This setting allows the network to dynamically adjust the contribution of the Mamba module. Step 4: x_res1 is normalized through the second layer to obtain x_norm2; Step 5: x_norm2 is input into a multilayer perceptron (MLP). The MLP contains two linear layers and a GELU activation function. The first layer expands the dimension by a factor of 4. After passing through the GELU activation function, the second layer reduces the dimension back to the original dimension, resulting in x_mlp. Step 6: Obtain x_out = x_res1 + α2 × x_mlp through the second residual connection, where α2 is another learnable residual weight; S3: Model Training: Input the training dataset obtained in S1 into the super-resolution network model built in S2. During the training process, calculate the loss through multiple loss functions, continuously iterate and update the model parameters, and finally obtain the trained super-resolution network model. S4: Use the super-resolution network model trained in S3 to input the low-resolution image of the remote sensing satellite to be tested for image super-resolution.

2. The method for generating super-resolution high-definition images based on optimized generative neural networks according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11: Obtain a high-resolution image dataset, denoted as... , where H and W represent the height and width of the image respectively, and 3 represents the RGB three channels; S12: Downsample the high-resolution image to obtain the corresponding low-resolution image; the downsampling uses bicubic interpolation, with a downsampling factor of 4. ; S13: Randomly crop a patch from the low-resolution and high-resolution image pairs; set the length and width of the high-resolution patch to be 4 times the length and width of the low-resolution patch, and crop the image at a random position in each Epoch to generate multiple training samples. S14: Perform data augmentation on the cropped image; the data augmentation includes random horizontal flipping, random vertical flipping, random rotation, and adding Gaussian noise. S15: Divide the enhanced low-resolution to high-resolution image samples into training, validation, and test sets, and use them as input to the super-resolution network model.

3. The method for generating super-resolution high-definition images based on optimized generative neural networks according to claim 1, characterized in that, The core of the Mamba module is the state space recursive formula: ； Where: h(t) represents the hidden state at a certain time, h(t-1) represents the hidden state at the previous time, x(t) is the input, y(t) is the output, and A, B, and C are the learned parameter matrices.

4. The method for generating super-resolution high-definition images based on optimized generative neural networks according to claim 1, characterized in that, In step S3, the loss function is L1 loss L L1 Perceived loss L perc Total variation loss L TV It consists of three parts; the final total loss function is a weighted combination: ； Where, λ1=1.0, λ perc =0.1,λ TV =0.01.

Images