Edge enhancement and frequency domain optimization based super-resolution method, system and medium

Abstract

The application discloses an edge enhancement and frequency domain optimization-based super-resolution method and system and a medium, relates to the technical field of remote sensing image processing and artificial intelligence, and generates a continuous space image based on a super-resolution model by fusing an edge confidence map and a multi-scale feature map, and adapts to the characteristics of a panchromatic band; meanwhile, the super-resolution model introduces a frequency domain optimization strategy, improves the high-frequency texture and edge reconstruction quality, and combines a dynamic rasterization rendering technology of the frequency domain optimization to solve the problem of a remote sensing edge enhancement module, optimize the edge details of the panchromatic band, and specifically optimize the edge details, and the performance is better, especially in building and vegetation texture.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and artificial intelligence technology, specifically to super-resolution methods, systems, and media based on edge enhancement and frequency domain optimization. Background Technology

[0002] Remote sensing image super-resolution reconstruction technology aims to restore or reconstruct high spatial resolution images from low spatial resolution remote sensing images through algorithmic processing, thereby enhancing the image's detail and practical value. Currently, this method has significant application needs in fields such as land resource surveys, environmental monitoring, urban planning, and disaster assessment. With the rapid development of deep learning technology, super-resolution methods based on convolutional neural networks (CNNs) have become mainstream. These methods typically employ end-to-end network architectures to learn the complex mapping relationship from low-resolution to high-resolution images, demonstrating strong capabilities in feature extraction and image reconstruction.

[0003] However, existing deep learning-based super-resolution methods, especially when processing high spatial resolution remote sensing images such as panchromatic images, still have several significant shortcomings that limit further improvements in reconstruction quality:

[0004] First, in terms of spatial relationship modeling, most existing methods rely on fixed-size convolutional kernels for local feature extraction. While this inductive bias is beneficial for local feature extraction, it struggles to effectively capture the complex and irregular spatial structures widely present in remote sensing images (such as winding road networks, irregular water body boundaries, and fragmented vegetation patches). The limited receptive field of fixed convolutional kernels results in insufficient modeling capabilities for long-distance dependencies and global spatial context, leading to blurred feature boundaries and distorted structures in reconstructed images, making it difficult to maintain the geometric structures and spatial topological relationships of the real scene.

[0005] Secondly, regarding the utilization of modal information, existing methods have failed to fully explore and utilize the high spatial resolution characteristics of panchromatic images. While panchromatic bands typically contain rich textural details and high-frequency information, in most super-resolution frameworks, this information is only introduced through simple feature stitching or shallow fusion, failing to achieve deep, multi-scale feature interaction and complementarity. Due to the lack of an effective cross-modal information integration mechanism, the reconstruction process is prone to getting trapped in local optima, resulting in limited high-frequency detail recovery capabilities, a lack of realism in the restored textures, and a failure to maintain sufficient consistency with the global semantic context of the image (such as land cover categories and distribution patterns) when recovering local details, thus affecting the overall naturalness and visual quality of the image.

[0006] Finally, in balancing local detail and global consistency, existing model architectures often struggle to achieve both accurate reconstruction of high-frequency local information and overall coordination of the global semantic structure. On the one hand, excessive focus on local details can lead to noise amplification, artifacts, or overly smooth textures in the reconstruction results; on the other hand, overemphasizing global consistency may result in the loss of important edge and texture features. This inadequacy in balancing local and global information makes it difficult to achieve an ideal balance between detail sharpness and overall structural coherence in reconstructed images, limiting the application of super-resolution technology in generating high-quality remote sensing images.

[0007] Therefore, there is an urgent need for a super-resolution reconstruction method that can more effectively model the complex spatial structure of remote sensing images, deeply integrate multimodal high-resolution information, and synergistically optimize local details and global consistency, so as to improve the spatial accuracy, visual quality and application value of reconstructed images. Summary of the Invention

[0008] To address the shortcomings of current remote sensing image super-resolution methods, which primarily rely on deep learning models for feature extraction and reconstruction, resulting in insufficient spatial relationship modeling, inadequate utilization of modal information, and insufficient balance between local and global consistency, thus affecting the overall quality of reconstructed images, this invention aims to provide a super-resolution method, system, and medium based on edge enhancement and frequency domain optimization. It generates continuous spatial images by fusing edge confidence maps and multi-scale feature maps based on a super-resolution model, adapting to panchromatic band characteristics. Simultaneously, the super-resolution model incorporates a frequency domain optimization strategy to improve the quality of high-frequency textures and edge reconstruction. Combined with frequency domain optimized dynamic rasterization rendering technology, it solves the remote sensing edge enhancement module problem, optimizes panchromatic band edge details, and specifically optimizes edge details, particularly showing better performance in building and vegetation textures.

[0009] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0010] This solution provides a super-resolution method based on edge enhancement and frequency domain optimization, including:

[0011] Acquire raw panchromatic band images and preprocess the raw panchromatic band images to obtain a first-resolution panchromatic band image;

[0012] A first-resolution panchromatic band image is input into a trained super-resolution model to generate a second-resolution panchromatic band image. Local texture and global structural information are captured from the first-resolution panchromatic band image to generate a multi-scale feature map. Edge features of the first-resolution panchromatic band image are extracted, and an edge confidence map is generated based on these edge features. The edge confidence map and the multi-scale feature map are fused to generate a continuous spatial image. The continuous spatial image is then subjected to block rasterization rendering and fusion to obtain the second-resolution panchromatic band image. The super-resolution model is trained using a multi-scale progressive strategy. The second resolution is higher than the first resolution.

[0013] A further optimized solution is that the preprocessing method includes:

[0014] First, crop the original panchromatic image to the target size;

[0015] The pixel values ​​of the cropped original panchromatic band image are normalized and adjusted to the range of [0, 1].

[0016] The normalized data is downsampled to obtain the first-resolution panchromatic band image.

[0017] A further optimized solution is that the method for generating the edge confidence map includes:

[0018] A multi-scale image set is obtained by smoothing the first-resolution panchromatic band image using bilateral filtering technology.

[0019] The gradient direction of each scale image in the multi-scale image set is determined, and adaptive high and low threshold calculations are performed to obtain a preliminary binary edge map;

[0020] Candidate edge points are obtained from the preliminary binary edge maps of images at various scales. The window standard deviation of the gradient direction is calculated within a local window centered on each candidate edge point. A preset window standard deviation threshold range is used to retain candidate edge points whose window standard deviations satisfy the window standard deviation threshold range, thus obtaining a refined binary edge map.

[0021] After fusing the refined binary edge maps of images at various scales, Euclidean distance transformation is performed to obtain the distance field from each pixel to the nearest edge point, and an edge confidence map is generated based on the distance field.

[0022] A further optimized approach is to include methods for acquiring multi-scale image sets, such as:

[0023] The first-resolution panchromatic band image is smoothed using K bilateral filters at different scales to obtain a multi-scale image set; among which, the scale image obtained by the k-th scale bilateral filter is... k=1,2,…,K;

[0024] ;

[0025] ;

[0026] in, This represents the standard deviation of the spatial domain at the k-th scale; This represents the standard deviation of the k-th value range, taking... , This represents the maximum pixel value of the panchromatic band image at the first resolution after normalization. Represents the minimum pixel value of the first-resolution panchromatic band image after normalization; represents the normalization weight. This represents the first-resolution panchromatic band image obtained after preprocessing. Indicates the center pixel; Represents neighboring pixels; Represents a local window; exp() represents the exponential function; This represents the pixel value of the center pixel p in the preprocessed first-resolution panchromatic band image; Represents the pixel value of the neighboring pixel q in the preprocessed first-resolution panchromatic band image; ||#|| represents the Euclidean distance.

[0027] A further optimized approach is to obtain the initial binary edge map using the following methods:

[0028] The horizontal gradient is calculated using the Sobel operator for the k-th scale image. and vertical gradient The gradient magnitude of the k-th scale image is determined based on the horizontal and vertical gradients. and gradient direction :

[0029] ;

[0030] ;

[0031] Where k(x, y) represents the pixel coordinates in the scaled image obtained by bilateral filtering at the k-th scale, x is the column coordinate and y is the row coordinate; arctan() represents the arctangent function;

[0032] Gradient magnitude histograms of images at various scales are obtained. High and low thresholds are determined based on Otsu's method. Non-maximum suppression and double thresholding are then applied to images at various scales to obtain preliminary binary edge maps.

[0033] A further optimized solution involves obtaining the distance field using the following methods:

[0034] The refined binary edge maps of images at various scales are processed pixel-by-pixel and subjected to morphological dilation and connection breaking to obtain a fused binary edge map. ;

[0035] For fused binary edge maps The range field is obtained by performing Euclidean distance transformation. :

[0036] ;

[0037] Where (x, y) represents pixel coordinates, x is the column coordinate and y is the row coordinate; Represents the fused binary edge map The set of coordinates of all edge points in the middle; This represents the L2 norm.

[0038] A further optimized approach is to acquire the second-resolution panchromatic band image using the following methods:

[0039] After fusing the edge confidence map with the multi-scale feature map, a set of two-dimensional Gaussian kernel parameters {( , , , ) | i=1,2,…, }, forming a continuous spatial image; among which, This represents the total number of Gaussian kernels obtained from the regression; Indicates the center coordinates of the i-th Gaussian kernel; The 2×2 covariance matrix of the i-th Gaussian kernel controls the shape and orientation of the ellipse; This represents the grayscale value of the i-th Gaussian kernel; The opacity of the i-th Gaussian kernel is represented by the edge confidence map modulation. Denotes the set of all real numbers;

[0040] The continuous spatial image is divided into N overlapping image blocks;

[0041] Differentiable rasterization rendering is performed on the m-th image patch to obtain the predicted high-resolution region of the m-th image patch. The rendering formula is:

[0042] ;

[0043] in, This represents the pixel coordinates within the m-th image block, where m represents the image block number, m=1,2,…,N; Represents the coordinates of the Gaussian kernel center. The set of all Gaussian kernels falling within the extended neighborhood of the m-th image patch; T denotes the matrix transpose symbol; Denotes the inverse of the covariance matrix; Represents an exponential function;

[0044] Predicted high-resolution regions for all image patches Weighted fusion is performed to obtain a complete second-resolution panchromatic image. .

[0045] A further optimization scheme is as follows: the multi-scale progressive strategy includes training based on a pyramid-style multi-scale progressive training strategy: configuring N progressively increasing scales, where at the nth scale, n=1,2,…,N, and the total loss function… for:

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] Where N represents the total number of scales; represents the pixel-level L1 loss at the nth scale; This represents the second-resolution panchromatic band image output by the network at the nth scale; The corresponding true high-resolution ground image at the nth scale; , These represent the height and width of the image at the nth scale, respectively. This represents the structural similarity loss at the nth scale; This represents the edge fidelity loss at the nth scale; This represents the high-frequency loss in the frequency domain at the nth scale; This represents the structural similarity loss weight at the nth scale; This represents the edge fidelity loss weight at the nth scale; This represents the frequency domain high-frequency loss weight at the nth scale; Represents the L1 norm; This is the edge confidence map at the nth scale, with values ​​ranging from [0,1], approaching 1 at edges and approaching 0 at non-edges; (x,y) represents the pixel coordinates, where x is the column coordinate and y is the row coordinate; Represents the pixel coordinates of the corresponding real high-resolution ground image at the nth scale; (u,v) represents the pixel coordinates of the second-resolution panchromatic band image output by the network at the nth scale; (u,v) represents the spectral coordinates. Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Represents the two-dimensional fast Fourier transform function; Indicates the high-frequency region.

[0051] This solution also provides a super-resolution system based on edge enhancement and frequency domain optimization to implement the aforementioned super-resolution method based on edge enhancement and frequency domain optimization. The system includes:

[0052] The input processing module is used to acquire the original panchromatic band image and preprocess the original panchromatic band image to obtain a first resolution panchromatic band image.

[0053] The super-resolution module is used to input a first-resolution panchromatic band image into a trained super-resolution model to generate a second-resolution panchromatic band image; the super-resolution model is trained using a multi-scale progressive strategy; the second resolution is higher than the first resolution.

[0054] The super-resolution model includes:

[0055] The feature encoding module is used to capture local texture information and global structural information from the first-resolution panchromatic band image to generate multi-scale feature maps;

[0056] An edge enhancement module is used to extract edge features from a first-resolution panchromatic band image and generate an edge confidence map based on the edge features.

[0057] A Gaussian scattering modeling module is used to fuse the edge confidence map and the multi-scale feature map to generate a continuous spatial image.

[0058] The block rasterization rendering module is used to perform block rasterization rendering and fusion on the continuous spatial image to obtain a second resolution panchromatic band image.

[0059] This solution also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, can implement the super-resolution method based on edge enhancement and frequency domain optimization as described above.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] This invention provides a super-resolution method, system, and medium based on edge enhancement and frequency domain optimization. Focusing on panchromatic band characteristics, it generates continuous spatial images by fusing edge confidence maps and multi-scale feature maps based on a super-resolution model, adapting to panchromatic band characteristics. At the same time, the super-resolution model introduces a frequency domain optimization strategy to improve the quality of high-frequency textures and edge reconstruction. Combined with frequency domain optimized dynamic rasterization rendering technology, it solves the problem of remote sensing edge enhancement module, optimizes panchromatic band edge details, and specifically optimizes edge details, especially showing better performance in building and vegetation textures. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0063] Figure 1 This is a schematic diagram of the super-resolution method based on edge enhancement and frequency domain optimization.

[0064] Figure 2 A schematic diagram illustrating the principle of super-resolution based on edge enhancement and frequency domain optimization;

[0065] Figure 3 This is a schematic diagram of the super-resolution model architecture;

[0066] Figure 4 This is a schematic diagram of the super-resolution model training process. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0068] In existing technologies, remote sensing image super-resolution methods mainly rely on deep learning models for feature extraction and reconstruction. Representative techniques include: CNN networks, which rely on fixed convolutional kernels and struggle to effectively capture complex and irregular spatial structures (such as building and vegetation textures) in panchromatic remote sensing images; and Transformer networks, while capable of modeling global dependencies, suffer from high computational complexity. Existing methods rarely fully utilize the high spatial resolution information of the panchromatic band, resulting in limited high-frequency detail recovery capabilities. In recent years, Gaussian scattering has shown excellent performance in continuous image representation, particularly in image super-resolution tasks, achieving smooth reconstruction by encoding continuous Gaussian fields. However, these methods primarily target natural images and are not optimized for the edge and frequency domain characteristics of the panchromatic band; furthermore, existing patented technologies suffer from drawbacks such as insufficient utilization of modal information, inadequate reconstruction of edge and frequency domain details, low computational efficiency, and high data acquisition costs. Therefore, this solution provides the following embodiments to address the aforementioned technical problems:

[0069] Example 1

[0070] This embodiment provides a super-resolution method based on edge enhancement and frequency domain optimization, such as... Figure 1 As shown, the method includes:

[0071] Step 1: Acquire the original panchromatic band image and preprocess it to obtain the first-resolution panchromatic band image. Generally, the original panchromatic band image is a single-channel grayscale image, usually derived from satellite sensors, such as the Panchromatic band of Landsat 8 or Sentinel-2.

[0072] In step one, the preprocessing methods include:

[0073] S11, First, crop the original panchromatic band image to the target size;

[0074] S12, normalize and adjust the pixel values ​​of the cropped original panchromatic band image to the range of [0, 1];

[0075] S13, downsample the normalized data to obtain the first resolution panchromatic band image.

[0076] First-resolution panchromatic band image Represented as:

[0077] ;

[0078] in, This represents the original panchromatic band image, and s represents the downsampling factor, such as 2 or 4;

[0079] This indicates bilinear interpolation or Gaussian downsampling. This indicates that the pixel value is normalized.

[0080] Step two involves inputting the first-resolution panchromatic band image into the trained super-resolution model to generate a second-resolution panchromatic band image, where the second resolution is higher than the first resolution. Specifically, step two includes the following sub-steps:

[0081] S21, capture local texture information and global structural information from the first-resolution panchromatic band image to generate a multi-scale feature map; specifically, the super-resolution model extracts multi-scale features based on the U-Net architecture, and captures local texture information and global structural information from the first-resolution panchromatic band image to generate a multi-scale feature map.

[0082] The first-resolution panchromatic band image is smoothed using K bilateral filters at different scales to obtain a multi-scale image set; among which, the scale image obtained by the k-th scale bilateral filter is... k=1,2,…,K;

[0083] ;

[0084] ;

[0085] in, This represents the standard deviation of the spatial domain at the k-th scale (e.g., 2 for k=1, 4 for k=2, and so on). This represents the standard deviation of the k-th value range, taking... , This represents the maximum pixel value of the first-resolution panchromatic band image after normalization, typically 1.0; This represents the minimum pixel value of the normalized, first-resolution panchromatic band image, typically 0.0. Therefore, in actual calculations... It is usually taken as 0.1 (a constant); Indicates the normalized weights; This represents the first-resolution panchromatic band image obtained after preprocessing. Indicates the center pixel; Represents neighboring pixels; Represents a local window; exp() represents the exponential function; This represents the pixel value of the center pixel p in the preprocessed first-resolution panchromatic band image; Represents the pixel value of the neighboring pixel q in the preprocessed first-resolution panchromatic band image; ||#|| represents the Euclidean distance.

[0086] In this embodiment, K=4, and bilateral filtering with four different spatial-range parameters is used for smoothing to obtain a multi-scale image set. ;

[0087] S22, extract the edge features of the first resolution panchromatic band image, and generate an edge confidence map based on the edge features;

[0088] Specifically, the super-resolution model introduces a Remote Sensing Edge Enhancement Module (RSEEM) to enhance edge details. The RSEEM first extracts edge features using an improved Canny edge detection method, and then uses convolutional layers to generate an edge confidence map. The edge confidence map is used to guide subsequent Gaussian scattering modeling, especially for optimizing the edges of ground features (such as building outlines or road boundaries).

[0089] In step S22, the method for generating the edge confidence map includes:

[0090] S221, determine the gradient direction of each scale image in the multi-scale image set, and perform adaptive high and low threshold calculations to obtain a preliminary binary edge map; specifically:

[0091] The horizontal gradient is calculated using the Sobel operator for the k-th scale image. and vertical gradient The gradient magnitude of the k-th scale image is determined based on the horizontal and vertical gradients. and gradient direction :

[0092] ;

[0093] ;

[0094] Where k(x, y) represents the current pixel coordinates in the scaled image obtained by bilateral filtering at the k-th scale, x is the column coordinate, and y is the row coordinate; arctan() represents the arctangent function;

[0095] S222: Obtain the gradient magnitude histograms of images at each scale. Determine the high and low thresholds based on Otsu's method, and perform non-maximum suppression and double thresholding on the images at each scale to obtain preliminary binary edge maps. , In this embodiment, the high threshold is... The low threshold is .

[0096] S223, extract the edge features of the first resolution panchromatic band image, and generate an edge confidence map based on the edge features, specifically including:

[0097] For the k-th scale image For each candidate edge point p=(x,y) in the initial binary edge map, calculate the standard deviation of the gradient direction θ within a local window of size W×W centered at p. Preset window standard deviation threshold range [ [Only keep] [ The candidate edge points are identified, and the remaining points are deleted to obtain the refined binary edge map of the image at the k-th scale.

[0098] The formula for calculating the standard deviation of the gradient direction window is as follows:

[0099] ;

[0100] in, The mean of the gradient direction within the window; p=(x,y) represents the pixel coordinates of the current candidate edge point, where x is the horizontal column coordinate and y is the vertical row coordinate; This represents the standard deviation of the gradient direction within a local window centered at p, in degrees (°). The gradient direction at the neighborhood pixel q within the window (calculated from Sobel, ranging from [-90°, 90°] or [0°, 180°]). This represents a local window set of pixels centered at the center pixel p (e.g., a 3×3 window contains 9 pixels). The total number of effective pixels within the local window (usually W², M=9 in this embodiment); Indicates the window Sum the pixels q in all neighborhoods.

[0101] In this embodiment, the local window size W=3 (i.e., a 3×3 window, M=9 pixels); the preset threshold range [ [0°, 18°] = Window standard deviation only at the current candidate edge point At 18°, the current candidate edge points are retained; otherwise, they are set to 0 to effectively suppress parallel false edges generated by periodic textures such as farmland and vegetation; finally, a refined binary edge map of each scale image is obtained.

[0102] S224, performs pixel-by-pixel OR operations on the refined binary edge maps of images at various scales and performs morphological dilation and connection breaking to obtain a fused binary edge map. ;

[0103] S225, merging binary edge maps The range field is obtained by performing Euclidean distance transformation. :

[0104] ;

[0105] Where (x, y) represents pixel coordinates, x is the column coordinate and y is the row coordinate; This represents the coordinates of edge points in the fused binary edge map; Represents the fused binary edge map The set of coordinates of all edge points in the middle; This represents the L2 norm.

[0106] Final edge confidence plot for:

[0107] ;

[0108] in, , represents the Sigmoid function, α=8 represents the steepness coefficient, =10 pixels (empirical optimal value); in this way, the confidence of pixels closer to the real edge is closer to 1, and the confidence of pixels far from the edge is smoothly reduced to 0.

[0109] S23, the continuous spatial image is segmented into rasterization rendering and fusion to obtain a second resolution panchromatic band image;

[0110] Specifically, this includes:

[0111] S231, perform pixel-by-pixel OR operations on the refined binary edge maps of images at each scale and perform morphological dilation and connection breaking to obtain a fused binary edge map. ;

[0112] ;

[0113] in, Represents logical OR, This indicates a morphological dilation operation. Represents a 3×3 square structural element;

[0114] S232, the continuous spatial image is divided into N overlapping image blocks;

[0115] S243, Perform differentiable rasterization rendering on the m-th image patch to obtain the predicted high-resolution region of the m-th image patch. The rendering formula is:

[0116] ;

[0117] in, This represents the pixel coordinates within the m-th image block, where m represents the image block number, m=1,2,…,N; Represents the coordinates of the Gaussian kernel center The set of all Gaussian kernels falling within the extended neighborhood of the m-th image patch; T denotes the matrix transpose symbol; Denotes the inverse of the covariance matrix; Represents an exponential function;

[0118] S243, Predicting high-resolution regions for all image patches. Perform weighted fusion (overlapping areas use) (Weighted average or distance-inverse weighted average) to finally obtain the complete second-resolution panchromatic image. .

[0119] The pixel value at pixel (x,y) in the second-resolution panchromatic image is: ;

[0120] In this embodiment, the weight of the overlapping region Using the reciprocal of the distance: ,in Let be the distance from pixel (x,y) to the boundary of image patch m. To prevent division by zero for small constants.

[0121] in, The final output second-resolution (high-resolution) panchromatic band image has the pixel value at pixel (x,y). The pixel value at pixel (x,y) of the predicted high-resolution region obtained by rendering the m-th image patch; m represents the image patch number, m=1,2,…,N; (x,y) represents the pixel coordinates, x is the horizontal column coordinate, and y is the vertical row coordinate; This represents the set of all image patch indices containing coordinates (x, y).

[0122] The super-resolution model is trained based on a pyramid-style multi-scale progressive strategy: In order to achieve panchromatic band super-resolution reconstruction from ×1.5 to ×4 (any non-integer / integer multiple), this scheme is based on a pyramid-style multi-scale progressive training strategy. During training, this embodiment uses N=3 magnification scales (the first scale r1=1.5, the second scale r2=2, and the third scale r3=4, respectively).

[0123] At the nth scale, n=1,2,…,N; the super-resolution model takes the corresponding first-resolution panchromatic band image as input and finally outputs the second-resolution panchromatic band image at that scale; the total loss function is defined as the weighted sum of multi-scale pixel loss, structural similarity loss, edge fidelity loss, and high-frequency loss in the frequency domain:

[0124] Configure N scales that gradually increase in size. At the nth scale, n=1,2,…,N, the total loss function is… for:

[0125] ;

[0126] ;

[0127] ;

[0128] ;

[0129] Where N represents the total number of scales; This represents the pixel-level L1 loss at the nth scale; This represents the second-resolution panchromatic band image output by the network at the nth scale; The corresponding true high-resolution ground image at the nth scale; , These represent the height and width of the image at the nth scale, respectively. This represents the structural similarity loss at the nth scale; This represents the edge fidelity loss at the nth scale; This represents the high-frequency loss in the frequency domain at the nth scale; This represents the weight of the multi-scale structural similarity loss at the nth scale. This represents the edge fidelity loss weight at the nth scale; This represents the frequency domain high-frequency loss weight at the nth scale; Represents the L1 norm; This is the edge confidence map at the nth scale, with values ​​ranging from [0,1], approaching 1 at edges and approaching 0 at non-edges; (x,y) represents the pixel coordinates, where x is the column coordinate and y is the row coordinate; Represents the pixel coordinates of the corresponding real high-resolution ground image at the nth scale; (u,v) represents the pixel coordinates of the second-resolution panchromatic band image output by the network at the nth scale; (u,v) represents the spectral coordinates. Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Represents the two-dimensional fast Fourier transform function; The high-frequency region is defined as all frequency points whose distance from the center of the spectrum is greater than 0.6 × Nyquist frequency; the optimal values ​​in this implementation are as follows: , , ;

[0130] To further enhance the mid-to-high frequency texture details of panchromatic remote sensing images, this invention utilizes a total loss function. The high-frequency loss in the frequency domain is applied to the second-resolution panchromatic image during the training phase. With the corresponding second-resolution ground truth image Apply high-frequency constraint loss in the frequency domain. The empirically optimal weights for structural similarity loss, edge fidelity loss, and high-frequency loss are as follows: , , Through the aforementioned multi-scale joint supervision, the model maintains sharp edges, clear textures, and natural spectral distribution at different magnification levels.

[0131] Example 2

[0132] This embodiment provides a super-resolution system based on edge enhancement and frequency domain optimization, such as... Figure 2 As shown, the system for implementing the super-resolution method based on edge enhancement and frequency domain optimization described in Example 1 includes:

[0133] The input processing module is used to acquire the original panchromatic band image and preprocess the original panchromatic band image to obtain a first resolution panchromatic band image.

[0134] The super-resolution module is used to input a first-resolution panchromatic band image into a trained super-resolution model to generate a second-resolution panchromatic band image; the super-resolution model is trained using a multi-scale progressive strategy.

[0135] The super-resolution model includes: a feature encoding and edge enhancement module, a Gaussian scattering modeling module, and a block rasterization rendering module. The feature encoding and edge enhancement module includes a feature encoding module and an edge enhancement module.

[0136] The feature encoding module is used to capture local texture information and global structural information from the first-resolution panchromatic band image to generate multi-scale feature maps;

[0137] An edge enhancement module is used to extract edge features from a first-resolution panchromatic band image and generate an edge confidence map based on the edge features.

[0138] A Gaussian scattering modeling module is used to fuse the edge confidence map and the multi-scale feature map to generate a continuous spatial image.

[0139] The block rasterization rendering module is used to perform block rasterization rendering and fusion on the continuous spatial image to obtain a second resolution panchromatic band image.

[0140] The super-resolution model training process also includes a frequency domain optimization and output generation module and a multi-scale training module. The frequency domain optimization and output generation module optimizes the high-frequency components of the second-resolution panchromatic band image obtained by the block rasterization rendering module and stitches them together for output. The multi-scale training module is used for multi-scale optimization training and total loss calculation. Specifically, it is trained based on a pyramid-style multi-scale progressive strategy. In the total loss calculation process, the total loss function is defined as the weighted sum of pixel-level L1 loss, structural similarity loss, edge fidelity loss and frequency domain high-frequency loss under multiple scales.

[0141] like Figure 3 As shown, the super-resolution model in this embodiment includes: an input layer for inputting the original panchromatic band image; a U-Net encoder for downsampling and feature extraction of the panchromatic band image; a multi-scale feature extraction layer and a RESSEEM edge enhancement module for fusing multi-scale features and generating edge confidence maps, respectively; a Gaussian scattering parameter mapping layer for parameterizing Gaussian kernels to generate continuous spatial images; a rasterization rendering layer for performing block rasterization rendering and fusion of the continuous spatial images; a frequency domain optimization layer for optimizing high-frequency details; and an output layer for outputting high-resolution images.

[0142] The specific super-resolution model training process is as follows: Figure 4As shown: After initializing the super-resolution model parameters, a low-resolution image (first-resolution panchromatic band image) is loaded, and forward propagation generates a high-resolution predicted image (second-resolution panchromatic band image). After passing through the U-Net encoder and the RESSEEM edge enhancement module, the total loss function is based on the weighted sum of pixel-level L1 loss (L1), structural similarity loss (MS-SSIM), edge fidelity loss, and high-frequency loss in the frequency domain at different scales. Calculate the multi-scale loss, perform backpropagation and parameter update based on the multi-scale loss results, implement gradient descent, optimize the model, save the model and output the results after the convergence condition is met, otherwise return to perform forward propagation again.

[0143] Example 3

[0144] This embodiment provides a computer-readable medium having a computer program stored thereon. The computer program, when executed by a processor, can implement the super-resolution method based on edge enhancement and frequency domain optimization as described in Embodiment 1.

[0145] Step 1: Acquire the original panchromatic band image and preprocess the original panchromatic band image to obtain the first resolution panchromatic band image;

[0146] Step 2: Input the first-resolution panchromatic band image into the trained super-resolution model to generate a second-resolution panchromatic band image; the super-resolution model is trained using a multi-scale progressive strategy.

[0147] S21, capture local texture information and global structure information from the first resolution panchromatic band image to generate a multi-scale feature map;

[0148] S22, extract the edge features of the first resolution panchromatic band image, and generate an edge confidence map based on the edge features;

[0149] S23, the edge confidence map and the multi-scale feature map are fused to generate a continuous spatial image;

[0150] S24, the continuous spatial image is segmented into rasterization rendering and fusion to obtain a second resolution panchromatic band image.

[0151] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A super-resolution method based on edge enhancement and frequency domain optimization, characterized in that the method... include: Acquire raw panchromatic band images and preprocess the raw panchromatic band images to obtain a first-resolution panchromatic band image; A first-resolution panchromatic band image is input into a trained super-resolution model to generate a second-resolution panchromatic band image. Local texture and global structural information are captured from the first-resolution panchromatic band image to generate a multi-scale feature map. Edge features of the first-resolution panchromatic band image are extracted, and an edge confidence map is generated based on these edge features. The edge confidence map and the multi-scale feature map are fused to generate a continuous spatial image. The continuous spatial image is then subjected to block-based rasterization rendering and fusion to obtain the second-resolution panchromatic band image. The super-resolution model is trained using a multi-scale progressive strategy. The second resolution is higher than the first resolution; The method for generating the edge confidence map includes: A multi-scale image set is obtained by smoothing the first-resolution panchromatic band image using bilateral filtering technology. The gradient direction of each scale image in the multi-scale image set is determined, and adaptive high and low threshold calculations are performed to obtain a preliminary binary edge map; Candidate edge points are obtained from the preliminary binary edge maps of images at various scales. The window standard deviation of the gradient direction is calculated within a local window centered on each candidate edge point. A preset window standard deviation threshold range is used to retain candidate edge points whose window standard deviations satisfy the window standard deviation threshold range, thus obtaining a refined binary edge map. After fusing the refined binary edge maps of images at various scales, Euclidean distance transformation is performed to obtain the distance field from each pixel to the nearest edge point, and an edge confidence map is generated based on the distance field. Methods for obtaining multi-scale image sets include: The first-resolution panchromatic band image is smoothed using K bilateral filters at different scales to obtain a multi-scale image set; where the scale image obtained by the k-th scale bilateral filter is ; k=1,2,…,K; ； ； in, This represents the standard deviation of the spatial domain at the k-th scale; This represents the standard deviation of the k-th value range, taking... , This represents the maximum pixel value of the panchromatic band image at the first resolution after normalization. This represents the minimum pixel value of the normalized first-resolution panchromatic band image; Indicates the normalized weights; This represents the first-resolution panchromatic band image obtained after preprocessing. Indicates the center pixel; Represents neighboring pixels; Represents a local window; exp() represents the exponential function; This represents the pixel value of the center pixel p in the preprocessed first-resolution panchromatic band image; Represents the pixel value of the neighboring pixel q in the preprocessed first-resolution panchromatic band image; ||#|| represents the Euclidean distance.

2. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 1, characterized in that, The preprocessing method includes: First, crop the original panchromatic image to the target size; The pixel values ​​of the cropped original panchromatic band image are normalized and adjusted to the range of [0, 1]. The normalized data is downsampled to obtain the first-resolution panchromatic band image.

3. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 1, characterized in that, Methods for obtaining preliminary binary edge maps include: The horizontal gradient is calculated using the Sobel operator for the k-th scale image. and vertical gradient The gradient magnitude of the k-th scale image is determined based on the horizontal and vertical gradients. and gradient direction : ； ； Where k(x, y) represents the pixel coordinates in the scaled image obtained by bilateral filtering at the k-th scale, x is the column coordinate and y is the row coordinate; arctan() represents the arctangent function; Gradient magnitude histograms of images at various scales are obtained. High and low thresholds are determined based on Otsu's method. Non-maximum suppression and double thresholding are then applied to images at various scales to obtain preliminary binary edge maps.

4. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 1, characterized in that, Methods for obtaining the distance field include: The refined binary edge maps of images at various scales are processed pixel-by-pixel and subjected to morphological dilation and connection breaking to obtain a fused binary edge map. ; For fused binary edge maps The range field is obtained by performing Euclidean distance transformation. : ； Where (x, y) represents pixel coordinates, x is the column coordinate and y is the row coordinate; This represents the coordinates of edge points in the fused binary edge map; Represents the fused binary edge map The set of coordinates of all edge points in the middle; This represents the L2 norm.

5. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 1, characterized in that, Methods for acquiring second-resolution panchromatic band imagery include: After fusing the edge confidence map with the multi-scale feature map, a set of two-dimensional Gaussian kernel parameters {( , , , ) | i=1,2,…, }, forming a continuous spatial image; among which, This represents the total number of Gaussian kernels obtained from the regression; Indicates the center coordinates of the i-th Gaussian kernel; The 2×2 covariance matrix of the i-th Gaussian kernel controls the shape and orientation of the ellipse; This represents the grayscale value of the i-th Gaussian kernel; The opacity of the i-th Gaussian kernel is represented by the edge confidence map modulation. Denotes the set of all real numbers; The continuous spatial image is divided into N overlapping image blocks; Differentiable rasterization rendering is performed on the m-th image patch to obtain the predicted high-resolution region of the m-th image patch. The rendering formula is: ； in, This represents the pixel coordinates within the m-th image block, where m represents the image block number, m=1,2,…,N; Represents the coordinates of the Gaussian kernel center. The set of all Gaussian kernels falling within the extended neighborhood of the m-th image patch; T denotes the matrix transpose symbol; Denotes the inverse of the covariance matrix; Represents an exponential function; Weighted fusion of the predicted high-resolution regions from all image patches yields a complete second-resolution panchromatic image. .

6. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 1, characterized in that, The multi-scale progressive strategy includes training based on a pyramid-style multi-scale progressive strategy: configuring N progressively increasing scales, where at the nth scale, n=1,2,…,N, the total loss function is: ； ； ； ； ； ； Where N represents the total number of scales; This represents the pixel-level L1 loss at the nth scale; This represents the second-resolution panchromatic band image output by the network at the nth scale; The corresponding high-resolution ground image at the nth scale; , These represent the height and width of the image at the nth scale, respectively. This represents the structural similarity loss at the nth scale; This represents the edge fidelity loss at the nth scale; This represents the high-frequency loss in the frequency domain at the nth scale; This represents the structural similarity loss weight at the nth scale; This represents the edge fidelity loss weight at the nth scale; This represents the frequency domain high-frequency loss weight at the nth scale; Represents the L1 norm; This is the edge confidence map at the nth scale, with values ​​ranging from [0,1], approaching 1 at edges and approaching 0 at non-edges; (x,y) represents the pixel coordinates, where x is the column coordinate and y is the row coordinate; Represents the pixel coordinates of the corresponding real high-resolution ground image at the nth scale; (u,v) represents the pixel coordinates of the second-resolution panchromatic band image output by the network at the nth scale; (u,v) represents the spectral coordinates. Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Represents the two-dimensional fast Fourier transform function; Indicates the high-frequency region.

7. A super-resolution system based on edge enhancement and frequency domain optimization, characterized in that, The system for implementing the super-resolution method based on edge enhancement and frequency domain optimization as described in any one of claims 1-6 includes: The input processing module is used to acquire the original panchromatic band image and preprocess the original panchromatic band image to obtain a first resolution panchromatic band image. The super-resolution module is used to input a first-resolution panchromatic band image into a trained super-resolution model to generate a second-resolution panchromatic band image; the super-resolution model is trained using a multi-scale progressive strategy; the second resolution is higher than the first resolution. The super-resolution model includes: The feature encoding module is used to capture local texture information and global structural information from the first-resolution panchromatic band image to generate multi-scale feature maps; An edge enhancement module is used to extract edge features from a first-resolution panchromatic band image and generate an edge confidence map based on the edge features. A Gaussian scattering modeling module is used to fuse the edge confidence map and the multi-scale feature map to generate a continuous spatial image. The block rasterization rendering module is used to perform block rasterization rendering and fusion on the continuous spatial image to obtain a second resolution panchromatic band image.

8. A computer-readable medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, can implement the super-resolution method based on edge enhancement and frequency domain optimization as described in any one of claims 1-6.

Images