Super-resolution method and system based on edge enhancement and frequency domain optimization, and medium

Abstract

The invention discloses a super-resolution method and system based on edge enhancement and frequency domain optimization, and a medium, and relates to the technical field of remote sensing image processing and artificial intelligence. Fusing the edge confidence map and the multi-scale feature map based on a super-resolution model to generate a continuous space image, and adapting to panchromatic wave band characteristics; meanwhile, a frequency domain optimization strategy is introduced into the super-resolution model, the quality of high-frequency texture and edge reconstruction is improved, a dynamic rasterization rendering technology of frequency domain optimization is combined, the problems of a remote sensing edge enhancement module, optimization of panchromatic wave band edge details and targeted optimization of the edge details are solved, and the performance is particularly better in building and vegetation textures.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of remote sensing image processing and artificial intelligence, in particular to an edge enhancement and frequency domain optimization based super-resolution method, system and medium. BACKGROUND

[0002] Remote sensing image super-resolution reconstruction technology aims to recover or reconstruct low spatial resolution remote sensing images into high spatial resolution images through algorithm processing, so as to improve the detail information and practical value of the image. At present, this method has important application needs in the fields of land resource investigation, environmental monitoring, urban planning and disaster assessment. With the rapid development of deep learning technology, the super-resolution method based on convolutional neural network (CNN) has become the mainstream. These methods usually learn the complex mapping relationship from low resolution images to high resolution images through end-to-end network architecture, and show strong ability in feature extraction and image reconstruction.

[0003] However, the existing deep learning based super-resolution method still has some obvious defects, especially in processing high spatial resolution remote sensing images such as panchromatic band, which restricts the further improvement of the reconstruction quality:

[0004] Firstly, in the aspect of spatial relationship modeling, most of the existing methods rely on fixed size convolution kernel for local feature extraction. This inductive bias is conducive to the extraction of local features, but it is difficult to effectively capture the complex and irregular spatial structures widely existing in remote sensing images (such as winding road network, irregular water boundary, fragmented vegetation patch, etc.). The receptive field of fixed convolution kernel is limited, and the modeling ability of long distance dependence and global spatial context relationship is insufficient, resulting in blurred feature boundary and distorted structure in the reconstructed image, which is difficult to maintain the geometric structure and spatial topological relationship in the real scene.

[0005] Secondly, in the aspect of modal information utilization, the existing method fails to fully exploit and utilize the high spatial resolution characteristics of panchromatic band image; panchromatic band usually contains rich texture details and high frequency information, but in most super-resolution frameworks, these information is introduced through simple feature splicing or shallow fusion method, without realizing deep and multi-scale feature interaction and complementation. Due to the lack of effective cross-modal information integration mechanism, the reconstruction process is easy to fall into local optimum, resulting in limited high frequency detail recovery ability, and the recovered texture lacks authenticity. When recovering local details, it cannot maintain full consistency with the global semantic context of the image (such as feature class, distribution pattern), affecting the naturalness and visual quality of the overall image.

[0006] Finally, in the balance between local details and global consistency, the existing model architecture is often difficult to balance the accurate reconstruction of local high-frequency information and the overall coordination of global semantic structure. On the one hand, excessive attention to local details can lead to noise amplification, artifacts or texture oversmoothing in the reconstruction result; on the other hand, excessive emphasis on global consistency may lose important edge and texture features. The lack of balance between local and global information makes it difficult to achieve an ideal trade-off between detail clarity and overall structural coherence in the reconstructed image, limiting the application effect of super-resolution technology in high-quality remote sensing image generation.

[0007] Therefore, there is an urgent need for a super-resolution reconstruction method that can more effectively model the complex spatial structure of remote sensing images, deeply integrate multi-modal high-resolution information, and cooperatively optimize local details and global consistency, in order to improve the spatial accuracy, visual quality and application value of the reconstructed image. SUMMARY

[0008] To solve the problem that the current remote sensing image super-resolution method mainly relies on deep learning model for feature extraction and reconstruction, there is a lack of spatial relationship modeling, insufficient use of modal information, and insufficient balance between local and global consistency, which affects the overall quality of the reconstructed image; The purpose of the present application is to provide an edge enhancement and frequency domain optimization based super-resolution method, system and medium, which generates a continuous spatial image based on the fusion of edge confidence map and multi-scale feature map in the super-resolution model, and adapts to the characteristics of panchromatic band; At the same time, the super-resolution model introduces a frequency domain optimization strategy to improve the reconstruction quality of high-frequency texture and edge, and combines the dynamic rasterization rendering technology of frequency domain optimization to solve the problem of remote sensing edge enhancement module, optimize the edge details of panchromatic band, and optimize the edge details in a targeted manner, especially in building and vegetation texture.

[0009] The above technical purpose of the present application is realized by the following technical scheme:

[0010] The present application provides an edge enhancement and frequency domain optimization based super-resolution method, which comprises:

[0011] An original panchromatic band image is collected, and the original panchromatic band image is preprocessed to obtain a first resolution panchromatic band image;

[0012] A first-resolution panchromatic band image is input into a trained super-resolution model to generate a second-resolution panchromatic band image. Local texture and global structural information are captured from the first-resolution panchromatic band image to generate a multi-scale feature map. Edge features of the first-resolution panchromatic band image are extracted, and an edge confidence map is generated based on these edge features. The edge confidence map and the multi-scale feature map are fused to generate a continuous spatial image. The continuous spatial image is then subjected to block rasterization rendering and fusion to obtain the second-resolution panchromatic band image. The super-resolution model is trained using a multi-scale progressive strategy. The second resolution is higher than the first resolution.

[0013] A further optimized solution is that the preprocessing method includes:

[0014] First, crop the original panchromatic image to the target size;

[0015] The pixel values ​​of the cropped original panchromatic band image are normalized and adjusted to the range of [0, 1].

[0016] The normalized data is downsampled to obtain the first-resolution panchromatic band image.

[0017] A further optimized solution is that the method for generating the edge confidence map includes:

[0018] A multi-scale image set is obtained by smoothing the first-resolution panchromatic band image using bilateral filtering technology.

[0019] The gradient direction of each scale image in the multi-scale image set is determined, and adaptive high and low threshold calculations are performed to obtain a preliminary binary edge map;

[0020] Candidate edge points are obtained from the preliminary binary edge maps of images at various scales. The window standard deviation of the gradient direction is calculated within a local window centered on each candidate edge point. A preset window standard deviation threshold range is used to retain candidate edge points whose window standard deviations satisfy the window standard deviation threshold range, thus obtaining a refined binary edge map.

[0021] After fusing the refined binary edge maps of images at various scales, Euclidean distance transformation is performed to obtain the distance field from each pixel to the nearest edge point, and an edge confidence map is generated based on the distance field.

[0022] A further optimized approach is to include methods for acquiring multi-scale image sets, such as:

[0023] The first-resolution panchromatic band image is smoothed using K bilateral filters at different scales to obtain a multi-scale image set; among which, the scale image obtained by the k-th scale bilateral filter is... k=1,2,…,K;

[0024] ; ;

[0025] in, This represents the standard deviation of the spatial domain at the k-th scale; This represents the standard deviation of the k-th range, taking... , This represents the maximum pixel value of the panchromatic band image at the first resolution after normalization. Represents the minimum pixel value of the first-resolution panchromatic band image after normalization; represents the normalization weight. This represents the first-resolution panchromatic band image obtained after preprocessing. Indicates the center pixel; Represents neighboring pixels; Represents a local window; exp() represents the exponential function; This represents the pixel value of the center pixel p in the preprocessed first-resolution panchromatic band image; Represents the pixel value of the neighboring pixel q in the preprocessed first-resolution panchromatic band image; ||#|| represents the Euclidean distance.

[0026] A further optimized approach is to obtain the initial binary edge map using the following methods:

[0027] The horizontal gradient is calculated using the Sobel operator for the k-th scale image. and vertical gradient The gradient magnitude of the k-th scale image is determined based on the horizontal and vertical gradients. and gradient direction :

[0028] ;

[0029] ;

[0030] Where k(x, y) represents the pixel coordinates in the scaled image obtained by bilateral filtering at the k-th scale, x is the column coordinate and y is the row coordinate; arctan() represents the arctangent function;

[0031] Gradient magnitude histograms of images at various scales are obtained. High and low thresholds are determined based on Otsu's method. Non-maximum suppression and double thresholding are then applied to images at various scales to obtain preliminary binary edge maps.

[0032] A further optimized solution is to obtain the distance field using the following methods:

[0033] The refined binary edge maps of images at various scales are processed pixel-by-pixel and subjected to morphological dilation and connection breaking to obtain a fused binary edge map. ;

[0034] Fusion binary edge map The range field is obtained by performing Euclidean distance transformation. :

[0035] ;

[0036] Where (x, y) represents pixel coordinates, x is the column coordinate and y is the row coordinate; Represents the fused binary edge map The set of coordinates of all edge points in the middle; This represents the L2 norm.

[0037] A further optimized approach is to acquire the second-resolution panchromatic band image using the following methods:

[0038] After fusing the edge confidence map with the multi-scale feature map, a set of two-dimensional Gaussian kernel parameters {( , , , ) | i=1,2,…, }, forming a continuous spatial image; among which, This represents the total number of Gaussian kernels obtained from the regression; Indicates the center coordinates of the i-th Gaussian kernel; The 2×2 covariance matrix of the i-th Gaussian kernel controls the shape and orientation of the ellipse; This represents the grayscale value of the i-th Gaussian kernel; The opacity of the i-th Gaussian kernel is represented by the edge confidence map modulation. Denotes the set of all real numbers;

[0039] The continuous spatial image is divided into N overlapping image blocks;

[0040] Differentiable rasterization rendering is performed on the m-th image patch to obtain the predicted high-resolution region of the m-th image patch. The rendering formula is:

[0041] ;

[0042] in, This represents the pixel coordinates within the m-th image block, where m represents the image block number, m=1,2,…,N; Represents the coordinates of the Gaussian kernel center The set of all Gaussian kernels falling within the extended neighborhood of the m-th image patch; T denotes the matrix transpose symbol; Denotes the inverse of the covariance matrix; Represents an exponential function;

[0043] Predicted high-resolution regions for all image patches Weighted fusion is performed to obtain a complete second-resolution panchromatic image. .

[0044] A further optimization scheme is as follows: the multi-scale progressive strategy includes training based on a pyramid-style multi-scale progressive training strategy: configuring N progressively increasing scales, where at the nth scale, n=1,2,…,N, and the total loss function… for:

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] Where N represents the total number of scales; represents the pixel-level L1 loss at the nth scale; This represents the second-resolution panchromatic band image output by the network at the nth scale; The corresponding true high-resolution ground image at the nth scale; , These represent the height and width of the image at the nth scale, respectively. This represents the structural similarity loss at the nth scale; This represents the edge fidelity loss at the nth scale; This represents the high-frequency loss in the frequency domain at the nth scale; This represents the structural similarity loss weight at the nth scale; This represents the edge fidelity loss weight at the nth scale; This represents the frequency domain high-frequency loss weight at the nth scale; Represents the L1 norm; This is the edge confidence map at the nth scale, with values ​​ranging from [0,1], approaching 1 at edges and approaching 0 at non-edges; (x,y) represents the pixel coordinates, where x is the column coordinate and y is the row coordinate; Represents the pixel coordinates of the corresponding real high-resolution ground image at the nth scale; (u,v) represents the pixel coordinates of the second-resolution panchromatic band image output by the network at the nth scale; (u,v) represents the spectral coordinates. Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Represents the two-dimensional fast Fourier transform function; Indicates the high-frequency region.

[0050] This solution also provides a super-resolution system based on edge enhancement and frequency domain optimization to implement the aforementioned super-resolution method based on edge enhancement and frequency domain optimization. The system includes:

[0051] The input processing module is used to acquire the original panchromatic band image and preprocess the original panchromatic band image to obtain a first resolution panchromatic band image.

[0052] The super-resolution module is used to input a first-resolution panchromatic band image into a trained super-resolution model to generate a second-resolution panchromatic band image; the super-resolution model is trained using a multi-scale progressive strategy; the second resolution is higher than the first resolution.

[0053] The super-resolution model includes:

[0054] The feature encoding module is used to capture local texture information and global structural information from the first-resolution panchromatic band image to generate multi-scale feature maps;

[0055] An edge enhancement module is used to extract edge features from a first-resolution panchromatic band image and generate an edge confidence map based on the edge features.

[0056] A Gaussian scattering modeling module is used to fuse the edge confidence map and the multi-scale feature map to generate a continuous spatial image.

[0057] The block rasterization rendering module is used to perform block rasterization rendering and fusion on the continuous spatial image to obtain a second resolution panchromatic band image.

[0058] This solution also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, can implement the super-resolution method based on edge enhancement and frequency domain optimization as described above.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] This invention provides a super-resolution method, system, and medium based on edge enhancement and frequency domain optimization. Focusing on panchromatic band characteristics, it generates continuous spatial images by fusing edge confidence maps and multi-scale feature maps based on a super-resolution model, adapting to panchromatic band characteristics. At the same time, the super-resolution model introduces a frequency domain optimization strategy to improve the quality of high-frequency textures and edge reconstruction. Combined with frequency domain optimized dynamic rasterization rendering technology, it solves the problem of remote sensing edge enhancement module, optimizes panchromatic band edge details, and specifically optimizes edge details, especially showing better performance in building and vegetation textures. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0062] Figure 1 This is a schematic diagram of the super-resolution method based on edge enhancement and frequency domain optimization.

[0063] Figure 2 A schematic diagram illustrating the principle of super-resolution based on edge enhancement and frequency domain optimization;

[0064] Figure 3 This is a schematic diagram of the super-resolution model architecture;

[0065] Figure 4 This is a schematic diagram of the super-resolution model training process. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0067] In existing technologies, remote sensing image super-resolution methods mainly rely on deep learning models for feature extraction and reconstruction. Representative techniques include: CNN networks, which rely on fixed convolutional kernels and struggle to effectively capture complex and irregular spatial structures (such as building and vegetation textures) in panchromatic remote sensing images; and Transformer networks, while capable of modeling global dependencies, suffer from high computational complexity. Existing methods rarely fully utilize the high spatial resolution information of the panchromatic band, resulting in limited high-frequency detail recovery capabilities. In recent years, Gaussian scattering has shown excellent performance in continuous image representation, particularly in image super-resolution tasks, achieving smooth reconstruction by encoding continuous Gaussian fields. However, these methods primarily target natural images and are not optimized for the edge and frequency domain characteristics of the panchromatic band; furthermore, existing patented technologies suffer from drawbacks such as insufficient utilization of modal information, inadequate reconstruction of edge and frequency domain details, low computational efficiency, and high data acquisition costs. Therefore, this solution provides the following embodiments to address the aforementioned technical problems:

[0068] Example 1

[0069] This embodiment provides a super-resolution method based on edge enhancement and frequency domain optimization, such as... Figure 1 As shown, the method includes:

[0070] Step 1: Acquire the original panchromatic band image and preprocess it to obtain the first-resolution panchromatic band image. Generally, the original panchromatic band image is a single-channel grayscale image, usually derived from satellite sensors, such as the Panchromatic band of Landsat 8 or Sentinel-2.

[0071] In step one, the preprocessing methods include:

[0072] S11, First, crop the original panchromatic band image to the target size;

[0073] S12, normalize and adjust the pixel values ​​of the cropped original panchromatic band image to the range of [0, 1];

[0074] S13, downsample the normalized data to obtain the first resolution panchromatic band image.

[0075] First-resolution panchromatic band image Represented as:

[0076] ;

[0077] in, This represents the original panchromatic band image, and s represents the downsampling factor, such as 2 or 4;

[0078] This indicates bilinear interpolation or Gaussian downsampling. This indicates that the pixel value is normalized.

[0079] Step two involves inputting the first-resolution panchromatic band image into the trained super-resolution model to generate a second-resolution panchromatic band image, where the second resolution is higher than the first resolution. Specifically, step two includes the following sub-steps:

[0080] S21, capture local texture information and global structural information from the first-resolution panchromatic band image to generate a multi-scale feature map; specifically, the super-resolution model extracts multi-scale features based on the U-Net architecture, and captures local texture information and global structural information from the first-resolution panchromatic band image to generate a multi-scale feature map.

[0081] The first-resolution panchromatic band image is smoothed using K bilateral filters at different scales to obtain a multi-scale image set; among which, the scale image obtained by the k-th scale bilateral filter is... k=1,2,…,K;

[0082] ; ;

[0083] in, This represents the standard deviation of the spatial domain at the k-th scale (e.g., 2 for k=1, 4 for k=2, and so on). This represents the standard deviation of the k-th range, taking... , This represents the maximum pixel value of the first-resolution panchromatic band image after normalization, typically 1.0; This represents the minimum pixel value of the normalized, first-resolution panchromatic band image, typically 0.0. Therefore, in actual calculations... It is usually taken as 0.1 (a constant); Indicates the normalized weights; This represents the first-resolution panchromatic band image obtained after preprocessing. Indicates the center pixel; Represents neighboring pixels; Represents a local window; exp() represents the exponential function; This represents the pixel value of the center pixel p in the preprocessed first-resolution panchromatic band image; Represents the pixel value of the neighboring pixel q in the preprocessed first-resolution panchromatic band image; ||#|| represents the Euclidean distance.

[0084] In this embodiment, K=4, and bilateral filtering with four different spatial-range parameters is used for smoothing to obtain a multi-scale image set. ;

[0085] S22, extract the edge features of the first resolution panchromatic band image, and generate an edge confidence map based on the edge features;

[0086] Specifically, the super-resolution model introduces a Remote Sensing Edge Enhancement Module (RSEEM) to enhance edge details. The RSEEM first extracts edge features using an improved Canny edge detection method, and then uses convolutional layers to generate an edge confidence map. The edge confidence map is used to guide subsequent Gaussian scattering modeling, especially for optimizing the edges of ground features (such as building outlines or road boundaries).

[0087] In step S22, the method for generating the edge confidence map includes:

[0088] S221, determine the gradient direction of each scale image in the multi-scale image set, and perform adaptive high and low threshold calculations to obtain a preliminary binary edge map; specifically:

[0089] The horizontal gradient is calculated using the Sobel operator for the k-th scale image. and vertical gradient The gradient magnitude of the k-th scale image is determined based on the horizontal and vertical gradients. and gradient direction :

[0090] ;

[0091] ;

[0092] Where k(x, y) represents the current pixel coordinates in the scaled image obtained by bilateral filtering at the k-th scale, x is the column coordinate, and y is the row coordinate; arctan() represents the arctangent function;

[0093] S222: Obtain the gradient magnitude histograms of images at each scale. Determine the high and low thresholds based on Otsu's method, and perform non-maximum suppression and double thresholding on the images at each scale to obtain preliminary binary edge maps. , In this embodiment, the high threshold is... The low threshold is .

[0094] S223, extract the edge features of the first resolution panchromatic band image, and generate an edge confidence map based on the edge features, specifically including:

[0095] For the k-th scale image For each candidate edge point p=(x,y) in the initial binary edge map, calculate the standard deviation of the gradient direction θ within a local window of size W×W centered at p. Preset window standard deviation threshold range [ [Only keep] [ The candidate edge points are identified, and the remaining points are deleted to obtain the refined binary edge map of the image at the k-th scale.

[0096] The formula for calculating the standard deviation of the gradient direction window is as follows:

[0097] ;

[0098] in, The mean of the gradient direction within the window; p=(x,y) represents the pixel coordinates of the current candidate edge point, where x is the horizontal column coordinate and y is the vertical row coordinate; This represents the standard deviation of the gradient direction within a local window centered at p, in degrees (°). The gradient direction at the neighborhood pixel q within the window (calculated from Sobel, ranging from [-90°, 90°] or [0°, 180°]). This represents a local window set of pixels centered at the center pixel p (e.g., a 3×3 window contains 9 pixels). The total number of effective pixels within the local window (usually W², M=9 in this embodiment); Indicates the window Sum all pixels q in the inner neighborhood.

[0099] In this embodiment, the local window size W=3 (i.e., a 3×3 window, M=9 pixels); the preset threshold range [ [0°, 18°] = Window standard deviation only at the current candidate edge point At 18°, the current candidate edge points are retained; otherwise, they are set to 0 to effectively suppress parallel false edges generated by periodic textures such as farmland and vegetation; finally, a refined binary edge map of each scale image is obtained.

[0100] S224, performs pixel-by-pixel OR operations on the refined binary edge maps of images at various scales and performs morphological dilation and connection breaking to obtain a fused binary edge map. ;

[0101] S225, merging binary edge maps The range field is obtained by performing Euclidean distance transformation. :

[0102] ;

[0103] Where (x, y) represents pixel coordinates, x is the column coordinate and y is the row coordinate; This represents the coordinates of edge points in the fused binary edge map; Represents the fused binary edge map The set of coordinates of all edge points in the middle; This represents the L2 norm.

[0104] Final Marginal Confidence Plot for:

[0105] ;

[0106] in, , represents the Sigmoid function, α=8 represents the steepness coefficient, =10 pixels (empirical optimal value); in this way, the confidence of pixels closer to the real edge is closer to 1, and the confidence of pixels far from the edge is smoothly reduced to 0.

[0107] S23, the continuous spatial image is segmented into rasterization rendering and fusion to obtain a second resolution panchromatic band image;

[0108] Specifically, this includes:

[0109] S231, perform pixel-by-pixel OR operations on the refined binary edge maps of images at each scale and perform morphological dilation and connection breaking to obtain a fused binary edge map. ;

[0110] ;

[0111] in, Represents logical OR, This indicates a morphological dilation operation. Represents a 3×3 square structural element;

[0112] S232, the continuous spatial image is divided into N overlapping image blocks;

[0113] S243, Perform differentiable rasterization rendering on the m-th image patch to obtain the predicted high-resolution region of the m-th image patch. The rendering formula is:

[0114] ;

[0115] in, This represents the pixel coordinates within the m-th image block, where m represents the image block number, m=1,2,…,N; Represents the coordinates of the Gaussian kernel center The set of all Gaussian kernels falling within the extended neighborhood of the m-th image patch; T denotes the matrix transpose symbol; Denotes the inverse of the covariance matrix; Represents an exponential function;

[0116] S243, Predicting high-resolution regions for all image patches. Perform weighted fusion (overlapping areas use) (Weighted average or distance-inverse weighted average) to finally obtain the complete second-resolution panchromatic image. .

[0117] The pixel value at pixel (x,y) in the second-resolution panchromatic image is: ;

[0118] In this embodiment, the weight of the overlapping region Using the reciprocal of the distance: ,in Let be the distance from pixel (x,y) to the boundary of image patch m. To prevent division by zero for small constants.

[0119] in, The final output second-resolution (high-resolution) panchromatic band image has the pixel value at pixel (x,y). The pixel value at pixel (x,y) of the predicted high-resolution region obtained by rendering the m-th image patch; m represents the image patch number, m=1,2,…,N; (x,y) represents the pixel coordinates, x is the horizontal column coordinate, and y is the vertical row coordinate; This represents the set of all image patch indices containing coordinates (x, y).

[0120] The super-resolution model is trained based on a pyramid-style multi-scale progressive strategy: In order to achieve panchromatic band super-resolution reconstruction from ×1.5 to ×4 (any non-integer / integer multiple), this scheme is based on a pyramid-style multi-scale progressive training strategy. During training, this embodiment uses N=3 magnification scales (the first scale r1=1.5, the second scale r2=2, and the third scale r3=4, respectively).

[0121] At the nth scale, n=1,2,…,N; the super-resolution model takes the corresponding first-resolution panchromatic band image as input and finally outputs the second-resolution panchromatic band image at that scale; the total loss function is defined as the weighted sum of multi-scale pixel loss, structural similarity loss, edge fidelity loss, and high-frequency loss in the frequency domain:

[0122] Configure N scales that gradually increase in size. At the nth scale, n=1,2,…,N, the total loss function is… for:

[0123] ;

[0124] ;

[0125] ;

[0126] ;

[0127] Where N represents the total number of scales; This represents the pixel-level L1 loss at the nth scale; This represents the second-resolution panchromatic band image output by the network at the nth scale; The corresponding true high-resolution ground image at the nth scale; , These represent the height and width of the image at the nth scale, respectively. This represents the structural similarity loss at the nth scale; This represents the edge fidelity loss at the nth scale; This represents the high-frequency loss in the frequency domain at the nth scale; This represents the weight of the multi-scale structural similarity loss at the nth scale. This represents the edge fidelity loss weight at the nth scale; This represents the frequency domain high-frequency loss weight at the nth scale; Represents the L1 norm; This is the edge confidence map at the nth scale, with values ​​ranging from [0,1], approaching 1 at edges and approaching 0 at non-edges; (x,y) represents the pixel coordinates, where x is the column coordinate and y is the row coordinate; Represents the pixel coordinates of the corresponding real high-resolution ground image at the nth scale; (u,v) represents the pixel coordinates of the second-resolution panchromatic band image output by the network at the nth scale; (u,v) represents the spectral coordinates. Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Represents the two-dimensional fast Fourier transform function; The high-frequency region is defined as all frequency points whose distance from the center of the spectrum is greater than 0.6 × Nyquist frequency; the optimal values ​​in this implementation are as follows: , , ;

[0128] To further enhance the mid-to-high frequency texture details of panchromatic remote sensing images, this invention utilizes a total loss function. The high-frequency loss in the frequency domain is applied to the second-resolution panchromatic image during the training phase. With the corresponding second-resolution ground truth image Apply high-frequency constraint loss in the frequency domain. The empirically optimal weights for structural similarity loss, edge fidelity loss, and high-frequency loss are as follows: , , Through the aforementioned multi-scale joint supervision, the model maintains sharp edges, clear textures, and natural spectral distribution at different magnification levels.

[0129] Example 2

[0130] This embodiment provides a super-resolution system based on edge enhancement and frequency domain optimization, such as... Figure 2 As shown, the system for implementing the super-resolution method based on edge enhancement and frequency domain optimization described in Example 1 includes:

[0131] The input processing module is used to acquire raw panchromatic band images and preprocess the raw panchromatic band images to obtain a first-resolution panchromatic band image.

[0132] The super-resolution module is used to input a first-resolution panchromatic band image into a trained super-resolution model to generate a second-resolution panchromatic band image; the super-resolution model is trained using a multi-scale progressive strategy.

[0133] The super-resolution model includes: a feature encoding and edge enhancement module, a Gaussian scattering modeling module, and a block rasterization rendering module. The feature encoding and edge enhancement module includes a feature encoding module and an edge enhancement module.

[0134] The feature encoding module is used to capture local texture information and global structural information from the first-resolution panchromatic band image to generate multi-scale feature maps;

[0135] An edge enhancement module is used to extract edge features from a first-resolution panchromatic band image and generate an edge confidence map based on the edge features.

[0136] A Gaussian scattering modeling module is used to fuse the edge confidence map and the multi-scale feature map to generate a continuous spatial image.

[0137] The block rasterization rendering module is used to perform block rasterization rendering and fusion on the continuous spatial image to obtain a second resolution panchromatic band image.

[0138] The super-resolution model training process also includes a frequency domain optimization and output generation module and a multi-scale training module. The frequency domain optimization and output generation module optimizes the high-frequency components of the second-resolution panchromatic band image obtained by the block rasterization rendering module and stitches them together for output. The multi-scale training module is used for multi-scale optimization training and total loss calculation. Specifically, it is trained based on a pyramid-style multi-scale progressive strategy. In the total loss calculation process, the total loss function is defined as the weighted sum of pixel-level L1 loss, structural similarity loss, edge fidelity loss and frequency domain high-frequency loss under multiple scales.

[0139] like Figure 3 As shown, the super-resolution model in this embodiment includes: an input layer for inputting the original panchromatic band image; a U-Net encoder for downsampling and feature extraction of the panchromatic band image; a multi-scale feature extraction layer and a RESSEEM edge enhancement module for fusing multi-scale features and generating edge confidence maps, respectively; a Gaussian scattering parameter mapping layer for parameterizing Gaussian kernels to generate continuous spatial images; a rasterization rendering layer for performing block rasterization rendering and fusion of the continuous spatial images; a frequency domain optimization layer for optimizing high-frequency details; and an output layer for outputting high-resolution images.

[0140] The specific super-resolution model training process is as follows: Figure 4As shown: After initializing the super-resolution model parameters, a low-resolution image (first-resolution panchromatic band image) is loaded, and forward propagation generates a high-resolution predicted image (second-resolution panchromatic band image). After passing through the U-Net encoder and the RESSEEM edge enhancement module, the total loss function is based on the weighted sum of pixel-level L1 loss (L1), structural similarity loss (MS-SSIM), edge fidelity loss, and high-frequency loss in the frequency domain at different scales. Calculate the multi-scale loss, perform backpropagation and parameter update based on the multi-scale loss results, implement gradient descent, optimize the model, save the model and output the results after the convergence condition is met, otherwise return to perform forward propagation again.

[0141] Example 3

[0142] This embodiment provides a computer-readable medium having a computer program stored thereon. The computer program, when executed by a processor, can implement the super-resolution method based on edge enhancement and frequency domain optimization as described in Embodiment 1.

[0143] Step 1: Acquire the original panchromatic band image and preprocess the original panchromatic band image to obtain the first resolution panchromatic band image.

[0144] Step 2: Input the first-resolution panchromatic band image into the trained super-resolution model to generate a second-resolution panchromatic band image; the super-resolution model is trained using a multi-scale progressive strategy.

[0145] S21, capture local texture information and global structure information from the first resolution panchromatic band image to generate a multi-scale feature map;

[0146] S22, extract the edge features of the first resolution panchromatic band image, and generate an edge confidence map based on the edge features;

[0147] S23, the edge confidence map and the multi-scale feature map are fused to generate a continuous spatial image;

[0148] S24, the continuous spatial image is segmented into rasterization rendering and fusion to obtain a second resolution panchromatic band image.

[0149] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A super-resolution method based on edge enhancement and frequency domain optimization, characterized in that the method... include: Acquire raw panchromatic band images and preprocess the raw panchromatic band images to obtain a first-resolution panchromatic band image; A first-resolution panchromatic band image is input into a trained super-resolution model to generate a second-resolution panchromatic band image. Local texture and global structural information are captured from the first-resolution panchromatic band image to generate a multi-scale feature map. Edge features of the first-resolution panchromatic band image are extracted, and an edge confidence map is generated based on these edge features. The edge confidence map and the multi-scale feature map are fused to generate a continuous spatial image. The continuous spatial image is then subjected to block-based rasterization rendering and fusion to obtain the second-resolution panchromatic band image. The super-resolution model is trained using a multi-scale progressive strategy; the second resolution is higher than the first resolution.

2. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 1, characterized in that, The preprocessing method includes: First, crop the original panchromatic image to the target size; The pixel values ​​of the cropped original panchromatic band image are normalized and adjusted to the range of [0, 1]. The normalized data is downsampled to obtain the first-resolution panchromatic band image.

3. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 1, characterized in that, The method for generating the edge confidence map includes: A multi-scale image set is obtained by smoothing the first-resolution panchromatic band image using bilateral filtering technology. The gradient direction of each scale image in the multi-scale image set is determined, and adaptive high and low threshold calculations are performed to obtain a preliminary binary edge map; Candidate edge points are obtained from the preliminary binary edge maps of images at various scales. The window standard deviation of the gradient direction is calculated within a local window centered on each candidate edge point. A preset window standard deviation threshold range is used to retain candidate edge points whose window standard deviations satisfy the window standard deviation threshold range, thus obtaining a refined binary edge map. After fusing the refined binary edge maps of images at various scales, Euclidean distance transformation is performed to obtain the distance field from each pixel to the nearest edge point, and an edge confidence map is generated based on the distance field.

4. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 3, characterized in that, Methods for obtaining multi-scale image sets include: The first-resolution panchromatic band image is smoothed using K bilateral filters at different scales to obtain a multi-scale image set; among which, the scale image obtained by the k-th scale bilateral filter is... k=1,2,…,K; ； ； in, This represents the standard deviation of the spatial domain at the k-th scale; This represents the standard deviation of the k-th range, taking... , This represents the maximum pixel value of the panchromatic band image at the first resolution after normalization. This represents the minimum pixel value of the normalized first-resolution panchromatic band image; Indicates the normalized weights; This represents the first-resolution panchromatic band image obtained after preprocessing. Indicates the center pixel; Represents neighboring pixels; Represents a local window; exp() represents the exponential function; This represents the pixel value of the center pixel p in the preprocessed first-resolution panchromatic band image; Represents the pixel value of the neighboring pixel q in the preprocessed first-resolution panchromatic band image; ||#|| represents the Euclidean distance.

5. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 3, characterized in that, Methods for obtaining preliminary binary edge maps include: The horizontal gradient is calculated using the Sobel operator for the k-th scale image. and vertical gradient The gradient magnitude of the k-th scale image is determined based on the horizontal and vertical gradients. and gradient direction : ； ； Where k(x, y) represents the pixel coordinates in the scaled image obtained by bilateral filtering at the k-th scale, x is the column coordinate and y is the row coordinate; arctan() represents the arctangent function; Gradient magnitude histograms of images at various scales are obtained. High and low thresholds are determined based on Otsu's method. Non-maximum suppression and double thresholding are then applied to images at various scales to obtain preliminary binary edge maps.

6. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 3, characterized in that, Methods for obtaining the distance field include: The refined binary edge maps of images at various scales are processed pixel-by-pixel and subjected to morphological dilation and connection breaking to obtain a fused binary edge map. ; Fusion binary edge map The range field is obtained by performing Euclidean distance transformation. : ； Where (x, y) represents pixel coordinates, x is the column coordinate and y is the row coordinate; This represents the coordinates of edge points in the fused binary edge map; Represents the fused binary edge map The set of coordinates of all edge points in the middle; This represents the L2 norm.

7. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 1, characterized in that, Methods for acquiring second-resolution panchromatic band imagery include: After fusing the edge confidence map with the multi-scale feature map, a set of two-dimensional Gaussian kernel parameters {( , , , ) | i=1,2,…, }, forming a continuous spatial image; among which, This represents the total number of Gaussian kernels obtained from the regression; Indicates the center coordinates of the i-th Gaussian kernel; The 2×2 covariance matrix of the i-th Gaussian kernel controls the shape and orientation of the ellipse; This represents the grayscale value of the i-th Gaussian kernel; The opacity of the i-th Gaussian kernel is represented by the edge confidence map modulation. Denotes the set of all real numbers; The continuous spatial image is divided into N overlapping image blocks; Differentiable rasterization rendering is performed on the m-th image patch to obtain the predicted high-resolution region of the m-th image patch. The rendering formula is: ； in, This represents the pixel coordinates within the m-th image block, where m represents the image block number, m=1,2,…,N; Represents the coordinates of the Gaussian kernel center The set of all Gaussian kernels falling within the extended neighborhood of the m-th image patch; T denotes the matrix transpose symbol; Denotes the inverse of the covariance matrix; Represents an exponential function; Predicted high-resolution regions for all image patches Weighted fusion is performed to obtain a complete second-resolution panchromatic image. .

8. The super-resolution method based on edge enhancement and frequency domain optimization according to claim 1, characterized in that, The multi-scale progressive strategy includes training based on a pyramid-style multi-scale progressive strategy: configuring N progressively increasing scales, where at the nth scale, n=1,2,…,N, and the total loss function… for: ； ； ； ； ； ； Where N represents the total number of scales; This represents the pixel-level L1 loss at the nth scale; This represents the second-resolution panchromatic band image output by the network at the nth scale; The corresponding true high-resolution ground image at the nth scale; , These represent the height and width of the image at the nth scale, respectively. This represents the structural similarity loss at the nth scale; This represents the edge fidelity loss at the nth scale; This represents the high-frequency loss in the frequency domain at the nth scale; This represents the structural similarity loss weight at the nth scale; This represents the edge fidelity loss weight at the nth scale; This represents the frequency domain high-frequency loss weight at the nth scale; Represents the L1 norm; This is the edge confidence map at the nth scale, with values ​​ranging from [0,1], approaching 1 at edges and approaching 0 at non-edges; (x,y) represents the pixel coordinates, where x is the column coordinate and y is the row coordinate; Represents the pixel coordinates of the corresponding real high-resolution ground image at the nth scale; (u,v) represents the pixel coordinates of the second-resolution panchromatic band image output by the network at the nth scale; (u,v) represents the spectral coordinates. Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Indicates to The spectral coordinates obtained after performing a two-dimensional fast Fourier transform; Represents the two-dimensional fast Fourier transform function; Indicates the high-frequency region.

9. A super-resolution system based on edge enhancement and frequency domain optimization, characterized in that, The system for implementing the super-resolution method based on edge enhancement and frequency domain optimization as described in any one of claims 1-8 includes: The input processing module is used to acquire the original panchromatic band image and preprocess the original panchromatic band image to obtain a first resolution panchromatic band image. The super-resolution module is used to input a first-resolution panchromatic band image into a trained super-resolution model to generate a second-resolution panchromatic band image; the super-resolution model is trained using a multi-scale progressive strategy; the second resolution is higher than the first resolution. The super-resolution model includes: The feature encoding module is used to capture local texture information and global structural information from the first-resolution panchromatic band image to generate multi-scale feature maps; An edge enhancement module is used to extract edge features from a first-resolution panchromatic band image and generate an edge confidence map based on the edge features. A Gaussian scattering modeling module is used to fuse the edge confidence map and the multi-scale feature map to generate a continuous spatial image. The block rasterization rendering module is used to perform block rasterization rendering and fusion on the continuous spatial image to obtain a second resolution panchromatic band image.

10. A computer-readable medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, can implement the super-resolution method based on edge enhancement and frequency domain optimization as described in any one of claims 1-8.

Images