A complex field sar image super-resolution reconstruction method and system

Abstract

The application discloses a kind of complex field SAR image super-resolution reconstruction method and system.Therein, method includes the following steps: S1, complex field data pre-processing;S2, data set construction is carried out and data augmentation is carried out;S3, depth learning model is constructed, and training area and verification area are divided, and model training is carried out;S4, using high-resolution original image data carries out precision evaluation;S5, according to the depth learning model of optimal scale training, obtains high-resolution SAR product.The scheme is under the improvement hardware processing capacity, obtains the result that only relies on amplitude diagram super-resolution reconstruction is superior, can restore more ground object details, improves the application value of existing image data natural disaster field and the identification ability of military target.

Description

Technical Field

[0001] This invention relates to the field of synthetic aperture radar (SAR) image processing technology, and in particular to a method and system for super-resolution reconstruction of complex domain SAR images. Background Technology

[0002] Synthetic Aperture Radar (SAR) possesses all-weather, all-day capabilities, making SAR imagery widely applicable in various fields such as target detection, land cover classification, and change detection. Image resolution is a crucial indicator of SAR image quality; higher-resolution images carry more semantic information, leading to better processing results in subsequent advanced vision tasks.

[0003] There are generally three ways to improve SAR image resolution: First, by improving and updating hardware, such as increasing the system's transmit bandwidth and synthetic large aperture to improve the system's theoretical resolution; second, by processing the raw echo data acquired by the SAR imaging system during the imaging stage, such as modern spectral estimation methods and data extrapolation methods, but these are highly dependent on the signal model and have a very large computational load; and third, by using image super-resolution (SR) methods, which apply image processing methods to the imaged SAR image to improve resolution. SAR complex domain imaging breaks through the bottleneck of traditional real-valued imaging by providing full information dimensions of "amplitude + phase," providing more accurate results for observation of high-noise environments, deep tissues, and large-scale dynamic scenes.

[0004] The current bottlenecks in SAR image super-resolution technology are as follows:

[0005] (1) Traditional real-field processing methods lose phase information (measured data show that phase error causes about 23% loss of detail);

[0006] (2) Existing complex domain methods ignore gradient consistency (comparative experiments show that gradient mismatch will reduce PSNR by 2-4dB). Summary of the Invention

[0007] To address the existing problems, this invention provides a method and system for super-resolution reconstruction of complex domain SAR images, the specific scheme of which is as follows:

[0008] A method for super-resolution reconstruction of complex domain SAR images includes the following steps:

[0009] S1, Preprocessing of complex domain data: Input SAR single-look complex image, perform radiometric calibration and thermal noise removal on real and imaginary data, and generate amplitude image to identify ground targets;

[0010] S2, Dataset Construction: Low-resolution sequences {ILR1, ILR2,..., ILR5} are generated using multi-resolution spaceborne SAR images. Phase continuity is maintained through complex domain bilinear interpolation, and data augmentation is performed by combining rotation, flipping, and mirroring operations.

[0011] S3: Construct a deep learning model, divide it into training and validation areas, and train the model.

[0012] S4, using high-resolution raw image data for accuracy evaluation;

[0013] S5, based on a deep learning model trained to the optimal scale, obtains high-resolution SAR products.

[0014] Preferably, step S1 specifically includes the following steps:

[0015] S11, select SAR images of different scenes and resolutions;

[0016] S12, use the amplitude map to determine the image position of the scene, and crop the real part image and the imaginary part image to obtain several samples.

[0017] Preferably, the low-resolution sequence in step S2 is generated through a 5-layer pyramid structure with a scaling factor of 0.8.

[0018] Preferably, the method for constructing the deep learning model in step S3 is as follows:

[0019] First, establish the composite loss function:

[0020] ,

[0021] in To output images in super-resolution, For the target image, The feature map obtained from the jth convolutional layer in the VGG network; Represents the number of image channels, Represents image height, Represents the image width.

[0022] We employ a dilated-residual convolutional neural network with feature reuse. This network uses DR-CNN as the main framework and constructs an FRDR-CNN network as the generative network in the super-resolution generative adversarial network. At the same time, we compare the reconstruction algorithms bicubic interpolation, sparse coding-based reconstruction algorithm ScSR, SRResNet with similar parameter scale, dilated-residual convolutional DR-CNN, and FRDR-CNN with feature reuse.

[0023] Preferably, the specific method for accuracy evaluation in step S4 is as follows:

[0024] Peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and edge preservation coefficient (EPD-ROA) were selected as objective evaluation indicators for the quality of reconstructed images.

[0025] PSNR is determined by mean squared error. Using the original high-resolution image as a benchmark, the results of super-resolution reconstruction are quantitatively evaluated, and the calculation is as follows:

[0026] ,

[0027] Where MSE(x,y) represents the mean square error between two images x and y, L is the gray level of the image, and the larger the PSNR value, the higher the similarity between the two images.

[0028] SSIM reflects the overall image structural similarity, and its definition is as follows:

[0029] ,

[0030] Where x and y represent the original image and the super-resolution reconstructed image, respectively, l(x,y) represents brightness contrast, s(x,y) represents structural information contrast, and c(x,y) represents contrast contrast. The three indices α, β, and γ are used to adjust the proportion of brightness, structural information, and contrast in the SSIM index. The closer the SSIM value is to 1, the stronger the structural similarity between the super-resolution reconstructed image and the original image.

[0031] The HD and VD values ​​of EPD-ROA reflect the edge preservation of the reconstructed image in the horizontal and vertical directions, which is beneficial to the recovery effect of super-resolution reconstruction algorithms on high-frequency information such as boundaries. They are defined as follows:

[0032] ,

[0033] Where m is the total number of pixels in the selected region. , This represents two adjacent pixels in the horizontal and vertical directions of the reconstructed image. , Represents the original high-resolution image with , For two adjacent pixels, the EPD-ROA analysis represents the total edge count of the test image, and the denominator represents the total true edge count. The closer the value is to 1, the better the edge preservation of the super-resolution reconstructed image.

[0034] The present invention also discloses a system based on any of the methods described above, comprising:

[0035] Data preprocessing module: performs radiometric calibration, noise removal, and amplitude map generation;

[0036] Dataset building module: generates low-resolution sequences and augments the data;

[0037] FRDR-CNN network module: includes complex convolutional layers, phase gradient attention module and joint loss function, used to improve the high frequency reconstruction fidelity through joint constraints in the complex domain;

[0038] Reconstruction Output Module: Generates super-resolution complex SAR images.

[0039] The present invention also discloses a computer-readable storage medium storing a computer program, which, when executed, performs the method described in any of the above-mentioned embodiments.

[0040] The present invention also discloses a computer system including a processor, a storage medium storing a computer program, and the processor reading from the storage medium and running the computer program to perform the method described in any of the preceding claims.

[0041] The beneficial effects of this invention are as follows:

[0042] This scheme mainly adopts a dilated-residual convolutional neural network with feature reuse. It constructs a deep learning dataset based on complex domain SAR images. By constructing a joint loss function of phase gradient smoothing term and amplitude edge alignment term, it forces the reconstruction results to conform to the physical laws of electromagnetic scattering and suppresses the generation of artifacts. At the same time, it introduces complex convolutional layers to directly extract features in the joint space of real and imaginary parts, and uses phase gradient information to guide the attention mechanism to focus on key scattering regions, thereby improving the reconstruction fidelity of high-frequency components.

[0043] This solution, by enhancing the processing power of the hardware, achieves results superior to those obtained solely through amplitude map super-resolution reconstruction. It can recover more details of ground features, thereby improving the application value of existing image data in the field of natural disasters and the ability to identify military targets. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 The first row shows low-resolution data, and the second row shows sample images of high-resolution data.

[0046] Figure 2 DR residual element structure diagram;

[0047] Figure 3 SAR super-resolution deep learning architecture;

[0048] Figure 4 : Result image after exceeding the limit;

[0049] Figure 5 : A block diagram illustrating the principle of the method of this invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Super-resolution of SAR amplitude maps only requires processing amplitude information, and the algorithm complexity is much lower than that of complex domain images. However, ignoring phase data makes it impossible to reconstruct key physical quantities such as three-dimensional deformation and micro-displacement of the earth's surface, which limits its application in high-precision scenarios such as settlement monitoring and geological activity analysis.

[0052] This invention constructs a joint loss function consisting of a phase gradient smoothing term and an amplitude edge alignment term, which forces the reconstruction results to conform to the physical laws of electromagnetic scattering and suppresses artifact generation. At the same time, it introduces a complex convolutional layer to directly extract features in the joint space of the real and imaginary parts, and uses phase gradient information to guide the attention mechanism to focus on key scattering regions, thereby improving the fidelity of high-frequency component reconstruction.

[0053] like Figures 1 to 5 A method for super-resolution reconstruction of complex domain SAR images includes the following steps:

[0054] S1, preprocessing of complex domain data: input SAR single-look complex image, perform radiometric calibration and thermal noise removal on real and imaginary data, and generate amplitude image to identify ground targets.

[0055] S11 allows you to select SAR images of different scenes and resolutions, including roads, farmland, buildings, trees, etc.

[0056] S12, use the amplitude map to determine the image position of the scene, and crop the real part image and the imaginary part image to obtain several samples.

[0057] S2. Dataset Construction: The data consists of multiple satellite-borne SAR images of different resolutions taken at similar times and in the same area. First, a 5-layer pyramid is constructed with a scaling factor of 0.8 to generate a low-resolution sequence {ILR1, ILR2, ..., ILR5}. Simultaneously, each layer is sampled using complex-domain bilinear interpolation to maintain phase continuity. Relevant scene images are manually selected and cropped based on the amplitude map. Data augmentation is performed by combining image rotations of 0, 90, 180, and 270°, image flipping, and image mirroring.

[0058] S3. Construct a deep learning model by dividing the generated dataset into a training set and a validation set in a 7:3 ratio for model training.

[0059] The methods for constructing deep learning models are as follows:

[0060] First, establish the composite loss function:

[0061] ,

[0062] in To output images in super-resolution, For the target image, The feature map obtained from the jth convolutional layer in the VGG network; Represents the number of image channels, Represents image height, Represents the image width.

[0063] We employ a dilated-residual convolutional neural network with feature reuse. This network uses DR-CNN as the main framework and constructs an FRDR-CNN network as the generative network in the super-resolution generative adversarial network. At the same time, we compare the reconstruction algorithms bicubic interpolation, sparse coding-based reconstruction algorithm ScSR, SRResNet with similar parameter scale, dilated-residual convolutional DR-CNN, and FRDR-CNN with feature reuse.

[0064] S4, using high-resolution raw image data for accuracy evaluation;

[0065] S5, based on a deep learning model trained to the optimal scale, obtains high-resolution SAR products.

[0066] The specific methods for conducting accuracy evaluation are as follows:

[0067] Peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and edge preservation coefficient (EPD-ROA) were selected as objective evaluation indicators for the quality of reconstructed images.

[0068] PSNR is determined by mean squared error. Using the original high-resolution image as a benchmark, the results of super-resolution reconstruction are quantitatively evaluated, and the calculation is as follows:

[0069] ,

[0070] Where MSE(x,y) represents the mean square error between two images x and y, L is the gray level of the image, and the larger the PSNR value, the higher the similarity between the two images.

[0071] SSIM reflects the overall image structural similarity, and its definition is as follows:

[0072] ,

[0073] Where x and y represent the original image and the super-resolution reconstructed image, respectively, l(x,y) represents brightness contrast, s(x,y) represents structural information contrast, and c(x,y) represents contrast contrast. The three indices α, β, and γ are used to adjust the proportion of brightness, structural information, and contrast in the SSIM index. The closer the SSIM value is to 1, the stronger the structural similarity between the super-resolution reconstructed image and the original image.

[0074] The HD and VD values ​​of EPD-ROA reflect the edge preservation of the reconstructed image in the horizontal and vertical directions, which is beneficial to the recovery effect of super-resolution reconstruction algorithms on high-frequency information such as boundaries. They are defined as follows:

[0075] ,

[0076] Where m is the total number of pixels in the selected region. , This represents two adjacent pixels in the horizontal and vertical directions of the reconstructed image. , Represents the original high-resolution image with , For two adjacent pixels, the EPD-ROA analysis represents the total edge count of the test image, and the denominator represents the total true edge count. The closer the value is to 1, the better the edge preservation of the super-resolution reconstructed image.

[0077] The present invention also discloses a system based on any of the methods described above, comprising:

[0078] 1. Data Preprocessing Module: Used to perform complex domain radiometric calibration, eliminate SAR system gain errors such as range antenna pattern distortion; remove thermal noise from the receiver, especially for weak scattering areas; generate amplitude images to provide ground feature location basis for cropping.

[0079] 2. Dataset Construction Module: Used to construct {ILR1~ILR5} through a 5-layer pyramid (scaling factor 0.8); complex domain bilinear interpolation to maintain phase continuity and avoid phase jumps caused by traditional downsampling; performs 0° / 90° / 180° / 270° rotation, flip, and mirroring to augment the data; breaks through the limitations of traditional real domain augmentation, and the rotation / mirroring operation processes the real and imaginary parts simultaneously, which conforms to the physical laws of electromagnetic scattering.

[0080] 3. FRDR-CNN Network Module: Includes complex convolutional layers, a phase gradient attention module, and a joint loss function. The convolutional layers are used to extract features in the joint real / imaginary space, instead of the traditional magnitude map input. The phase gradient attention mechanism is used to generate an attention mask using the phase gradient to focus on strong scattering regions. The joint loss function is calculated so that the user can simultaneously optimize the VGG feature loss, phase gradient smoothing constraint, and magnitude edge alignment constraint to solve the gradient mismatch problem (PSNR drops by 2-4dB). The joint constraints in the complex domain improve the fidelity of high-frequency reconstruction.

[0081] 4. Reconstruction Output Module: Generates super-resolution complex SAR images and outputs high-resolution amplitude and phase maps, supporting military reconnaissance / disaster monitoring applications. The output format is compatible with professional processing platforms such as ENVI / SNAP.

[0082] This scheme mainly adopts a dilated-residual convolutional neural network with feature reuse. It constructs a deep learning dataset based on complex domain SAR images. By constructing a joint loss function of phase gradient smoothing term and amplitude edge alignment term, it forces the reconstruction results to conform to the physical laws of electromagnetic scattering and suppresses the generation of artifacts. At the same time, it introduces complex convolutional layers to directly extract features in the joint space of real and imaginary parts, and uses phase gradient information to guide the attention mechanism to focus on key scattering regions, thereby improving the reconstruction fidelity of high-frequency components.

[0083] This solution, by enhancing the processing power of the hardware, achieves results superior to those obtained solely through amplitude map super-resolution reconstruction. It can recover more details of ground features, thereby improving the application value of existing image data in the field of natural disasters and the ability to identify military targets.

[0084] The present invention also discloses a computer-readable storage medium and a computer system, wherein the computer-readable storage medium stores a computer program, and the computer program, when executed, performs the method described above. A computer system includes a processor and a storage medium, the storage medium storing a computer program, and the processor reading from the storage medium and running the computer program to perform the method described above.

[0085] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0086] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0087] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0088] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for super-resolution reconstruction of complex-valued SAR images, characterized in that, The method comprises the following steps: S1, pre-processing of complex domain data: inputting the SAR single-view complex image, performing radiation scaling and thermal noise removal on the real part and imaginary part data to generate an amplitude image to identify ground objects; S2, data set construction: generating a low-resolution sequence {ILR1, ILR2,..., ILR5} from multi-resolution space-borne SAR images, keeping phase continuity through complex domain bilinear interpolation, and expanding data through rotation, flipping and mirroring operations for data augmentation; S3, constructing a deep learning model and dividing a training area and a verification area to train the model; specifically, the method for constructing the deep learning model is as follows: First, establish a compound loss function: ， wherein is a super-resolution output image, is a target image, is a feature map obtained by a convolutional layer in the jth layer of the VGG network, represents a number of image channels, represents an image height, represents an image width; An inflation-residual convolutional neural network with feature reuse is adopted, which uses DR-CNN as the main framework, constructs the FRDR-CNN network as the generation network in the super-resolution generative adversarial network, and simultaneously adopts the reconstruction algorithms bicubic, ScSR based on sparse coding, SRResNet with a comparable parameter size, DR-CNN and FRDR-CNN with feature reuse for comparison; S4, precision evaluation using high-resolution original image data; S5, obtaining high-resolution SAR products according to the deep learning model of the optimal scale.

2. The method of claim 1, wherein, Step S1 specifically comprises the following steps: S11, selecting SAR images of different scenes and different resolutions; S12, determining the image position of the scene by using the amplitude graph, and cropping the real part image and the imaginary part image to obtain a plurality of samples.

3. The method of claim 1, wherein, The low-resolution sequence in step S2 is generated through a 5-layer pyramid structure, and the scaling factor is 0.

8.

4. The method of claim 1, wherein, The specific method for precision evaluation in step S4 is as follows: The peak signal-to-noise ratio (PSNR), the structural similarity index (SSIM) and the edge preservation coefficient (EPD-ROA) are selected as objective evaluation indexes of the reconstructed image quality; Wherein, the PSNR is determined by the mean square error, and the original high-resolution image is taken as the reference to quantitatively evaluate the super-resolution reconstruction result, which is calculated as follows ， Wherein, MSE(x,y) represents the mean square error between x and y, L is the gray level of the image, and the larger the PSNR value is, the higher the similarity between the two images is; The SSIM reflects the overall image structural similarity, which is defined as follows ， Wherein, x and y represent the original image and the super-resolution reconstruction image respectively, l(x,y) represents the brightness contrast, s(x,y) represents the structural information contrast, c(x,y) represents the contrast contrast, and alpha, beta and gamma are three indexes used to adjust the proportion of brightness, structural information and contrast in the SSIM index; the closer the SSIM value is to 1, the stronger the structural similarity between the super-resolution reconstruction image and the original image is; The HD and VD calculation values of EPD-ROA reflect the edge preservation degree of the reconstructed image in the horizontal and vertical directions, which is beneficial to the recovery effect of the super-resolution reconstruction algorithm on the boundary high-frequency information, and is defined as ， where m is the total number of pixels in the selected region, , represents two adjacent pixels along horizontal and vertical directions of the reconstructed image, , represents two adjacent pixels in the original high-resolution image, , The analysis of EPD-ROA represents the total amount of edges of the test image, and the denominator represents the true total amount of edges. The closer the value is to 1, the better the edge preservation of the super-resolution reconstructed image.

5. A system based on the method according to any one of claims 1 to 4, characterized in that, It comprises: A data preprocessing module: performing radiation scaling, noise removal and amplitude graph generation; A dataset construction module: generates a low-resolution sequence and expands data; An FRDR-CNN network module: contains a complex convolution layer, a phase gradient attention module, and a joint loss function, and is used for improving high-frequency reconstruction fidelity through complex domain joint constraint; A reconstruction output module: generates a super-resolution complex SAR image.

6. A computer-readable storage medium, characterized in that: A computer program is stored on a medium, and the computer program, when executed, performs the method of any one of claims 1 to 4.

7. A computer system, characterized by: A device includes a processor and a storage medium, and the storage medium stores a computer program. The processor reads and executes the computer program from the storage medium to perform the method of any one of claims 1 to 4.

Images