Complex domain SAR image super-resolution reconstruction method and system

Abstract

The invention discloses a complex field SAR (Synthetic Aperture Radar) image super-resolution reconstruction method and system. The method comprises the following steps: S1, preprocessing complex field data; s2, performing data set construction and data augmentation; s3, constructing a deep learning model, dividing the deep learning model into a training area and a verification area, and performing model training; s4, performing precision evaluation by using the high-resolution original image data; and S5, obtaining a high-resolution SAR product according to the deep learning model for training the optimal scale. According to the scheme, under the condition that the processing capacity of hardware is improved, the result superior to that of super-division reconstruction only depending on the amplitude diagram is obtained, more surface feature details can be recovered, and the application value of existing image data in the field of natural disasters and the recognition capacity of military targets are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of synthetic aperture radar (SAR) image processing, in particular to a complex domain SAR image super-resolution reconstruction method and system. BACKGROUND

[0002] Synthetic aperture radar (SAR) has the characteristics of all-weather and all-day, so that SAR images are widely used in various fields, such as target detection, ground object classification and change detection. As an important indicator for evaluating the quality of SAR images, the higher the resolution of the image, the more semantic information it carries, and in subsequent advanced visual tasks, it can produce better processing results.

[0003] There are usually three ways to improve the resolution of SAR images: one is to improve and update the hardware, such as increasing the system transmission bandwidth and synthesizing a large aperture to improve the system theoretical resolution; the second is to process the original echo data collected by the SAR imaging system in the imaging stage, such as modern spectrum estimation method and data extrapolation method, but they are strongly dependent on the signal model and have a very large amount of calculation; the third is the image super-resolution reconstruction (SR) method, that is, using image processing methods on the SAR image after imaging, so as to realize the resolution improvement. SAR complex domain imaging breaks through the bottleneck of traditional real value imaging by using the full information dimension of "amplitude + phase", and provides more accurate results for high-noise environment, deep tissue observation and large-scale dynamic scene.

[0004] The current bottlenecks of SAR image super-resolution technology are as follows:

[0005] (1) The traditional real number domain processing method loses the phase information (experimental data shows that the phase error leads to about 23% of the detail loss);

[0006] (2) The existing complex domain method ignores the gradient consistency (comparative experiments show that the gradient mismatch will make the PSNR drop by 2-4 dB). SUMMARY

[0007] In order to solve the existing problems, the present application provides a complex domain SAR image super-resolution reconstruction method and system, and the specific scheme is as follows:

[0008] A complex domain SAR image super-resolution reconstruction method, comprising the following steps:

[0009] S1, pre-processing of complex domain data: inputting the SAR single view complex image, performing radiation scaling and thermal noise removal on the real part and imaginary part data to generate an amplitude image to identify ground objects;

[0010] S2, data set construction: generate low resolution sequence {ILR1, ILR2,..., ILR5} from multi-resolution space-borne SAR image, keep phase continuity through complex domain bilinear interpolation, expand data through rotation, flip and mirror operation, and combine data augmentation;

[0011] S3, construct deep learning model, divide training area and verification area, and train model;

[0012] S4, use high resolution original image data to evaluate precision;

[0013] S5, obtain high resolution SAR product according to deep learning model of optimal scale.

[0014] Preferably, step S1 specifically comprises the following steps:

[0015] S11, select SAR images of different scenes and different resolutions;

[0016] S12, determine image position of the scene by using amplitude graph, and crop real part image and imaginary part image to obtain a plurality of samples.

[0017] Preferably, the low resolution sequence in step S2 is generated through a 5-layer pyramid structure, and the scaling factor is 0.8.

[0018] Preferably, the method for constructing deep learning model in step S3 is as follows:

[0019] First, establish a composite loss function:

[0020] ,

[0021] Wherein is the super-resolution output image, is the target image, is the feature map obtained by the jth convolution layer in the VGG network layer; represents the number of image channels, represents the height of the image, represents the width of the image.

[0022] An inflation-residual convolutional neural network with feature reuse is adopted, which uses DF-CNN as the main framework, constructs FRFR-CNN network as the generation network in the super-resolution generative adversarial network, and adopts reconstruction algorithms bicubic, ScSR based on sparse coding, SRResNet with the same parameter scale, DR-CNN with inflation-residual convolution, and FRDR-CNN after feature reuse for comparison.

[0023] Preferably, the specific method of precision evaluation in step S4 is as follows:

[0024] The peak signal-to-noise ratio (PSNR), the structural similarity index (SSIM) and the edge preservation distortion-robustness analysis (EPD-ROA) are selected as the objective evaluation indexes of the reconstructed image quality.

[0025] The PSNR is determined by the mean square error (MSE), and the original high-resolution image is taken as the reference to quantitatively evaluate the result of the super-resolution reconstruction, and the calculation is as shown below

[0026] ,

[0027] The MSE (x, y) represents the mean square error between the two images of x and y, L is the gray level of the image, and the larger the PSNR value is, the higher the similarity of the two images is.

[0028] The SSIM reflects the overall structural similarity of the image, and the definition is as shown below

[0029] ,

[0030] The x and y represent the original image and the super-resolution reconstructed image respectively, l (x, y) represents the brightness contrast, s (x, y) represents the structural information contrast, c (x, y) represents the contrast contrast, and the three indexes of alpha, beta and gamma are used to adjust the proportion of the brightness, the structural information and the contrast in the SSIM index; the closer the SSIM value is to 1, the stronger the structural similarity of the super-resolution reconstructed image and the original image is.

[0031] The HD and VD calculation values of the EPD-ROA reflect the edge preservation degree of the reconstructed image in the horizontal and vertical directions, and are beneficial to the recovery effect of the super-resolution reconstruction algorithm on the high-frequency information of the boundary, and the definition is as follows

[0032] ,

[0033] The m is the total number of pixels of the selected area, , The two adjacent pixels of the reconstructed image along the horizontal direction and the vertical direction are represented by u and v respectively, , The two adjacent pixels of the original high-resolution image are represented by u' and v' respectively, , The analysis of the EPD-ROA represents the total amount of edges of the test image, and the denominator represents the total amount of true edges, and the closer the value is to 1, the better the edge preservation of the super-resolution reconstructed image is.

[0034] The application also discloses a system based on any of the above-mentioned methods, which comprises:

[0035] Data preprocessing module: perform radiation calibration, noise removal and amplitude map generation;

[0036] Data set construction module: generate low resolution sequence and expand data;

[0037] FRDR-CNN network module: containing complex convolution layer, phase gradient attention module and joint loss function, used to improve high frequency reconstruction fidelity by complex domain joint constraint;

[0038] Reconstruction output module: generate super-resolution complex SAR image.

[0039] The application also discloses a computer readable storage medium, and the computer readable storage medium has a computer program stored thereon.

[0040] The application also discloses a computer system, which comprises a processor and a storage medium, and the storage medium has a computer program stored thereon.

[0041] The application has the following beneficial effects:

[0042] The scheme mainly adopts the feature multiplexing dilated residual convolutional neural network, constructs the deep learning data set based on the complex domain SAR image, constructs the joint loss function of the phase gradient smoothing term and the amplitude edge alignment term, forces the reconstruction result to comply with the electromagnetic scattering physical law, and suppresses the generation of artifacts, simultaneously introduces the complex convolution layer to directly extract the features in the real part / imaginary part joint space, and utilizes the phase gradient information to guide the attention mechanism to focus on the key scattering area, and improves the high frequency component reconstruction fidelity.

[0043] The scheme can obtain a result superior to the amplitude map super-resolution reconstruction, can restore more ground object details, and improves the application value in the natural disaster field and the identification ability of the military target of the existing image data. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0045] Figure 1 : the first row is a sample example diagram of low resolution data, and the second row is a sample example diagram of high resolution data;

[0046] Figure 2 : DR residual unit structure diagram;

[0047] Figure 3 : SAR super-resolution deep learning architecture;

[0048] Figure 4 : super-resolution result map;

[0049] Figure 5 : method principle block diagram of the application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are some but not all of the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the application.

[0051] SAR amplitude map super-resolution only needs to process amplitude information, and the algorithm complexity is much lower than that of complex domain image, but ignoring the phase data leads to the inability to reconstruct key physical quantities such as ground three-dimensional deformation and micro-displacement, which limits its application in high-precision scenes such as settlement monitoring and geological activity analysis.

[0052] The application constructs a joint loss function of a phase gradient smoothing term and an amplitude edge alignment term, forces the reconstruction result to comply with the electromagnetic scattering physical law, and suppresses the generation of artifacts; meanwhile, a complex convolution layer is introduced to extract features in the real part / imaginary part joint space, and the phase gradient information is used to guide the attention mechanism to focus on the key scattering area, and the high-frequency component reconstruction fidelity is improved.

[0053] As Figures 1 to 5 A complex domain SAR image super-resolution reconstruction method, comprising the following steps:

[0054] S1, pre-processing of complex domain data: inputting SAR single-view complex image, performing radiation calibration and thermal noise removal on real part and imaginary part data, and generating amplitude image to identify ground objects.

[0055] S11, select SAR images of different scenes and different resolutions, including roads, farmland, buildings, trees and the like;

[0056] S12, determine the image position of the scene by using the amplitude map, and crop the real part image and the imaginary part image to obtain a plurality of samples.

[0057] S2, data set construction: the data uses different resolution multi-scene satellite SAR images of the same area and similar shooting time, first establishes a 5-layer pyramid with a scaling factor of 0.8 to generate a low-resolution sequence {ILR1, ILR2,..., ILR5}, and each layer is sampled by complex domain bilinear interpolation to maintain phase continuity. Referring to the amplitude map, the relevant scene images are manually selected and cropped, and the data is augmented by combining 0, 90, 180 and 270° image rotation, image flipping and image mirroring operations;

[0058] S3, constructing a deep learning model, dividing the generated data set into a training set and a validation set in a 7:3 ratio for model training.

[0059] The method for constructing the deep learning model is as follows:

[0060] First, establish a compound loss function:

[0061] ,

[0062] wherein is the super-resolution output image, is the target image, is the feature map obtained by the jth convolution layer in the VGG network layer; represents the number of image channels, represents the image height, represents the image width.

[0063] An inflation-residual convolutional neural network with feature reuse is adopted, which uses DF-CNN as the main framework, constructs FRFR-CNN network as the generation network in the super-resolution generative adversarial network, and uses bicubic interpolation, sparse coding-based reconstruction algorithm ScSR, SRResNet with the same parameter size, inflation-residual convolution DR-CNN and FRDR-CNN after feature reuse for comparison.

[0064] S4, precision evaluation using high-resolution original image data;

[0065] S5, obtaining high-resolution SAR products according to the deep learning model of the optimal scale.

[0066] The specific method for precision evaluation is as follows:

[0067] The peak signal-to-noise ratio PSNR, the structural similarity index SSIM and the edge preservation coefficient EPD-ROA are selected as objective evaluation indexes of the reconstructed image quality;

[0068] Wherein, PSNR is determined by mean square error, taking original high resolution image as a benchmark, quantitatively evaluating the result after super resolution reconstruction, and the calculation is as shown below

[0069]

[0070] Wherein, MSE(x,y) represents the mean square error between x and y two images, L is the gray level of the image, and the larger the PSNR value is, the higher the similarity of the two images is;

[0071] SSIM reflects the overall image structure similarity, and the definition is as shown below

[0072]

[0073] Wherein, x and y respectively represent the original image and the super resolution reconstructed image, l(x,y) represents the brightness contrast, s(x,y) represents the structure information contrast, c(x,y) represents the contrast contrast, and α, β and γ are three indexes for adjusting the proportion of brightness, structure information and contrast in the SSIM index; the closer the SSIM value is to 1, the stronger the structure similarity of the super resolution reconstructed image and the original image is;

[0074] The HD and VD calculation values of EPD-ROA reflect the edge preservation degree of the reconstructed image in the horizontal and vertical directions, and are beneficial to the recovery effect of the super resolution reconstruction algorithm on the boundary high frequency information, and the definition is

[0075]

[0076] Wherein, m is the total number of pixels of the selected area, Represent two adjacent pixels of the reconstructed image along the horizontal direction and the vertical direction, Represent two adjacent pixels of the original high resolution image, The analysis of EPD-ROA represents the total amount of edges of the test image, and the denominator represents the true total amount of edges, and the closer the value is to 1, the better the edge preservation of the super resolution reconstructed image is.

[0077] The application also discloses a system based on any of the above-mentioned methods, comprising:

[0078] 1. Data preprocessing module: for performing complex domain radiation calibration, eliminating SAR system gain error, such as distance antenna pattern distortion; thermal noise removal, directly removing the thermal noise of the receiver, especially for weak scattering area; generating amplitude image, providing ground object positioning basis for cropping.

[0079] ​​​​​​2. Dataset Construction Module: This module constructs {ILR1-ILR5} using a five-layer pyramid (scaling factor 0.8). It also performs bilinear interpolation in the complex domain to maintain phase continuity and avoid phase jumps caused by traditional downsampling. It also performs 0° / 90° / 180° / 270° rotation, flipping, and mirroring to expand the data. This module breaks through the limitations of traditional real-domain augmentation by performing rotation and mirroring operations simultaneously on the real and imaginary parts, in accordance with the physical laws of electromagnetic scattering.

[0080] 3. FRDR-CNN network module: includes complex convolutional layers, phase gradient attention modules, and joint loss functions. The complex convolutional layers are used to extract features in the real / imaginary joint space, rather than the traditional amplitude map input. The phase gradient attention mechanism uses phase gradients to generate attention masks and focus on strong scatterer areas. In the joint loss function calculation, users simultaneously optimize the VGG feature loss, phase gradient smoothing constraints, and amplitude edge alignment constraints to address the gradient mismatch problem (PSNR drops by 2-4dB), and improve the high-frequency reconstruction fidelity through complex domain joint constraints.

[0081] 4. Reconstruction output module: Generates super-resolution complex SAR images and outputs high-resolution amplitude and phase maps simultaneously, supporting military reconnaissance / disaster monitoring applications. The output format is compatible with professional processing platforms such as ENVI / SNAP.

[0082] This scheme mainly adopts a feature-reused dilated-residual convolutional neural network to construct a deep learning dataset based on complex domain SAR images. By constructing a joint loss function of phase gradient smoothing terms and amplitude edge alignment terms, the reconstruction results are forced to conform to the physical laws of electromagnetic scattering and suppress artifact generation. At the same time, a complex convolutional layer is introduced to directly extract features in the real / imaginary part joint space, and the phase gradient information is used to guide the attention mechanism to focus on key scattering areas, thereby improving the fidelity of high-frequency component reconstruction.

[0083] By improving the processing power of the hardware, this solution can obtain results that are superior to those relying solely on amplitude map super-resolution reconstruction, restore more ground object details, and enhance the application value of existing image data in the field of natural disasters and the ability to identify military targets.

[0084] The present invention also discloses a computer-readable storage medium and a computer system, wherein the computer-readable storage medium stores a computer program, which, when executed, performs the above-described method. A computer system includes a processor and a storage medium, wherein the storage medium stores the computer program, and the processor reads and executes the computer program from the storage medium to perform the above-described method.

[0085] Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0086] In one or more exemplary embodiments, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0087] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0088] Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood that modifications can be made to the foregoing embodiments, or additional implementations can be implemented, without departing from the spirit and scope of the inventive subject matter. Accordingly, the present application is not limited to the implementations described herein, but is intended to be defined by the claims set forth below, and equivalents thereof.

Claims

1. A complex domain SAR image super-resolution reconstruction method, characterized by: The following steps are involved: S1, preprocessing of complex domain data: inputting SAR single-view complex image, performing radiometric calibration and thermal noise removal on the real and imaginary data, generating amplitude image to identify ground objects; S2, dataset construction: multi-resolution spaceborne SAR images are used to generate low-resolution sequences {ILR1, ILR2,..., ILR5}, phase continuity is maintained through complex domain bilinear interpolation, and data augmentation is performed by combining rotation, flipping, and mirroring operations to expand the data; S3: Build a deep learning model and divide it into training and validation areas for model training. S4, accuracy evaluation using high-resolution raw image data; S5, obtains high-resolution SAR products based on the deep learning model trained at the optimal scale.

2. The method according to claim 1, characterized in that Step S1 specifically includes the following steps: S11, select SAR images of different resolutions for different scenes; S12, using the amplitude map to determine the image position of the scene, and cropping the real image and the imaginary image to obtain a number of samples.

3. The method according to claim 1, characterized in that The low-resolution sequence in step S2 is generated by a 5-layer pyramid structure with a scaling factor of 0.

8.

4. The method according to claim 1, wherein The method for constructing the deep learning model in step S3 is as follows: First, establish the composite loss function: ， in is the super-resolution output image, is the target image, is the feature map obtained by the j-th convolutional layer in the VGG network layer, Represents the number of image channels, Represents the image height, Represents the image width; A feature-reused dilated-residual convolutional neural network is adopted. The network uses DF-CNN as the main framework to construct the FRFR-CNN network as the generation network in the super-resolution generative adversarial network. At the same time, the reconstruction algorithm bicubic interpolation, the sparse coding-based reconstruction algorithm ScSR, and the SRResNet with similar parameter scale are compared with the dilated-residual convolution DR-CNN and the FRDR-CNN after feature reuse.

5. The method according to claim 1, wherein The specific method for performing accuracy evaluation in step S4 is as follows: Peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and edge preservation coefficient (EPD-ROA) were selected as objective evaluation indicators of reconstructed image quality. Among them, PSNR is determined by the mean square error. Taking the original high-resolution image as the benchmark, the results of super-resolution reconstruction are quantitatively evaluated. The calculation is as follows ， Where MSE(x,y) represents the mean square error between the two images x and y, L is the grayscale of the image, and the larger the PSNR value, the higher the similarity between the two images. SSIM reflects the overall image structural similarity and is defined as follows: ， Among them, x, y represent the original image and the super-resolution reconstructed image respectively, l(x, y) represents the brightness contrast, s(x, y) represents the structural information contrast, c(x, y) represents the contrast contrast, and the three indices α, β and γ are used to adjust the proportion of brightness, structural information and contrast in the SSIM index; the closer the SSIM value is to 1, the stronger the structural similarity between the super-resolution reconstructed image and the original image; The HD and VD calculated values ​​of EPD-ROA reflect the edge preservation of the reconstructed image in the horizontal and vertical directions, which is beneficial to the super-resolution reconstruction algorithm's response to high-frequency information such as boundaries. It is defined as ， Where m is the total number of pixels in the selected area, , Represents two adjacent pixels in the horizontal and vertical directions of the reconstructed image, , Represents the original high-resolution image with , For two adjacent pixels, the EPD-ROA analysis represents the total amount of edges in the test image, and the denominator represents the total amount of true edges. The closer its value is to 1, the better the edge preservation of the super-resolution reconstructed image.

6. A system based on the method according to any one of claims 1 to 5, characterized in that: include: Data preprocessing module: performs radiation calibration, noise removal and amplitude map generation; Dataset building module: generate low-resolution sequences and expand data; FRDR-CNN network module: includes complex convolutional layers, phase gradient attention modules, and joint loss functions, which are used to improve high-frequency reconstruction fidelity through joint constraints in the complex domain; Reconstruction output module: Generate super-resolution complex SAR images.

7. A computer-readable storage medium, characterized in that: The medium stores a computer program, and after the computer program is run, the method according to any one of claims 1 to 5 is executed.

8. A computer system, characterized in that: The method comprises a processor and a storage medium, wherein the storage medium stores a computer program, and the processor reads and runs the computer program from the storage medium to execute the method according to any one of claims 1 to 5.

Images