Quality optimization method and device for synthetic aperture radar image

Abstract

The invention relates to the technical field of remote sensing imaging, in particular to a synthetic aperture radar image quality optimization method and device, and the method comprises the steps: extracting image features from a to-be-optimized synthetic aperture radar image based on an anomaly detection model constructed by a defocus image data set; determining a defocused image according to the image features, and calculating an abnormal image; the method comprises the steps of obtaining a defocus mask by using an abnormal image, further recovering defocus coverage information, training a target neural network by using a constructed independent and identically distributed noise image, and inputting a to-be-optimized synthetic aperture radar image into the trained target neural network to remove speckle noise. Therefore, the problems that in the prior art, in airborne synthetic aperture radar imaging, energy defocusing of a moving object body is serious, and it is difficult to obtain a high-quality low-noise image in a multi-frame averaging mode are solved.

Description

Technical Field

[0001] The present application relates to the field of remote sensing imaging technology, and in particular to a method and device for optimizing the quality of synthetic aperture radar images. Background Art

[0002] Synthetic aperture radar imaging is widely used in resource monitoring, landform mapping and other fields due to its advantages such as high resolution, strong penetration and all-weather operation. As an interferometric measurement technology, the imaging process of synthetic aperture radar uses the coherent superposition of multiple echo signals to enhance the lateral resolution of the image, but the phase fluctuation of the echo signal will bring signal-related noise, namely speckle noise. The denoising of synthetic aperture radar images has always been a research focus in the field.

[0003] In the related art, for the denoising problem of a single image, a filtering method based on local statistical information can be used, or a method that assumes that speckle noise follows a gamma distribution and solves the corresponding MAP (Maximum A Posteriori) equation can be used, or a SAR (Synthetic Aperture Rader)-BM3D (Block-matching and 3D filtering) method can be used to obtain the best performance of single image denoising; for multi-phase images, denoising can be achieved by filtering after weighted summation and spatial denoising, or by using deep learning for denoising, such as SAR-CNN (Convolutional Neural Networks), SAR2SAR, DualNoise2Noise (Dual Noise-to-Noise), etc.

[0004] However, in the related technology, the relative motion between the airborne synthetic aperture radar platform and the moving object is obvious, and there is still obvious defocus noise, which causes serious defocus and offset of the moving object's main energy, causing trouble for the identification of the moving object. Moreover, compared with the satellite-borne synthetic aperture radar image, the relatively low platform height leads to a larger offset of moving targets in adjacent frames in the same field of view, and a greater change in shadows between time frames. The traditional method of obtaining low-noise images by averaging multiple frames will lead to more obvious artifacts, which urgently needs to be improved. Summary of the invention

[0005] The present application provides a method and device for optimizing the quality of synthetic aperture radar images, so as to solve the problems in the related art that the energy defocusing of the moving object body in airborne synthetic aperture radar imaging is serious, and it is difficult to obtain high-quality and low-noise images by multi-frame averaging.

[0006] The first aspect of the present application provides a method for optimizing the quality of a synthetic aperture radar image, comprising the following steps: extracting image features from the synthetic aperture radar image to be optimized based on an anomaly detection model constructed from a defocused image data set; determining an image with defocus based on the image features, and calculating an anomaly map corresponding to the image with defocus; and obtaining a defocus mask using the anomaly map to restore information covered by the defocus based on the defocus mask.

[0007] Optionally, in one embodiment of the present application, it also includes: constructing an independent and identically distributed noise image; training a target neural network based on the noise image to obtain a trained target neural network; inputting the synthetic aperture radar image to be optimized into the trained target neural network to remove speckle noise and obtain a denoised image.

[0008] Optionally, in one embodiment of the present application, the noise image acquisition formula may be but is not limited to:

[0009] I l1 * =D(I l1 )+I l2 -D(I l2 ),

[0010] Among them, (Il1, Il1*) is a noise image pair, Il1, Il2 are logarithmic images in two adjacent time frames and the same field of view.

[0011] Optionally, in one embodiment of the present application, the restoring the defocused covered information according to the defocus mask includes: constructing a target memory library; determining an image block in the target memory library that is closest to the features of the synthetic aperture radar image to be optimized; and filling in information at a corresponding position of the mask based on the closest image block to obtain a defocused image.

[0012] Optionally, in one embodiment of the present application, the formula for acquiring the defocused image may be, but is not limited to,:

[0013] p ab * =p m ⊙M ab +p ab ⊙(1-M ab ),

[0014] Among them, p m For the abnormal image p in the memory bank ab The image patch with the closest feature, M ab is the defocus mask.

[0015] A second aspect of the present application provides a quality optimization device for a synthetic aperture radar image, comprising: an extraction module, used to extract image features from a synthetic aperture radar image to be optimized based on an anomaly detection model constructed from a defocused image data set; a calculation module, used to determine an image with defocus based on the image features, and calculate an anomaly map corresponding to the image with defocus; and a defocusing module, used to obtain a defocus mask using the anomaly map, so as to restore information covered by the defocus based on the defocus mask.

[0016] Optionally, in one embodiment of the present application, it also includes: a construction module for constructing an independent and identically distributed noise image; a training module for training a target neural network based on the noise image to obtain a trained target neural network; a denoising module for inputting the synthetic aperture radar image to be optimized into the trained target neural network to remove speckle noise and obtain a denoised image

[0017] Optionally, in one embodiment of the present application, the noise image acquisition formula may be but is not limited to:

[0018] I l1 * =D(I l1 )+I l2 -D(I l2 ),

[0019] Among them, (I l1 , I l1 * ) is a noise image pair, I l1 , I l2 These are logarithmic images of two adjacent time frames and the same field of view.

[0020] Optionally, in one embodiment of the present application, the defocusing module includes: a construction unit, used to construct a target memory library; a determination unit, used to determine an image block in the target memory library that is closest to the features of the synthetic aperture radar image to be optimized; and a generation unit, used to fill in information of a corresponding position of a mask based on the closest image block to obtain a defocused image.

[0021] Optionally, in one embodiment of the present application, the formula for acquiring the defocused image may be, but is not limited to,:

[0022] p ab * =p m ⊙M ab +p ab ⊙(1-M ab ),

[0023] Among them, p mFor the abnormal image p in the memory bank ab The image patch with the closest feature, M ab is the defocus mask.

[0024] A third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for optimizing the quality of synthetic aperture radar images as described in the above embodiment.

[0025] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned synthetic aperture radar image quality optimization method.

[0026] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, implements the above-mentioned method for optimizing the quality of synthetic aperture radar images.

[0027] The embodiment of the present application can extract image features from the synthetic aperture radar image to be optimized based on the anomaly detection model constructed from the defocused image data set, and then determine the image with defocus, and calculate the defocus mask to restore the information covered by the defocus. From the perspective of the image domain, an automated machine annotation algorithm is designed and an anomaly detection model is constructed to realize the discrimination and positioning of defocus. Most defocused images and segmentation labels that do not require defocus information can be automatically screened, and the information covered by defocus can be restored without additional fine-tuning training, which greatly reduces the cost of data annotation and has better versatility. As a result, technical problems in related technologies such as the serious energy defocusing of the moving object body in airborne synthetic aperture radar imaging and the difficulty in obtaining high-quality, low-noise images through multi-frame averaging are solved.

[0028] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0030] Figure 1 A flowchart of a method for optimizing the quality of a synthetic aperture radar image provided according to an embodiment of the present application;

[0031] Figure 2 A schematic diagram of the structure of a Unet (U-shaped Network) network provided according to an embodiment of the present application;

[0032] Figure 3 A flowchart of a defocusing method provided according to an embodiment of the present application;

[0033] Figure 4 A flowchart of a denoising method provided according to an embodiment of the present application;

[0034] Figure 5 This is a block diagram of an exemplary device for optimizing the quality of synthetic aperture radar images provided according to an embodiment of the present application;

[0035] Figure 6 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0037] The following describes the quality optimization method and device of the synthetic aperture radar image of the embodiment of the present application with reference to the accompanying drawings. In view of the problem that the energy defocus of the moving object in the airborne synthetic aperture radar imaging mentioned in the above background technology is serious, and it is difficult to obtain a high-quality low-noise image by multi-frame averaging, the present application provides a quality optimization method for synthetic aperture radar images. In this method, based on the anomaly detection model constructed by the defocused image data set, image features can be extracted from the synthetic aperture radar image to be optimized, and then the image with defocus is determined, and the defocus mask is calculated to restore the information covered by the defocus. From the perspective of the image domain, an automated machine annotation algorithm is designed and an anomaly detection model is constructed to realize the discrimination and positioning of defocus. Most of the defocused images and the segmentation labels that do not require defocus information can be automatically screened, and the information covered by defocus can be restored without additional fine-tuning training, which greatly reduces the cost of data annotation and has better versatility. Therefore, the technical problems in the related art that the energy defocus of the moving object in the airborne synthetic aperture radar imaging is serious, and it is difficult to obtain a high-quality low-noise image by multi-frame averaging are solved.

[0038] Specifically, Figure 1 The present invention is a flowchart of a method for optimizing the quality of a synthetic aperture radar image provided according to an embodiment of the present application.

[0039] like Figure 1 As shown, the quality optimization method of the synthetic aperture radar image includes the following steps:

[0040] In step S101 , image features are extracted from the synthetic aperture radar image to be optimized based on an anomaly detection model constructed from a defocused image dataset.

[0041] In the embodiment of the present application, the defocused image data set can be understood as training data that does not contain defocused images. Its construction may include: assuming that the video synthetic aperture radar image frame at the tth time step is I t , find the adjacent frame I t , I t-1 Residual plot R t , whose expression can be but is not limited to:

[0042] R t =I t -I t-1 ,

[0043] It should be noted that the residual image in the embodiment of the present application contains a lot of noise information. Since the brightness of the defocus information is high, the grayscale threshold T g Binarize the residual image to filter out a lot of noise information and get the binary residual mask M t , whose expression can be but is not limited to:

[0044]

[0045] Furthermore, the present embodiment calculates M t The connected domains are sorted from large to small according to their area, and all connected domains with an area greater than the threshold T are retained. a The connected domain of t The threshold value can be set by a person skilled in the art according to actual conditions, and this application does not impose any specific limitation thereto.

[0046] In the embodiment of the present application, the image containing the defocus mask is regarded as an abnormal image, and image blocks of equal size are taken from the image, and the image blocks not containing the defocus mask are screened out as the initially screened normal image set P N ={p1, p2, ...}. Furthermore, the embodiment of the present application can also manually screen the normal image set and perform secondary screening based on subjective judgment.

[0047] Furthermore, the embodiment of the present application uses a normal image set to construct an anomaly detection model. n Perform feature extraction to obtain corresponding image features, which can be expressed as but not limited to:

[0048] f n =φ(p n ),

[0049] Among them, f n For image p nThe corresponding image features.

[0050] Furthermore, in the embodiment of the present application, the normal image set P N The images in the image are extracted for features and the memory library F is constructed. N ={f1, f2, ...}.

[0051] Furthermore, in the embodiment of the present application, the same anomaly detection model is used to extract image features f for the image block p to be detected. The most similar image feature f in the memory is found. m * :

[0052]

[0053] Where S is a function that evaluates the difference between two vectors. If the difference is greater than the abnormal threshold T ab , it is classified as an abnormal image.

[0054]

[0055] Among them, C is a binary classifier. If the output is 1, it is classified as an abnormal image, that is, the image contains defocus, and if the output is 0, it is classified as a normal image, that is, the image does not contain defocus.

[0056] For example, in the embodiment of the present application, a total of 200 frames of video synthetic aperture radar reconstructed image data I∈R H ×W×200 , where H and W are the number of pixel rows and columns of the synthetic aperture radar image frame, respectively. The residuals of adjacent frames are calculated to obtain the residual map R∈R H×W×199 . Count the grayscale histogram of R, take the grayscale value at 95% of the number of grayscale pixels as the grayscale threshold G, and binarize the image to obtain M∈R H×W×199 , whose expression can be but is not limited to:

[0057]

[0058] Furthermore, the embodiment of the present application calculates the connected domains of M, sorts the connected domains from large to small according to their areas, and retains all the top 5% of the connected domains as the defocus mask M′∈R H×W×199 The image containing the defocus mask is regarded as an abnormal image. A 224×224 image block is taken from the image, and the image blocks that do not contain the defocus mask are screened out and manually screened again to obtain the normal data set P∈R 224×224×N , N represents the number of image blocks after filtering.

[0059] Furthermore, the embodiments of the present application use anomaly detection models to perform defocus anomaly detection. For example, PatchCore, this method uses a pre-trained network to extract all normal image features and construct a normal image memory library after sampling. The nearest neighbor search is used to find the features in the memory library that are closest to the image features to be detected, and then perform anomaly detection. Use the pre-trained network as a feature extractor, use the features of layer 2 and layer 3, and crop the neighborhood features with a size of 3×3. Based on the normal data set P, extract features to construct a memory library F∈R 156016×N For an image block p∈R to be detected 224×224 , use the PatchCore method to perform anomaly detection, obtain the image feature f obtained after feature extraction, and find the image feature f that is most similar to the memory library m * :

[0060]

[0061] In step S102, a defocused image is determined according to image features, and an abnormality map corresponding to the defocused image is calculated.

[0062] As a possible implementation method, the embodiment of the present application determines the presence of defocused images based on image features, and then calculates the corresponding anomaly score matrix.

[0063] For example, in the embodiment of the present application, for a defocused image p ab , calculate the anomaly score matrix s ab , whose expression can be but is not limited to:

[0064] s ab =D(f,f m * ),

[0065] Here, D is a function that evaluates the difference of elements at the same position in two vectors.

[0066] Furthermore, the embodiment of the present application scales the anomaly score matrix to the original image block size, that is, p ab Abnormal graph of m ab .

[0067] In step S103, a defocus mask is obtained using the anomaly map, so as to restore the defocus covered information according to the defocus mask.

[0068] As a possible implementation method, the embodiment of the present application obtains a defocus mask based on the anomaly map, and restores the defocus covered information according to the defocus mask to obtain a defocused image.

[0069] For example, the embodiment of the present application can obtain p abAbnormal graph of m ab , with T map The defocus mask M is obtained by thresholding ab , and then restore the defocus coverage information, where the expression of the defocus mask can be but is not limited to:

[0070]

[0071] Based on the characteristics of large deviations and irregular shapes of adjacent frames of synthetic aperture radar defocus, the embodiment of the present application designs an automatic screening and labeling method, and introduces a semi-supervised anomaly detection model to distinguish and locate defocus. It can automatically screen most defocused images and does not require segmentation labels for defocus information, greatly reducing the cost of data labeling. The anomaly detection model can be used directly, and the information covered by defocus can be detected and restored without additional fine-tuning training.

[0072] Optionally, in one embodiment of the present application, the defocused information is restored according to the defocus mask, including: constructing a target memory library; determining the image block in the target memory library that is closest to the feature of the synthetic aperture radar image to be optimized; and filling the information of the corresponding position of the mask based on the closest image block to obtain a defocused image. The formula for obtaining the defocused image can be, but is not limited to,:

[0073] p ab * =p m ⊙M ab +p ab ⊙(1-M ab ),

[0074] Among them, p m For the abnormal image p in the memory bank ab The image patch with the closest feature, M ab is the defocus mask.

[0075] In some embodiments, the embodiment of the present application determines the image block in the target memory bank that is closest to the feature of the synthetic aperture radar image to be optimized, restores the information covered by the defocus, fills the information of the corresponding position of the mask, and then obtains the defocused image p ab * , where the formula for obtaining the defocused image can be but is not limited to:

[0076] p ab * =p m ⊙M ab +p ab ⊙(1-M ab ),

[0077] Among them, p mFor the abnormal image p in the memory bank ab The image patch with the closest feature, M ab is the defocus mask.

[0078] Optionally, in one embodiment of the present application, it further includes: constructing an independent and identically distributed noise image; training a target neural network based on the noise image to obtain a trained target neural network; inputting the synthetic aperture radar image to be optimized into the trained target neural network to remove speckle noise and obtain a denoised image. The noise image acquisition formula may be, but is not limited to,:

[0079] I l1 * =D(I l1 )+I l2 -D(I l2 ),

[0080] Among them, (I l1 , I l1 * ) is a noise image pair, I l1 , I l2 These are logarithmic images of two adjacent time frames and the same field of view.

[0081] It can be understood that the target neural network image in the embodiment of the present application can be a Noise2Noise network or other neural networks. The specific setting can be made by technicians in this field according to actual conditions, and this application does not make any specific limitations.

[0082] In the actual implementation process, the embodiment of the present application can use the Noise2Noise network trained by independent and identically distributed noise images to denoise the optimized synthetic aperture radar image, remove the speckle noise, and then obtain a denoised image.

[0083] Exemplarily, the embodiment of the present application constructs an independent and identically distributed noise image to train the Noise2Noise network, and denoises the optimized synthetic aperture radar image to remove speckle noise. The main process is:

[0084] First, in the embodiment of the present application, the pixel values ​​of the optimized synthetic aperture radar image are taken logarithmically, and then the logarithmic image I is obtained. l , and converts the multiplicative noise into additive noise.

[0085] I=w⊙v,

[0086] I l =ln(w)+ln(v),

[0087] Where w is the actual signal and v is the speckle noise.

[0088] Furthermore, in the embodiment of the present application, if there are two adjacent time frames and logarithmic images I in the same field of view, l1 , I l2 , use the existing denoiser D to perform denoising, and obtain two low-noise images respectively, and then use the original image to subtract the corresponding low-noise image to obtain the noise in the adjacent time frame and the same field of view, and then add the noise to the original time frame low-noise image to obtain the denoised image pair (I l1 , I l1 * ).

[0089] I l1 * =D(I l1 )+I l2 -D(I l2 ),

[0090] Furthermore, the embodiment of the present application uses the constructed noise image to train the Noise2Noise network, and during the training process, methods such as downsampling can be used to reduce the correlation between noise pixels. The specific settings can be made by technicians in this field according to actual conditions, and this application does not make any specific restrictions.

[0091] Exemplarily, the embodiment of the present application calculates the logarithmic image of the defocused image to obtain I f ∈R H×W×200 , use the existing denoising method to denoise first, for example, use the ZS-N2N (Zero-shot Noise2Noise, zero sample noise to noise) method to denoise the logarithmic image. Other methods can also be used, which can be set by technical personnel in this field according to actual conditions, and this application does not make specific restrictions. Among them, ZS-N2N is a deep learning denoising method for zero-sample learning. By designing a diagonal convolution operator to downsample the image to construct an approximately independent and identically distributed noise image pair, the image pair is used to train the neural network, and the consistency loss function is designed to ensure that the neural network can process the original image before downsampling, thereby denoising the original image. In the embodiment of the present application, the model used is a three-layer convolutional layer. The first two layers will also pass through an additional LeakyReLU activation layer, and the last convolutional layer outputs a matrix of the same size as the input image. Use this method to I f Denoise to obtain a low-noise image I d .

[0092] Furthermore, the embodiment of the present application constructs a noise image pair dataset, taking the tth image and its adjacent frames The corresponding low-noise image is Then construct the noise image pair

[0093]

[0094] Similarly, the embodiment of the present application can also select adjacent frames To construct a noise image pair, two noise samples can be constructed for each image. f All frames construct a noise image pair dataset and get I n ∈R H×W×398 .

[0095] Furthermore, the embodiment of the present application uses the constructed noise image pair to train the neural network, which is based on the Noise2Noise network and uses the Unet network structure, such as Figure 2 As shown in the figure, the loss function is the L2 loss function. During the training process, random downsampling is used to reduce the correlation between image pixels. The number of downsampling in each round decreases linearly to 0 with the number of training times. After random downsampling, 10 image blocks of 256×256 are randomly cropped as network input, which meets the default input size of the network.

[0096] Furthermore, in the embodiment of the present application, when using a trained network, the synthetic aperture radar image to be optimized is divided into image blocks of the same size as the network input, and the division principle is that the overlapping area between the image blocks is greater than or equal to 50%, and the image blocks are used as input, and then denoising is performed. After the denoising is completed, the image blocks are spliced ​​according to the original image positions, and the overlapping parts are averaged to obtain the denoised synthetic aperture radar image, that is, the noise image.

[0097] The embodiment of the present application does not rely on high-quality low-noise images, and has better versatility by constructing approximately independent and identically distributed noise images to train the denoising network.

[0098] Combine the following Figure 3 and Figure 4 , a specific example is used to explain the quality optimization method of synthetic aperture radar image proposed in the embodiment of the present application.

[0099] in, Figure 3 The present invention is a flowchart of a defocusing method provided according to an embodiment of the present application.

[0100] Figure 4 The present invention is a flowchart of a denoising method provided according to an embodiment of the present application.

[0101] like Figure 3 As shown, the defocusing process of the embodiment of the present application is as follows:

[0102] Step S301: Obtain the video synthetic aperture radar image frame at the tth time step as I t , and adjacent frame I t .

[0103] Step S302: Calculate the residual map.

[0104] Step S303: binarize the residual image to obtain a residual mask.

[0105] Step S304: Calculate the connected domain and obtain the defocus mask.

[0106] Step S305: Acquire a normal image set.

[0107] The embodiment of the present application selects the image blocks that do not contain the defocus mask as the initially selected normal image set P N ={p1, p2, ...}. Furthermore, the embodiment of the present application can also manually screen the normal image set and perform secondary screening based on subjective judgment.

[0108] Step S306: perform feature extraction and calculate image features.

[0109] Among them, the embodiment of the present application uses a feature extractor φ to extract p n Perform feature extraction to obtain corresponding image features.

[0110] Step S307: construct a memory library.

[0111] Among them, the embodiment of the present application is for the normal image set P N The images in the image are extracted for features and the memory library F is constructed. N ={f1, f2, ...}.

[0112] Step S308: Acquire the image features to be detected.

[0113] In this embodiment of the present application, the same anomaly detection model is used to extract image features for the image block p to be detected.

[0114] Step S309: abnormality detection.

[0115] Among them, the embodiment of the present application searches for the image features to be detected that are most similar to the image features in the memory library, and then performs anomaly detection.

[0116] Step S310: Obtain an abnormality map.

[0117] Among them, the embodiment of the present application calculates an anomaly score matrix for an image with defocus, and scales the anomaly score matrix to the size of the original image block to obtain an anomaly map.

[0118] Step S311: Calculate the defocus mask.

[0119] Among them, the embodiment of the present application is based on T map The defocus mask M is obtained by thresholding ab, and then restore the information covered by the defocus to obtain the defocused image.

[0120] Step S312: obtaining a defocused image.

[0121] like Figure 4 As shown, the denoising process of the embodiment of the present application is:

[0122] Step S401: obtaining a first logarithmic image of a synthetic aperture radar image to be optimized.

[0123] In this embodiment of the present application, for the optimized synthetic aperture radar image, the pixel value is logarithmic, thereby obtaining a logarithmic image.

[0124] Step S402: obtaining a second logarithmic image of the synthetic aperture radar image to be optimized.

[0125] In this embodiment of the present application, logarithmic image 1 and logarithmic image 2 are images of adjacent time frames and the same field of view.

[0126] Step S403: Denoiser.

[0127] Among them, the embodiment of the present application uses a denoiser to obtain a denoised image pair.

[0128] Step S404: Noise image pair.

[0129] Step S405: construct a noise image pair data set.

[0130] Step S406: training the Noise2Noise network.

[0131] According to the quality optimization method of synthetic aperture radar images proposed in the embodiment of the present application, image features can be extracted from the synthetic aperture radar images to be optimized based on the anomaly detection model constructed from the defocused image data set, and then the image with defocus can be determined, and the defocus mask can be calculated to restore the information covered by the defocus. From the perspective of the image domain, an automated machine annotation algorithm is designed and an anomaly detection model is constructed to realize the discrimination and positioning of defocus. Most defocused images and segmentation labels that do not require defocus information can be automatically screened, and the information covered by defocus can be restored without additional fine-tuning training, which greatly reduces the cost of data annotation and has better versatility. As a result, technical problems in related technologies such as the serious energy defocusing of the moving object body in airborne synthetic aperture radar imaging and the difficulty in obtaining high-quality, low-noise images through multi-frame averaging are solved.

[0132] Next, a device for optimizing the quality of synthetic aperture radar images according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0133] Figure 5A block diagram of a synthetic aperture radar image quality optimization device provided according to an embodiment of the present application.

[0134] like Figure 5 As shown, the synthetic aperture radar image quality optimization device 10 includes: an extraction module 100, a calculation module 200 and a defocusing module 300.

[0135] The extraction module 100 is used to extract image features from the synthetic aperture radar image to be optimized based on an anomaly detection model constructed from a defocused image data set.

[0136] The calculation module 200 is used to determine the defocused image according to the image features, and calculate the anomaly map corresponding to the defocused image.

[0137] The defocusing module 300 is used to obtain a defocus mask using the anomaly map, so as to restore the defocus covered information according to the defocus mask.

[0138] Optionally, in one embodiment of the present application, it also includes: a construction module, a training module and a denoising module.

[0139] Among them, the construction module is used to construct independent and identically distributed noise images.

[0140] The training module is used to train the target neural network based on the noise image to obtain a trained target neural network.

[0141] The denoising module is used to input the synthetic aperture radar image to be optimized into the trained target neural network to remove speckle noise and obtain a denoised image.

[0142] Optionally, in one embodiment of the present application, the noise image acquisition formula may be, but is not limited to,:

[0143] I l1 * =D(I l1 )+I l2 -D(I l2 ),

[0144] Among them, (I l1 , I l1 * ) is a noise image pair, I l1 , I l2 These are logarithmic images of two adjacent time frames and the same field of view.

[0145] Optionally, in one embodiment of the present application, the defocusing module 300 includes: a construction unit, a determination unit and a generation unit.

[0146] Among them, the construction unit is used to construct the target memory library.

[0147] The determination unit is used to determine the image block in the target memory bank that is closest to the feature of the synthetic aperture radar image to be optimized.

[0148] The generating unit is used to fill the information of the corresponding position of the mask based on the closest image block to obtain a defocused image.

[0149] Optionally, in one embodiment of the present application, a formula for obtaining a defocused image may be, but is not limited to,:

[0150] p ab * =p m ⊙M ab +p ab ⊙(1-M ab ),

[0151] Among them, p m For the abnormal image p in the memory bank ab The image patch with the closest feature, M ab is the defocus mask.

[0152] It should be noted that the above explanation of the embodiment of the synthetic aperture radar image quality optimization method is also applicable to the synthetic aperture radar image quality optimization device of this embodiment, and will not be repeated here.

[0153] According to the quality optimization device for synthetic aperture radar images proposed in the embodiment of the present application, image features can be extracted from the synthetic aperture radar image to be optimized based on the anomaly detection model constructed from the defocused image data set, and then the image with defocus can be determined, and the defocus mask can be calculated to restore the information covered by the defocus. From the perspective of the image domain, an automated machine annotation algorithm is designed and an anomaly detection model is constructed to realize the discrimination and positioning of defocus. Most defocused images and segmentation labels that do not require defocus information can be automatically screened, and the information covered by defocus can be restored without additional fine-tuning training, which greatly reduces the cost of data annotation and has better versatility. As a result, technical problems in related technologies such as the serious energy defocusing of the moving object body in airborne synthetic aperture radar imaging and the difficulty in obtaining high-quality, low-noise images through multi-frame averaging are solved.

[0154] Figure 6 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. The electronic device may include:

[0155] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .

[0156] When the processor 602 executes the program, the quality optimization method of the synthetic aperture radar image provided in the above embodiment is implemented.

[0157] Furthermore, the electronic device further comprises:

[0158] The communication interface 603 is used for communication between the memory 601 and the processor 602 .

[0159] The memory 601 is used to store computer programs that can be executed on the processor 602 .

[0160] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0161] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0162] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.

[0163] The processor 602 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0164] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for optimizing the quality of synthetic aperture radar images.

[0165] An embodiment of the present application further provides a computer program product, including a computer program, which implements the above synthetic aperture radar image quality optimization method when executed.

[0166] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0167] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0168] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0169] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0170] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0171] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0172] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0173] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for optimizing the quality of synthetic aperture radar images, characterized in that: The following steps are involved: Based on the anomaly detection model constructed from the defocused image dataset, image features are extracted from the synthetic aperture radar image to be optimized; Determine a defocused image according to image features, and calculate an anomaly map corresponding to the defocused image; A defocus mask is obtained by using the anomaly map, so as to restore the defocus coverage information according to the defocus mask.

2. The method according to claim 1, characterized in that: Also includes: Construct independent and identically distributed noise images; Training a target neural network based on the noise image to obtain a trained target neural network; The synthetic aperture radar image to be optimized is input into the trained target neural network to remove speckle noise and obtain a denoised image.

3. The method according to claim 2, characterized in that The acquisition formula of the noise image is: I l1 * =D(I l1 )+I l2 -D(I l2 ), Among them, (I l1 ,I l1 * ) is a noise image pair, I l1 , I l2 These are logarithmic images of two adjacent time frames and the same field of view.

4. The method according to claim 1, characterized in that The restoring the defocus covered information according to the defocus mask comprises: Build a target memory bank; Determine an image block in the target memory bank that is closest to the features of the synthetic aperture radar image to be optimized; Based on the closest image block, information of the corresponding position of the mask is filled in to obtain a defocused image.

5. The method according to claim 4, characterized in that The acquisition formula of the defocused image is: p ab * =p m ⊙M ab +p ab ⊙(1-M ab ), Among them, p m For the abnormal image p in the memory bank ab The image patch with the closest feature, M ab is the defocus mask.

6. A quality optimization device for synthetic aperture radar images, characterized in that: include: An extraction module, for extracting image features from the synthetic aperture radar image to be optimized based on an anomaly detection model constructed from a defocused image dataset; A calculation module, used for determining a defocused image according to image features, and calculating an abnormality map corresponding to the defocused image; A defocusing module is used to obtain a defocus mask using the abnormal image, so as to restore the defocus covered information according to the defocus mask.

7. The device according to claim 6, characterized in that Also includes: A construction module for constructing independent and identically distributed noise images; A training module, used for training a target neural network based on the noise image to obtain a trained target neural network; The denoising module is used to input the synthetic aperture radar image to be optimized into the trained target neural network to remove speckle noise and obtain a denoised image.

8. The device according to claim 7, characterized in that The acquisition formula of the noise image is: I l1 * =D(I l1 )+I l2 -D(I l2 ), Among them, (I l1 ,I l1 * ) is a noise image pair, I l1 , I l2 These are logarithmic images of two adjacent time frames and the same field of view.

9. The device according to claim 6, characterized in that The defocusing module comprises: Construction unit, used to construct the target memory bank; A determination unit, used for determining an image block in a target memory bank that is closest to the features of the synthetic aperture radar image to be optimized; A generating unit is used to fill in the information of the corresponding position of the mask based on the closest image block to obtain a defocused image.

10. The device according to claim 9, characterized in that The acquisition formula of the defocused image is: p ab * =p m ⊙M ab +p ab ⊙(1-M ab ), Among them, p m For the abnormal image p in the memory bank ab The image patch with the closest feature, M ab is the defocus mask.

Images