A SAR image denoising method based on self-supervised learning

Abstract

The application provides a SAR image denoising method based on self-supervised learning, comprising the following steps: constructing a SAR noise image model; constructing a minimum empirical loss function L; designing an adjacent co-directional subsampler; taking a noise image pair as input and target, modifying an empirical risk minimization task, and constructing a self-supervised SAR image denoising network; designing a multi-feature loss function; using the above method, training a SAR-CNN denoising network; transforming a SAR image to be denoised into a log domain and inputting the SAR image into the network for denoising, and then obtaining an output image through inverse log transformation. Compared with the prior art, the application can be migrated to any existing denoising network, realize self-supervised denoising in the absence of a clean target, significantly suppress speckle noise, reliably retain image texture details and other features, and solve the problem of poor practicability of current speckle data and real SAR image self-supervised denoising.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a SAR image denoising method based on self-supervised learning. BACKGROUND

[0002] Synthetic Aperture Radar (SAR) is one of the important sources of remote sensing big data, which can provide all-weather and all-day ground image acquisition capability. Therefore, as an effective means of earth space information acquisition, SAR images are widely used in detection, segmentation, classification and other tasks, and play an important role in national economy, geological survey, military reconnaissance and other fields. However, due to the unique imaging mechanism of SAR, SAR image information is disturbed by speckle noise, which seriously affects the visual quality of the image and the interpretability of the image. Therefore, speckle suppression is an important step in SAR image preprocessing.

[0003] In recent years, domestic and foreign scholars have carried out a large number of researches on SAR image speckle suppression, and have achieved good results. The existing SAR speckle suppression methods can be roughly summarized as filter-based methods, transform domain methods, Markov model-based methods, non-local mean (NLM) methods and deep learning methods. The most common filter-based methods are Lee filter, Frost filter and Kuan filter, which are assumed to be performed on homogeneous regions, so they usually cannot preserve the texture and edges in real SAR images. In the transform domain, wavelet-based methods can effectively separate signals and noise according to the different characteristics of images and speckle noise in the wavelet domain, but it is difficult to express high-dimensional features, and the edge and texture features will become blurred after processing. Markov random field (MRF) can provide quantitative description of image prior information by modeling the context information, but it will introduce complex prior constraints. In contrast, the non-local mean method uses the similarity between image blocks to perform weighted filtering on the entire image, which is beneficial to the preservation of details. However, the denoising effect of the non-local mean method depends largely on the selection of algorithm parameters, and it takes a considerable amount of time to process super-large size images. Many convolutional neural network (CNN) based denoising methods have been proposed and have shown excellent performance, such as DnCNN, FFDNet, and CBDNet. SAR-CNN performs homogenization through logarithmic transformation, and then maps the noise-free image back to the original domain through an exponential function to obtain the restored image, successfully applying CNN to SAR speckle suppression. Subsequently, the proposed ID-CNN and SAR-DRN methods further improve the denoising quality.

[0004] However, these deep learning-based denoising methods all focus on novel network architectures, ignoring the real problem of lacking noise-free SAR images that needs to be considered first. There are two strategies to solve this problem: using synthetic speckle data and multi-temporal SAR data. But the domain difference problem between synthetic speckle data and real SAR images seriously limits the practicality of supervised learning-based methods. Noise2Noise first tried a self-supervised denoising method, and some work also tried to apply it to SAR image speckle suppression, but the problem of multi-temporal SAR data acquisition difficulty has not been solved. SUMMARY

[0005] In view of the deficiencies in the prior art, the present application provides a SAR image denoising method based on self-supervised learning to solve the technical problem of lacking SAR image self-supervised denoising in the prior art.

[0006] The present application provides a SAR image denoising method based on self-supervised learning, comprising:

[0007] S1, acquiring a SAR image and constructing a SAR noise image model;

[0008] S2, constructing a minimum empirical loss function for training the SAR image;

[0009] S3, designing an adjacent co-directional sub-sampler, generating a noise image pair through the adjacent co-directional sub-sampler from the SAR image;

[0010] S4, taking the generated noise image pair as the input and target of network training, modifying the empirical risk minimization task, and constructing a self-supervised SAR image denoising network;

[0011] S5, designing a multi-feature loss function for preserving the texture details of the SAR image;

[0012] S6, training the SAR image denoising network based on steps S1-S5 to obtain a denoised SAR image.

[0013] Optionally, the SAR image is acquired and the SAR noise image model is constructed, comprising:

[0014] The SAR image is acquired by observation, and the SAR image includes a noise image with speckle noise and a clean image without speckle at the bottom layer, and is represented by a multiplication model as:

[0015] Y=X*N

[0016] Wherein, Y represents the noise image, X represents the clean image, x represents the element product of two matrices, N represents speckle noise, and the noise image is converted to a log domain for denoising, and the multiplicative noise is converted to additive noise:

[0017] log Y = log X + log N

[0018] And since the speckle noise distribution has a unit mean, E(log Y ) = E(log X )

[0019] Optionally, the constructed minimum empirical loss function is used for training the SAR image, comprising:

[0020] Training the denoising network f θ by the minimum empirical loss function L i , x i ), which is expressed as:

[0021]

[0022] The above formula is equivalent to:

[0023]

[0024] After the logarithmic domain transformation of the noise image and the clean image, the above formula is expressed as:

[0025]

[0026] Optionally, the adjacent co-directional sub-sampler is designed, and the SAR image is generated into a noise image pair through the adjacent co-directional sub-sampler, comprising:

[0027] S301, dividing the noise image log Y into [W / k]x[H / k] patches, wherein k is the block size;

[0028] S302, randomly extracting two pixels of the patch transversely or longitudinally, the two pixels satisfying the conditions of adjacent and same direction, and they are respectively taken as the (i, j) element of the sub-sampler G=(g1, g2);

[0029] S303, for [W / k]x[H / k] patches, repeating S302 to obtain a noise image pair {g1(log y), g2(log y)} with a size of [W / k]x[H / k].

[0030] Optionally, the generated noise image pair is modified as the input and target of network training to modify the empirical risk minimization task, and a self-supervised SAR image denoising network is constructed, comprising:

[0031] S401, sample the noise image through the adjacent co-directional sub-sampler to obtain an image pair (g1(y), g2(y)) containing the same underlying clean target, and generate a noise image pair (g1(log y), g2(log y)) from a single noise image;

[0032] S402, according to the characteristic that speckle noise has the same mean value, the distribution of the sampled image pair and the noise image pair has the same expected value, that is:

[0033] E(g1(log Y ))=E(g2(log Y ))=E(log Y )

[0034] S403, when the distribution p(y|x) of the input conditional target distribution is replaced by an arbitrary distribution with the same expected value, the optimal parameter θ of network training also remains unchanged, and therefore the empirical risk minimization task is modified as:

[0035]

[0036] Optionally, the multi-feature loss function is designed to preserve the texture details of the SAR image, comprising:

[0037] S501, in order to meet the multiplication characteristic of speckle and accelerate the convergence speed, the MSE is logarithmically scaled to obtain a first regularization term L desp , which is expressed as:

[0038]

[0039] Wherein, g1(log y) and g2(log y) are the first noise image pair and the second noise image pair generated by the sub-sampler, f θ (g1(log y)) and f θ (g2(log y)) are the first noise image and the second noise image obtained by the denoising network;

[0040] S502, a second regularization term L reg is added to compensate for the first noise image pair, the second noise image pair, the first noise image and the second noise image:

[0041] L reg =||f θ (g1(log y))-g2(log y)+g1(fθ (log y))-g2(f θ (log y))|

[0042] wherein, f θ is a denoising network model for training, g1(·) and g2(·) represent two sub-samplers respectively;

[0043] S503, adding a third regular term L per to improve the quality of the denoised image:

[0044]

[0045] wherein, represents a VGG network, represents a feature map with a size of CxHxW;

[0046] S504, adding the first regular term, the second regular term and the third regular term, and matching with a proportional coefficient, to obtain a loss function of a SAR image denoising network:

[0047] L=L desp +αL reg +βL per .

[0048] Optionally, based on steps S1-S5, the SAR image denoising network is trained to obtain a denoised SAR image, comprising:

[0049] S601, generating a noise image pair by the sub-sampler based on the SAR image, as an input and a target of the denoising network;

[0050] S602, setting parameters α and β of a multi-feature loss function, and adopting an Adam algorithm as a gradient descent optimizer;

[0051] S603, setting a learning rate, a training frequency and a learning rate of the denoising network;

[0052] S604, inputting a SAR image to be denoised into the denoising network to obtain a denoised SAR image.

[0053] Compared with the prior art, the present application has the following beneficial effects:

[0054] The application discloses a SAR image denoising method based on self-supervised learning, comprising the following steps: constructing a SAR noise image model; constructing a minimum empirical loss function L; designing an adjacent co-directional sub-sampler; taking a noise image pair as input and target, modifying an empirical risk minimization task, and constructing a self-supervised SAR image denoising network; designing a multi-feature loss function; using the above method, training the SAR-CNN denoising network; and transforming a SAR image to be denoised into a log domain and inputting the SAR image into the network for denoising, and then obtaining an output image through inverse log transformation. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative work.

[0057] Figure 1 The flowchart of the present application;

[0058] Figure 2 The schematic diagram of the adjacent co-directional sub-sampler in the present application. DETAILED DESCRIPTION

[0059] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work belong to the scope of protection of the present application. The function units with the same and similar structures and functions have the same and similar structures and functions in the embodiments of the present application.

[0060] Referring to Figure 1 The present application provides a SAR image denoising method based on self-supervised learning, comprising the following steps:

[0061] S1, acquiring a SAR image and constructing a SAR noise image model;

[0062] S2. Construct a minimum empirical loss function to train the SAR image;

[0063] S3. Design adjacent co-directional sub-samplers to generate noisy image pairs from the SAR image through the adjacent co-directional sub-samplers;

[0064] S4. Use the generated noisy image pairs as the input and target for network training, modify the empirical risk minimization task, and construct a self-supervised SAR image denoising network.

[0065] S5. Design a multi-feature loss function to preserve the texture details of the SAR image;

[0066] S6. Based on steps S1-S5, the SAR image denoising network is trained to obtain the denoised SAR image.

[0067] In this embodiment, to address the problems existing in the prior art, firstly, the observed SAR image is acquired. The SAR image includes a SAR image with speckle noise (noisy image) and a corresponding SAR image without speckle at the bottom layer (clean image), which is represented by a multiplicative model:

[0068] Y = X × N

[0069] Where Y represents the noisy image, X represents the clean image, and N represents speckle noise; the SAR image is then converted to the log domain for denoising, converting multiplicative noise into additive noise.

[0070] log Y =log X +log N

[0071] Furthermore, since the speckle noise distribution has a unit mean, we have E(log Y ) = E(log X ).

[0072] Secondly, a minimum empirical loss function L is constructed to train the SAR image, and the denoising network f is trained using the minimum empirical loss function L. θ Multiple pairs of noisy and clean images are needed {(y i x i )}, its formula is expressed as:

[0073]

[0074] The above formula is equivalent to:

[0075]

[0076] By the log domain transformation of the noisy image and the clean image, the above formula is expressed as:

[0077]

[0078] Secondly, the adjacent co-directional sub-samplers are designed, and the noisy image log Y is divided into [W / k]x[H / k] patches, where k is the block size; two pixels of the patch extracted randomly in the horizontal or vertical direction, which satisfy the conditions of being adjacent and in the same direction, are taken as the (i, j)th element of the sub-sampler G=(g1, g2), respectively; for the [W / k]x[H / k] patches, the noisy image pair {g1(log y), g2(log y)} with the size of [W / k]x[H / k] is obtained by repeating the above step.

[0079] Further, the generated noisy image pair is taken as the input and target of network training, the empirical risk minimization task is modified, and a self-supervised SAR image denoising network is constructed. The self-supervised SAR image denoising network is constructed, and the image pair (g1(y), g2(y)) containing the same underlying clean target is sampled out by the noisy image pair generator, and the noisy image pair (g1(log y), g2 (log y)) is generated from a single noisy image y.

[0080] Where we assume that the noise of each pixel is independent and there is no correlation between the noise at different positions, then the pair of noisy images is independent given the true value of the original noisy image. Considering that the contents of the two sampled images (g1(log y), g2 (log y)) are very similar, the input and target have the same underlying clean target, and the noise comes from the same distribution. According to the characteristic that the speckle noise has the same mean value, the sub-sampled image distribution has the same expected value:

[0081] E(g1(log Y ))=E(g2(log Y ))=E(log Y )

[0082] When the distribution p(y|x) of the input conditional target distribution is replaced by an arbitrary distribution with the same conditional expected value, the optimal parameter θ of the network training also remains unchanged. Therefore, the empirical risk minimization task is modified as:

[0083]

[0084] Then, a multi-feature loss function is designed to remove noise while preserving image texture details as much as possible, considering the spatial and statistical characteristics of SAR images, and reducing the error between the noise pairs generated by the generator.

[0085] First, in order to meet the multiplication characteristics of speckle and speed up the convergence rate, the MSE is logarithmically scaled to obtain the first regularization term L desp , which is expressed as:

[0086]

[0087] Where g1(log y) and g2(log y) are the first and second noise image pairs generated by the sub-samplers, respectively, f θ (g1(log y)) and f θ (g2(log y)) are the first and second noise images obtained by the denoising network;

[0088] A second regularization term L reg is added to compensate for the first and second noise image pairs, the first and second noise images, and the first and second noise image pairs:

[0089] L reg =||f θ (g1(log y))-g2(log y)+g1(f θ (log y))-g2(f θ (log y))||

[0090] Where f θ is the denoising network model used for training, and g1(·) and g2(·) represent two sub-samplers, respectively;

[0091] A third regularization term L per is added to improve the quality of the denoised image:

[0092]

[0093] Where, represents the VGG network, represents a feature map of size CxHxW;

[0094] The first, second, and third regularization terms are added together with proportional coefficients to obtain the loss function of the SAR image denoising network:

[0095] L = L desp + alpha L reg + beta L per

[0096] Finally, the self-supervised SAR image denoising network is trained by using the self-learning supervised learning method, the SAR image is generated into a noise image pair by the sub-sampler, the patch size of the sub-sampler is preferably 2, the parameters a and β of the multi-feature loss function are set, preferably the parameters a and β are set to 2 and 1 respectively, the Adam algorithm is used as the gradient descent optimizer, and the learning rate, the training frequency and the learning rate of the denoising network are set, the initial learning rate is set to 0.0003, the training frequency is set to 200, and the learning rate is halved every 20 cycles; the SAR image to be denoised is input into the trained denoising network to obtain the denoised SAR image.

[0097] Compared with the prior art, the present application can be migrated to any existing denoising network, realize self-supervised denoising without clean targets, and significantly suppress speckle noise while reliably preserving image texture details and other features, solving the problem of poor practicability of current speckle data and real SAR image self-supervised denoising.

[0098] It should be noted that in this document, relational terms such as“first” and“second”, and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms“comprises”,“comprising”, or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by“comprises... a” does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0099] The above description is merely one specific implementation of the application, enabling a person skilled in the art to understand or implement the application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the application shall not be limited to the embodiments shown herein, but shall conform to the widest scope consistent with the principles and novel features set forth herein.

Claims

1. A method for denoising SAR images based on self-supervised learning, characterized in that, Comprise: S1, acquire SAR image, and construct SAR noise image model, acquire SAR image through observation, the SAR image includes noise image with speckle noise and bottom layer clean image without speckle, and is expressed as multiplication model: where Y denotes the noisy image and X denotes the clean image, denotes the element-wise product of two matrices, N denotes the speckle noise, and converting the noisy image to the log domain for denoising converts the multiplicative noise into additive noise: Also, since the speckle noise distribution has a unit mean value, there are ; S2, constructing a minimization empirical loss function for training the SAR image, the denoising network is trained by the minimization empirical loss function L Pairs of noisy and clean images are needed which is expressed as: The above formula is equivalent to: After the logarithmic domain transformation of noise image and clean image, the above formula is expressed as: ； S3, design adjacent co-directional sub-samplers, generate a pair of noise images by passing the SAR image through the adjacent co-directional sub-samplers, including S301, pass the noise images through the adjacent co-directional sub-samplers to generate a pair of noise images is divided into patches, wherein is a block size; S302, two pixels of the patch extracted randomly in the transverse direction or the longitudinal direction, the two pixels satisfying the condition of being adjacent and having the same direction, are respectively taken as the first element and the second element of the sub-sampler . ​ S303、For each patch, repeat S302 to obtain a pair of noise images with a size of ;​​ S4, modifying the empirical risk minimization task by taking the generated noisy image pair as the input and target of network training, constructing a self-supervised SAR image denoising network, including S401, sampling the noisy image by the adjacent co-directional sub-sampler to obtain an image pair containing the same underlying clean target , and generating a noisy image pair from a single noisy image ; S402, according to the characteristics of speckle noise having the same mean value, therefore the distribution of the sampled image pair and noise image pair has the same expected value, that is: S403, Distribution of input condition target distribution When replaced with any distribution having the same expected value, the optimal parameter θ for network training remains unchanged. Therefore, the task of minimizing empirical risk using zero mean is modified as follows: ； S5, design a multi-feature loss function for preserving the texture details of the SAR image, including S501, to meet the multiplication characteristics of the speckle and speed up the convergence rate, the MSE is scaled by logarithm to obtain a first regularization term , which is represented as: wherein g1(logy) and g2(logy) are the first and second pairs of noise images generated by the sub-sampler, respectively, f θ (g1(logy)) and f θ (g2(logy)) are the first and second pairs of noise images generated by the sub-sampler, respectively, f S502, adding a second regular term compensate the first noise image pair, the second noise image pair, the first noise image and the second noise image wherein, is a denoising network model for training, and denote two sub-samplers, respectively; S503, adding a third regularization term to improve the quality of the denoised image: wherein, represents a VGG network, represents a feature map of size S504, add the first regular term, the second regular term and the third regular term, and match with the proportional coefficient, obtain the loss function of SAR image denoising network: ； S6, based on steps S1-S5, train the SAR image denoising network to obtain the denoised SAR image. 2.The self-supervised learning based SAR image denoising method of claim 1, wherein, The SAR image denoising network is trained based on steps S1-S5 to obtain the denoised SAR image, comprising: S601, generate noise image pair through the sub-sampler as the input and target of the denoising network; S602, set the parameters α and β of the multi-feature loss function, and adopt Adam algorithm as the gradient descent optimizer; S603, set the learning rate, training frequency and learning rate of the denoising network; S604, input the SAR image to be denoised into the denoising network to obtain the denoised SAR image.

Images