A generative approach to suppress speckle noise in SAR images based on dilated residuals

Abstract

The present invention relates to a generative adversarial SAR image speckle noise suppression method based on dilated residuals, and belongs to the field of remote sensing image processing. The SAR image speckle noise suppression network model constructed by the present invention includes a generator network G, a texture enhancement network, and a discriminator network D. A pretrained texture enhancement network is used to calculate the feature perception error between the predicted image and the true image. A classification network is used as the discriminator. The predicted image output by the generator is combined with the true image loss, the predicted image features output by the texture enhancement network are combined with the feature loss of the true image, and the adversarial loss of the discriminator is combined with appropriate weights to form a final loss function. The model is trained, and speckle noise is suppressed on SAR images using the trained generator. The present invention achieves better speckle noise suppression with less spatial complexity and in a shorter time, improving the ability to predict detailed information such as edge texture in SAR images.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image processing, and relates to a synthetic aperture radar image speckle noise suppression technology, in particular to a generative anti-SAR image speckle noise suppression method based on dilated residuals. Background Art

[0002] Synthetic Aperture Radar (SAR) is a coherent imaging sensor operating in the microwave frequency band, using synthetic aperture technology to obtain large-scale, high-resolution radar images. SAR does not rely on sunlight, has strong penetrating power, and can image at night and in adverse weather conditions. It is widely used in military reconnaissance, terrain mapping and other fields. However, the inherent speckle noise problem in SAR images seriously affects the acquisition of SAR image information, increases the difficulty of image interpretation and analysis, and reduces the efficiency of image segmentation and feature classification. Existing classical methods often cause loss of image texture, edge and other detailed information while suppressing coherent speckle. Therefore, how to maintain image detail information while effectively suppressing coherent speckle in SAR images is one of the difficulties in SAR image speckle noise suppression.

[0003] Over the past two decades, deep convolutional neural networks have demonstrated powerful feature learning capabilities, capable of learning useful image features from large amounts of training data, and have found increasingly widespread application in image processing. In 2017, a SAR-CNN network was proposed for removing speckle multiplicative noise. Compared to traditional algorithms, it improved both speckle noise suppression performance and efficiency, marking the official entry of deep learning into the research of SAR image speckle reduction algorithms. In the same year, the ID-CNN network eliminated the pre-processing logarithmic transform and post-processing exponential transform. By dividing the input image by the noise fitted by the network to obtain the output image, it achieved end-to-end training, significantly improving the efficiency of image speckle noise reduction. In 2018, the SAR image speckle noise reduction network (SAR-DRN) based on dilated residuals was proposed, combining dilated convolution with a residual structure. This network achieved improvements in speckle noise reduction and image edge detail preservation. During the same period, generative adversarial networks (GANs) were used to suppress speckle noise in SAR images, and the ID-GAN model was proposed (Reference: Wang P, Zhang H, Patel VM. Generative adversarial network-based restoration of speckled SAR images[C] / / 2017IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing(CAMSAP).IEEE, 2018.DOI:10.1109 / CAMSAP.2017.8313133.). It is comparable to other speckle noise suppression networks in terms of speckle noise suppression and image detail preservation capabilities.

[0004] Existing neural network-based speckle noise suppression algorithms can achieve performance similar to that of traditional state-of-the-art algorithms, but there is still significant room for improvement. For example, the SAR-DRN network, currently recognized as a promising candidate, only provides end-to-end prediction output. As the convolutional layers become deeper, the image features learned by the filters become increasingly abstract. Therefore, this network, which only extracts features from the image through multiple convolutional layers and then reconstructs the speckle-suppressed image, is unable to recover details such as edges and textures. Furthermore, this network model has high spatial complexity and numerous parameters, requiring a large dataset. If the dataset is not large enough, it is prone to overfitting. However, compared to optical image datasets, datasets of both real SAR images and simulated speckle noise images are relatively scarce. As for the ID-GAN network, its generator still uses a traditional autoencoder structure, which consists of a deep stack of convolutional layers. Although batch normalization is incorporated within the convolutional layers to prevent problems such as vanishing and exploding gradients, the increased network depth leads to performance degradation. Summary of the Invention

[0005] To address the above problems, the present invention provides a generative anti-speckle noise suppression method for SAR images based on dilated residuals. A simplified dilated residual network is used as the generator network to achieve better speckle noise suppression with less spatial complexity and shorter time, and improve the ability to predict detailed information such as edge texture of SAR images.

[0006] The present invention provides a method for suppressing speckle noise in SAR images based on a generative approach based on dilated residuals, comprising the following steps:

[0007] Step 1: Construct a SAR image speckle noise suppression network model. The network model consists of three parts: generator network G, texture enhancement network, and discriminator network D.

[0008] The generator network G is a residual structure neural network based on dilated convolution, which includes an input layer, a hidden layer and an output layer connected in sequence; the hidden layer consists of 6 convolutional layers and a skip connection; each of the 1st to 5th layers includes a dilated convolutional layer and a ReLU activation function, and the 6th layer only has a dilated convolutional layer; the outputs of the 2nd and 5th layers are skipped; the input layer receives the original SAR image and inputs it into the hidden layer for feature extraction, and then the output layer subtracts the original SAR image from the output feature map of the hidden layer 6 to obtain the predicted denoised SAR image.

[0009] The texture enhancement network adopts the VGG16 model pre-trained based on ImageNet, and selects the feature map processed by the relu layer of the 7th layer of the VGG16 model as the output; the predicted image generated by the generator network G and the corresponding noise-free true value image are respectively input into the texture enhancement network to obtain the feature map after texture enhancement.

[0010] The discriminator network D is a two-class network used to determine the probability that the SAR image input to the network D is a real image.

[0011] Step 2: Train the constructed SAR image speckle noise suppression network model to obtain the trained optimal generator network G new .

[0012] The training set is input into the generator network model G, which outputs the predicted speckle noise suppressed image; the predicted image and the corresponding true value image are sent to the texture enhancement network, which outputs the feature maps of the two respectively; the predicted image and the corresponding noise-free true value image are sent to the discriminator network D, which outputs the probability that the discriminant image is true.

[0013] When training the SAR image speckle noise suppression network model, the loss function Loss is calculated as follows:

[0014] Loss = L E +λ a L A +λ p L P

[0015] Among them, L E It represents the per-pixel mean square error between the predicted image and the noise-free true image, and is calculated by minimizing L E Make the predicted image infinitely close to the noise-free true value image; L P is the feature perception loss, which is obtained by calculating the mean square error between the feature maps of the predicted image extracted by the texture enhancement network and the noise-free ground-truth image, and by minimizing L P Make the detailed features of the predicted image infinitely close to the detailed features of the noise-free true value image; L A is the adversarial loss from the discriminator network D; a and λ p is the loss weight set;

[0016] Step 3: Input the remote sensing image containing speckle noise into the trained generator network G new , output the remote sensing image after speckle noise suppression.

[0017] Compared with existing technologies, the advantages and positive effects of this invention are as follows: The method achieves better speckle noise suppression with less spatial complexity and in a shorter time, improving the ability to predict details such as edge texture in SAR images. The method simplifies the SAR-DRN network by reducing convolutional layers and connection structures, reducing spatial complexity, network parameters, and the required computer hardware resources and runtime. The method also introduces generative adversarial thinking to the simplified residual network, enhancing the prediction ability of the dilated residual network for details such as edge texture in SAR images by adding a texture enhancement network and a discriminator network. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of training and testing of a generative anti-SAR image speckle noise suppression network based on dilated residuals according to the present invention;

[0019] Figure 2 This is a network structure diagram of a generative anti-SAR image speckle noise suppression network generator based on dilated residuals of the present invention;

[0020] Figure 3 This is a network structure diagram of the discriminator network for suppressing speckle noise in SAR images based on the generative method of the present invention based on dilated residual;

[0021] Figure 4 A comparison diagram of speckle noise suppression results on a simulated image superimposed with speckle noise using different methods according to an embodiment of the present invention;

[0022] Figure 5 This is a comparison chart of speckle noise suppression results on real SAR images using different methods in the embodiments of the present invention. DETAILED DESCRIPTION

[0023] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is described in further detail and in depth below with reference to the accompanying drawings and specific implementation methods.

[0024] This dilated residual-based generative adversarial SAR image speckle noise suppression method uses a pretrained texture enhancement network to calculate the feature perception error between the predicted image and the true image. A classification network is then used as a discriminator network to determine whether the speckle noise-suppressed image predicted by the generator is true to the corresponding noise-free true image. Finally, the final loss function is formed by combining the predicted image output by the generator network with the true image loss, the predicted image features output by the texture enhancement network with the feature loss of the true image, and the adversarial loss of the discriminator network with appropriate weights. This method achieves better speckle noise suppression with less space complexity and faster time, improving the ability to predict detailed information such as edge texture in SAR images.

[0025] like Figure 1 As shown, the generative anti-SAR image speckle noise suppression method based on dilated residual implemented in an embodiment of the present invention includes the following three steps.

[0026] Step 1: Construct a SAR image speckle noise suppression network model. The network model consists of three parts: generator network G, texture enhancement network, and discriminator network D.

[0027] After training the SAR image speckle noise suppression network model, the collected SAR image is input into the model and the image with speckle noise suppressed is output.

[0028] like Figure 2 As shown, the generator network G is a residual structure neural network based on dilated convolution, comprising an input layer, a hidden layer, and an output layer connected in sequence; the hidden layer consists of the following: 6 convolutional layers and a skip connection. Each of the first to fifth layers includes a dilated convolution layer and a ReLU activation function, and the sixth layer only has a dilated convolution layer. In this embodiment of the present invention, each of the first five convolutional layers has 64 convolution kernels, and the sixth layer has only one convolution kernel. All convolution kernels are 3×3 in size, with a moving step of 1. The convolution kernel expansion rates of the entire network from the first to the sixth layers are 1, 2, 3, 3, 2, and 1, respectively. A skip connection connects the output of the second layer to the output of the fifth layer, and then inputs them together into the sixth layer. The input layer receives the original SAR image, which is input into the hidden layer for feature extraction. Then, at the output layer, the original SAR image is subtracted from the output feature map of the sixth hidden layer to obtain the predicted denoised SAR image.

[0029] The texture enhancement network adopts the VGG16 model pre-trained based on ImageNet, and the predicted image generated by the generator network G and the corresponding noise-free true image are respectively input into the texture enhancement network, and the feature map processed by the relu7 layer of the VGG16 model, that is, the relu layer of the 7th layer of the network, is selected as the output of the texture enhancement network. The feature map after feature extraction of the predicted image and the true value image is calculated as the feature perception loss. In the training stage, the true value image refers to the image without adding speckle noise. The present invention sets a texture enhancement network, extracts features from the predicted image and the true value image output by the generator G, and introduces the error of the extracted feature map into the comprehensive loss function with a certain weight. This avoids the single error based on the pixel value of the predicted image and the true value image. The feature map error reflects the information of the texture structure and other aspects of the image, so that the error consideration is more comprehensive, so as to improve the prediction of detail information such as the edge texture of the SAR image.

[0030] like Figure 3As shown, the discriminator network D is essentially a two-classification network, which includes five convolution layers and a sigmoid function layer. In the embodiment of the present invention, the first convolution layer includes two structures: convolution and LeakeyReLU activation function (Conv+LeakeyReLU), with a total of 48 convolution kernels. The second, third, and fourth layers are convolution, batch normalization, and rectified linear unit (Conv+BN+ReLu) structures, with 96, 192, and 384 convolution kernels in each layer, respectively. The fifth layer has only a convolution structure, and the number of convolution kernels is 1. The size of all convolution kernels is 3×3, and the moving step is 1. The last layer is a sigmoid function layer, which normalizes the output value mapping to a probability score of [0, 1]. The closer the output value is to 1, the more likely the discriminator D thinks the image is a real image.

[0031] Step 2: Train the SAR image speckle noise suppression network model constructed in step 1 to obtain the trained optimal generator G network model.

[0032] In the embodiment of the present invention, optical image points are multiplied by a simulated random noise matrix with a gamma distribution and a view count of 1 to produce simulated speckle noise images as training data sets and test sets, and real SAR images are also selected as the test set.

[0033] The training set is input into the generator network model G, which outputs the predicted image after speckle noise suppression; the predicted image and the corresponding true image are sent to the texture enhancement network, which outputs the feature maps of the two respectively; the predicted image and the corresponding true image are sent to the discriminator network D, which outputs the probability that the discriminant image is true.

[0034] Calculate the per-pixel mean square error L between the predicted image and the true image E By minimizing the mean square error of the two, the output speckle noise suppression image is infinitely close to the noise-free true value image. Let the training set image pair be (X i，j , Y i，j ), where Y i，j is the input image containing speckle noise, X i，j is the corresponding true image without speckle noise, X i，j and Y i，j The size of is W×H, θ is the parameter in the generator G network model, and the predicted output image The specific calculation method is as follows:

[0035]

[0036] Among them, Θ G Indicates that the parameters of the generator network G need to be optimized.

[0037] Calculate the feature-aware loss L of the texture enhancement networkP , which is essentially and X i，j By minimizing the mean square error of the feature map output by the relu7 layer of the pre-trained VGG16 model, the detail features of the output speckle noise suppression image are infinitely close to the detail features of the noise-free true value image. The specific calculation method is as follows:

[0038]

[0039] Where V represents the texture enhancement network.

[0040] Compute the adversarial loss L from the discriminator network D A , assuming that each time N pairs of image samples are input to the generator subnetwork G The output prediction image group is γ is a parameter in the discriminator D network model. The adversarial loss of each set of predicted images and true value images is calculated as follows:

[0041]

[0042] Among them, Γ D Indicates that the parameters of the discriminator network D need to be optimized. represents the output of the discriminator network D.

[0043] Assign certain weights to the three to form a combined loss function. The combined loss function Loss is defined as follows:

[0044] Loss = L E +λ a L A +λ p L P

[0045] In the embodiment of the present invention, the weight λ in the combined loss function is set a =6.6×10 -3 ,λ p =1, the batch size is 16, the ADAM optimization method is used for training, the exponential decay rate of the first-order moment estimate β1 is set to 0.9, the exponential decay rate of the second-order moment estimate β2 is set to 0.999, and the parameter ε is set to 10 -8 The learning rate is 0.001. The parameters of the neural network with the best noise suppression performance are recorded to obtain the trained convolutional neural network. The standard for the best noise suppression performance is when the loss function value is minimized.

[0046] Step 3: Input the remote sensing image containing speckle noise into the trained generator network, output the remote sensing image after speckle noise suppression, and test or use the speckle noise suppression network.

[0047] In order to verify the method of the present invention, the following simulation experiments are carried out:

[0048] A dataset of 400 256×256 pixel images from various scenes was selected from the UC Merced land use dataset. These images were multiplied by a simulated gamma-distributed random noise matrix with a look-count of 1 to create simulated speckle noise images. 90% of these images served as the training set, and the remaining 10% served as the test set. Real SAR images based on TerraSAR-X data were also selected and cut into 256×256 pixels to serve as the test set. The network was first trained using the simulated image dataset. The trained network was then tested for speckle noise suppression using both simulated and real SAR images. The equivalent look-count (ENL), edge preservation index (EPI), and the ratio (R) of the original image to the despeckled image were used to evaluate and compare the filtering performance in homogeneous areas, edge preservation, and overall image information preservation.

[0049] The equivalent view number (ENL) is used to evaluate a filter's ability to remove speckle in homogeneous areas. A larger ENL indicates a filter's stronger speckle noise suppression capability in homogeneous areas. The edge preservation index (EPI) measures a filter's ability to preserve horizontal or vertical edges in an image. An ideal edge preservation index (EPI) is 1 without any processing. After processing, the EPI is less than 1. The closer the EPI is to 1, the stronger the filter's edge preservation capability. The ratio (R) of the original image to the despeckled image is very useful in both homogeneous and non-homogeneous areas. The mean and variance of R are generally used as evaluation metrics. Ideally, R should follow a gamma distribution with a mean of 1 and a variance of 1 / L. The closer the mean of R is to the ideal, the better the despeckling algorithm preserves the original image's radiometric information. The despeckled image will have a smaller deviation from the original image, and the probability of false targets is reduced. The closer the variance of R is to the ideal, the stronger the image's overall speckle noise suppression capability.

[0050] The test results of the method of the present invention and the comparative method on the optical simulation image with superimposed speckle noise are shown in the following figure: Figure 4 As shown in Table 1, the coherent speckle suppression effect and edge preservation ability evaluation index data in the homogeneous area are shown.

[0051] Table 1 Methods for Figure 4 Evaluation indicators of simulation image experiments in

[0052]

[0053] The comparison method of the embodiment of the present invention includes the SAR-DRN method and the SAR-BM3D method. The speckle noise is superimposed on the noise-free true value images (c) and (d) to obtain the optical simulation images (a) and (b). Four methods are used to denoise (a) and (b). The comparison results of the three groups of methods are shown in Table 1. Figure 4 Images (g) and (h) processed by the ID-GAN method are also provided.

[0054] As can be seen from the above, compared with the SAR-DRN method, the method of the present invention improves edge texture preservation while maintaining the coherent speckle suppression capability in homogeneous areas. Compared with the SAR-BM3D algorithm, the method of the present invention also improves both coherent speckle suppression in homogeneous areas and edge preservation.

[0055] The test results of the method of the present invention and the comparative method on real SAR images are as follows: Figure 5 As shown in Table 2, the coherent speckle suppression effect and edge preservation ability evaluation index data in the homogeneous area are shown.

[0056] Table 2 Methods for Figure 5 Evaluation metrics for real SAR image experiments

[0057]

[0058] As shown in the figure and Table 2, compared with the SAR-DRN method, the proposed method significantly improves edge texture preservation while maintaining both speckle suppression in homogeneous areas and overall image speckle suppression and information preservation. Compared with the SAR-BM3D algorithm, the proposed method is comparable in terms of overall image information preservation and edge detail preservation, demonstrating superior speckle suppression in homogeneous areas.

[0059] The test running time of the method of the present invention and the comparative method on real SAR images is shown in Table 3.

[0060] Table 3 Comparison of time consumption of different methods

[0061] algorithm SAR-BM3D SAR-DRN Method of the present invention Run time (s) 14.67 0.38 0.30

[0062] It can be seen from Table 3 that the time consumption of generating a speckle noise suppressed image by the method of the present invention is less than that of the comparison method, and the processing speed is faster.

[0063] Except for the technical features described in the specification, all other technical features are known to those skilled in the art. The present invention omits descriptions of well-known components and well-known technologies to avoid redundancy and unnecessary limitation of the present invention. The implementation methods described in the above embodiments do not represent all implementation methods consistent with the present application. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the present invention.

Claims

1. A generative anti-speckle noise suppression method for SAR images based on dilated residuals, characterized by: The steps include: Step 1: Construct a SAR image speckle noise suppression network model; The network model includes a generator network G, a texture enhancement network and a discriminator network D; The generator network G is a residual structure neural network based on dilated convolution, which includes an input layer, a hidden layer and an output layer connected in sequence; The hidden layer consists of 6 convolutional layers and a skip connection. Each of the 1st to 5th layers includes a dilated convolution layer and a ReLU activation function. The 6th layer only has a dilated convolution layer, and the outputs of the 2nd and 5th layers are skipped. The input layer receives the original SAR image and inputs it into the hidden layer for feature extraction. The output layer then subtracts the original SAR image from the output feature map of the 6th hidden layer to obtain the denoised SAR image. The texture enhancement network uses the VGG16 model pre-trained on ImageNet, and selects the feature map processed by the relu layer of the 7th layer of the VGG16 model as the output; the predicted image generated by the generator network G and the corresponding noise-free true value image are respectively input into the texture enhancement network to obtain the feature map; The discriminator network D is a two-class network used to determine the probability that the SAR image input to the network D is a real image; Step 2: training the SAR image speckle noise suppression network model; The training set is input into the generator network G to predict the image after speckle noise suppression; the predicted image and the noise-free true value image are respectively input into the texture enhancement network to obtain the feature maps of the two; the predicted image and the noise-free true value image are sent to the discriminator network D to determine the probability of the image being real; When training the SAR image speckle noise suppression network model, the loss function Loss is calculated as follows: Loss=L E +λ a L A +λ p L P Among them, L E It represents the per-pixel mean square error between the predicted image and the noise-free true image, and is calculated by minimizing L E Make the predicted image infinitely close to the noise-free true value image; L P is the feature perception loss, which is obtained by calculating the mean square error between the feature maps of the predicted image extracted by the texture enhancement network and the noise-free ground-truth image, and by minimizing L P Make the detailed features of the predicted image infinitely close to the detailed features of the noise-free true value image; L A is the adversarial loss from the discriminator network D; a and λ p is the loss weight set; Step 3: Input the remote sensing image containing speckle noise into the trained generator network G new , output the remote sensing image after speckle noise suppression.

2. The method according to claim 1, characterized in that In the generator network G, each of the first five convolutional layers of the hidden layer has 64 convolution kernels, the sixth layer has only one convolution kernel, the size of all convolution kernels is 3×3, the moving step is 1, and the convolution kernel expansion rates from the first layer to the sixth layer are set to 1, 2, 3, 3, 2, and 1 respectively.

3. The method according to claim 1, characterized in that The discriminator network D includes 5 convolutional layers and 1 sigmoid function layer; the first convolutional layer includes two parts: convolution and LeakeyReLU activation function, with a total of 48 convolution kernels; the second to fourth convolutional layers all include convolution, batch normalization and rectified linear unit structures, and the corresponding numbers of convolution kernels are 96, 192, and 384, respectively; the fifth convolutional layer contains only convolution structure, with 1 convolution kernel; the size of all convolution kernels is 3×3, and the moving step is 1; the final sigmoid function layer normalizes the output mapping of the fifth convolutional layer to the probability of [0, 1], and the probability represents the probability that the image is a real image.

4. The method according to claim 1, wherein In the step 2, let the training sample pair (X i，j , Y i，j ), where Y i，j is the input image containing speckle noise, X i，j is the corresponding noise-free true value image, and the generator network G outputs the predicted image Then the error L E The calculation is as follows: Among them, X i，j and Y i，j The size of is W×H, θ is the parameter of the generator network G; The predicted image and the noise-free ground-truth image X i，j Input texture enhancement network, calculate the mean square error L of the extracted feature maps of the two P as follows: Among them, V(G θ (Y i，j ))、V(X i，j ) represent the texture enhancement network and X i，j Extracted feature maps; Adversarial loss L of the discriminator network D A The calculation is as follows: Among them, N is the number of input training samples, input N training sample pairs The predicted image group is obtained by the generator network G γ is the parameter of the discriminator network D, For the discriminator network D The judgment result of .

5. The method according to claim 1 or 4, characterized in that In the step 2, the weight λ is set a =6.6×10 -3 ,λ p =1.

6. The method according to claim 1, characterized in that In the step 2, a simulated speckle noise image is produced as a training data set and a test set by multiplying the optical image by a simulated gamma distribution and a random noise matrix with a view count of 1. At the same time, a real SAR image is also selected as a test set.

Images