A SAR image denoising optimization method, system and storage medium

By combining DPDNN and U-Net PDSNet denoising networks, using attention mechanism and improved structural loss function, the problem of multiplied coherent noise removal in SAR images is solved, which significantly improves the noise suppression and detail retention capabilities of the image. In the detection and classification application of synthetic aperture radar images.

CN116523790BActive Publication Date: 2024-11-05GUANGDONG OCEAN UNIVERSITY +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310510108.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-08
Publication Date
2024-11-05
Estimated Expiration
2043-05-08

AI Technical Summary

Technical Problem

The prior art is difficult to effectively remove multipliable coherence noise in SAR images, resulting in image edge damage and texture detail information loss, affecting image application.

Method used

Using the combination of DPDNN and U-Net neural network structures to build a PDSNet denoising network, use feature encoder and feature decoder to replace the pooling layer with attention mechanism module, and introduce improved structural loss function, and improve Canny in combination with universal gravitation theory. Edge gradient algorithm, performs image denoising processing.

Benefits of technology

It realizes more accurate image denoising, retains more texture and detailed information, improves noise suppression ability and edge information preservation, and is suitable for applications such as detection and classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116523790B_ABST
    Figure CN116523790B_ABST
Patent Text Reader

Abstract

The present invention discloses a SAR image denoising optimization method, system and storage medium. By constructing a deep CNN model PDSNet based on prior drive and structural loss function, the model can more accurately identify key information in the image by learning implicit prior knowledge of remote sensing images and adding structural loss functions, thereby achieving accurate noise removal while retaining more image texture and detail information. The SAR image denoising optimization method based on the PDSNet network has stronger noise suppression ability on real images than existing methods, and can also well preserve the edge information of the original noise image, which is helpful for applying synthetic aperture radar images to detection, classification and other related work.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a SAR image denoising optimization method, system and storage medium. Background Art

[0002] Synthetic aperture radar (SAR) plays an important role in various earth observation tasks due to its various advantages. However, due to the coherence of SAR imaging, multiplicative coherent speckle noise and artifacts are inevitably present in SAR images. Speckle noise destroys the edge of the target in the SAR image, affecting people's reading of image information and the subsequent application of SAR images. Therefore, it is particularly important to perform despeckle processing on SAR images.

[0003] In recent years, with the rapid development of artificial intelligence and deep learning technology, many researchers have used convolutional neural networks (CNNs) as the main tool for SAR image coherent speckle suppression. The standard CNN-based workflow requires noise-free images to train the network. However, due to the coherence of SAR imaging, there are no clean speckle-free images. To this end, scholars have proposed multi-temporal remote sensing image technology and self-supervised algorithms for research. However, whether using multi-temporal sequence denoising methods or self-supervised denoising algorithms, the texture detail information in the image cannot be correctly extracted due to the pollution information in the SAR image, and good denoising performance cannot be achieved. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a SAR image denoising optimization method, comprising the following steps:

[0005] S1, select sensor image data as training data, perform grayscale processing on the training data, and then add multiplicative noise as the training data set of the network;

[0006] S2, by combining the neural network structures of DPDNN and U-Net, a denoising network PDSNet based on U-Net is constructed, wherein the denoising network PDSNet includes a feature encoder and a feature decoder;

[0007] The feature encoder is composed of multiple convolutional layers and a nonlinear ReLU function, and an attention mechanism module is used to replace the pooling layer of the original U-Net network. The attention mechanism module includes a channel attention module and a spatial attention module, wherein the channel attention module is composed of a maximum pooling layer, an average pooling layer and multiple convolutional layers, and the spatial attention module is composed of a convolutional layer and an activation function;

[0008] The feature decoder comprises a plurality of convolutional layers grouped into four feature decoders, and an upsampling layer for fusing a feature map of the same spatial resolution generated in the encoding stage with an upsampled feature map generated in the decoding stage to obtain a new upsampled feature map;

[0009] S3, improve the loss function of the denoising network PDSNet, given N pairs of images The improved structural loss function is expressed as follows:

[0010]

[0011] Where N is the number of noise-free image pairs, Θ is the network parameter, and x noise Represents the noisy image input to the network, y ground represents ground truth, R(x noise ; Θ) represents the reconstructed image obtained by the PDSNet network, and C(·) represents the structural information of the image;

[0012] Use the training dataset to jointly train the denoising network PDSNet that introduces the improved structural loss function;

[0013] S4, inputting the SAR image to be denoised into the trained denoising network PDSNet to obtain the denoised SAR image.

[0014] Preferably, the step S2 specifically includes:

[0015] A feature encoder consisting of multiple 3×3 convolutional layers and a nonlinear ReLU function is used. A scaling factor of 2 is used in the downsampling layer to downsample along both axes of the feature map.

[0016] Preferably, the step S3 specifically includes:

[0017] The Canny edge gradient algorithm is improved based on the theory of universal gravitation. The gravitational field strength replaces the original edge gradient algorithm. It is assumed that the total gravitational field strength generated at each point is a combination of the gravitational field strengths generated by the surrounding pixels. The edge point is the point with the highest gradient strength in the image. The total result field strength for each point can be calculated by the following formula:

[0018] Where E is the gravitational field strength, m is the gray value of the target pixel, G is the gravitational field number, is the distance between pixel 1 and pixel 2.

[0019] The present invention also discloses a SAR image denoising optimization system, comprising:

[0020] A training data acquisition module is used to select sensor image data as training data, perform grayscale processing on the training data, and then add multiplicative noise as a training data set for the network;

[0021] A network component module, used to construct a denoising network PDSNet based on U-Net by combining the neural network structures of DPDNN and U-Net, wherein the denoising network PDSNet includes a feature encoder and a feature decoder; the feature encoder consists of multiple convolutional layers and a nonlinear ReLU function, and an attention mechanism module is used to replace the pooling layer of the original U-Net network, and the attention mechanism module includes a channel attention module and a spatial attention module, wherein the channel attention module consists of a maximum pooling layer, an average pooling layer and multiple convolutional layers, and the spatial attention module consists of a convolutional layer and an activation function; the feature decoder includes multiple convolutional layers grouped into four feature decoders, and an upsampling layer for fusing a feature map of the same spatial resolution generated in the encoding stage and an upsampled feature map generated in the decoding stage to obtain a new upsampled feature map;

[0022] The loss function improvement module is used to improve the loss function of the denoising network PDSNet. Given N pairs of images The improved structural loss function is expressed as follows:

[0023]

[0024] Where N is the number of noise-free image pairs, Θ is the network parameter, and x noise Represents the noisy image input to the network, y ground represents ground truth, R(x noise ; Θ) represents the reconstructed image obtained by the PDSNet network, and C(·) represents the structural information of the image; the denoising network PDSNet with the improved structural loss function is jointly trained using the training dataset;

[0025] The image denoising module is used to input the SAR image to be denoised into the trained denoising network PDSNet to obtain the denoised SAR image.

[0026] Preferably, the feature encoder is composed of a plurality of 3×3 convolutional layers and a nonlinear ReLU function, and a scaling factor of 2 is used in the downsampling layer to perform downsampling along two axes of the feature map.

[0027] Preferably, the loss function improvement module is configured to improve the Canny edge gradient algorithm based on the theory of universal gravitation, replace the original edge gradient algorithm with the gravitational field strength, and assume that the total gravitational field strength generated at each point is a combination of the gravitational field strengths generated by surrounding pixels, then the edge point is the point with the highest gradient strength in the image, and the total result field strength for each point can be calculated by the following formula: Where E is the gravitational field strength, m is the gray value of the target pixel, G is the gravitational field number, is the distance between pixel 1 and pixel 2.

[0028] The present invention also discloses a SAR image denoising and optimization device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of any of the above-mentioned SAR image denoising and optimization methods are implemented.

[0029] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned SAR image denoising optimization methods are implemented.

[0030] The SAR image denoising optimization method, system and storage medium disclosed in this embodiment propose a deep CNN model PDSNet based on prior drive and structural loss function. By learning the implicit prior knowledge of remote sensing images and adding structural loss functions, the model can more accurately identify the key information in the image, thereby achieving accurate noise removal while retaining more image texture and detail information. In addition, by adding an integral term to the denoising network, the robustness of the network can be effectively guaranteed under a large amount of noise interference. This SAR image denoising optimization method has stronger noise suppression capabilities than existing methods on real images, both in terms of subjective visual quality evaluation and objective evaluation indicators, and can also well preserve the edge information of the original noisy image, which is helpful for applying synthetic aperture radar images to detection, classification and other related tasks.

[0031] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0033] Picture 1The present invention is a flowchart of a SAR image denoising optimization method disclosed in an embodiment of the present invention.

[0034] Picture 2 A schematic diagram of the architecture of a deep convolutional neural network disclosed in one embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Unless otherwise defined, the technical or scientific terms used herein shall have the common meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the patent application specification and claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "a" do not indicate a quantity limitation, but indicate the existence of at least one.

[0037] In order to solve the problems of the lack of discrimination ability of the current remote sensing despeckle network based on deep learning and the loss of image details after restoration, this embodiment proposes a deep CNN model PDSNet based on prior drive and structural loss function, so that a new SAR image filter or functional module can be formed based on this method. By learning the implicit prior knowledge of remote sensing images and adding a structural loss function, the method model can more accurately identify the key information in the image, thereby achieving accurate noise removal while retaining more image texture and detail information. As shown in the attached figure Picture 1 As shown, the SAR image denoising optimization method may include the following contents.

[0038] Step S1, selecting sensor image data as training data, grayscale processing the training data, and then adding multiplicative noise as a training data set for the network.

[0039] At present, whether using a multi-phase sequence denoising method or a self-supervised denoising algorithm, the texture detail information in the image cannot be correctly extracted due to the pollution information in the SAR image. By training the network with simulated images, the learning network can be used to learn the detail information in various scenes more widely, which is more helpful to retain the texture information in the image in the real SAR image restoration task. In this embodiment, the NWPU-RESISC45 sensor image public data created by NWPU can be selected as the training data set. The data set contains 31,500 images, covering 45 scene classes, each class has 700 images, and each image has the same size, 256×256. In this embodiment, 800 images are randomly selected as the training set, and after each stage of training, 10 images that are not in the training set are used as the test set. At the same time, since the real SAR image is usually a grayscale image, the data set is first grayscale processed, and then multiplicative noise is added as the training data of the network.

[0040] In this embodiment, before constructing the denoising network of the subsequent step S2, the SAR image denoising optimization method also includes first formulating a SAR image denoising strategy. Specifically, for the denoising problem of SAR images, a feedforward residual network framework based on prior driving is disclosed. The framework separates the data item and the prior item by applying the variable splitting technology and the semi-quadratic splitting method to the SAR speckle removal or denoising method, and guides the iterative optimization problem with the image prior information. In order to solve the optimization problem, a feedforward residual denoising network PDSNet based on prior driving is proposed. The weights of the network are like the parameters of the restored image, so that the denoiser and parameters can be optimized end-to-end.

[0041] Among them, the SAR image is formed by the coherence of the echo signal of the radar pulse. When the electromagnetic wave irradiates a relatively rough surface or irradiates multiple scatterers at the same time, due to the relative motion between the radar target and the radar station, the multiple scatterers have different distances and relative speeds from the radar. This results in the echo signal received by the radar receiver being coherent in frequency, but no longer coherent in phase. As a result, the grayscale index of adjacent pixels will fluctuate randomly around a certain mean due to coherence, and inevitable coherent speckle noise will be generated on the image. This noise is different from the common additive Gaussian noise model and is a multiplicative noise. Therefore, it is also more difficult to remove than the additive noise existing in general images. The multiplicative noise model can be expressed as: I(x,y)=u(x,y)×N(x,y);

[0042] Where (x, y) represents the coordinates of the unit center pixel in azimuth and range; I(x, y) represents the actual observed image intensity, that is, the intensity of each image pixel contaminated by coherent speckle noise; u(x, y) represents the radar echo of a random ground target, that is, the observed real image intensity not contaminated by coherent speckle noise; N(x, y) represents the coherent speckle noise caused by the fading process.

[0043] And u(x,y) and N(x,y) are independent of each other, and N(x,y) obeys the generalized Gamma distribution: Wherein, L>1, N>0, Γ(·) represents the Gamma distribution function, and L is the number of views of the SAR image. in and var are the mean and variance of the SAR image I, respectively.

[0044] At present, the methods of image restoration can be divided into two categories: one is the learning-based method, and the other is the model-based method. The model-based method does not require training, but relies on the inductive bias as prior knowledge. The performance of this method depends on the selected prior knowledge. This method is often an optimization problem with an additional regularization term, and the objective function is as follows: Among them, E(x,x0) is a task-related data item, such as MSE, which is used to describe the gap between the reconstructed image and the target image. x0 is the damaged image. R(x) is a regularization term that constrains the solution space based on prior knowledge. The choice of data item E(x,x0) is usually determined by the task.

[0045] In this embodiment, in combination with the above multiplicative noise model and objective function formula, the variable splitting technique can be used to decouple the data term and the regularization term for the above iterative optimization problem, and an auxiliary variable d is introduced. The above multiplicative noise model and objective function formula can be converted into:

[0046]

[0047] Where λΦ(x) is the weighted term of R(x), then the ideal solution to the above objective function formula is the solution that minimizes both the data term and the regularization term weighted by the parameter λ. On this basis, the semi-quadratic splitting method can be used to transform the equal-constrained optimization problem into an unconstrained optimization problem:

[0048]

[0049] W=N T (Nx k -x * )+μ(x k -d)+H

[0050]

[0051] Where x = W -1 b, W is a matrix related to the coherent noise matrix N, and x is calculated by the Newton-Raphson iterative algorithm with an integral term k+1 , γ is a proportional factor greater than zero, and H is the integral term.

[0052] In this embodiment, since the prototype of the above optimization algorithm is a continuous function, it needs to be discretized to adapt to the computing requirements of the computer system. The PID algorithm can be modified as follows:

[0053]

[0054] where e k is a steady-state error, K p ,T d ,T i Represent different system parameters. The integral term is used to eliminate steady-state errors and can be combined with proportional and differential control by adjusting the performance of the system. Before reaching the node, e i is a positive number, and its integral is always greater than zero. When the system has a steady-state error, the error will remain the same value, but the integral will change, which means that the value of the previous steady-state error neutralizes the proportional control algorithm, and the integral term will play a continuous role, which can make x k The output of keeps increasing, thus eliminating the steady-state error. The subproblem of d is a neighbor operator of Φ(d), which can be solved by the denoiser: d k+1 =f(x k ), where f(·) represents the denoiser. In summary, the denoising task for SAR images can be summarized as the following denoising task algorithm:

[0055]

[0056] In this embodiment, after formulating the SAR image denoising strategy, the implicit prior information of the image is first captured by selecting a specific generator network structure, and then sent to the denoiser as prior knowledge for network training. By learning the implicit prior information of various types of images themselves, the network itself can extract valuable information from multiple types of images for denoising, so that the network can also adapt when it is extended to real SAR images.

[0057] In this embodiment, d k+1 =f(x k ) in the form of a neural network, the solution of the formula can be expressed as d k+1 =f θ (x k ), where θ is the network parameter, through fθ (·) The denoising network maps parameters θ to the denoised signal d, which include the weights and biases of the filters in the network. The network itself has a standard structure and alternates filtering operations such as linear convolution, upsampling, and nonlinear activation functions, while capturing implicit priors by dropping the explicit regularizer R(x) and using neural network parameterization as follows:

[0058]

[0059] The above formula can be used to expand the denoising task algorithm as shown in the following figure: Picture 2 The deep convolutional neural network with the architecture shown in the figure can accurately perform the t iterations of the aforementioned denoising task algorithm to obtain the noise information in the noisy image. Picture 2 As shown in the figure, the input degraded image I first passes through a linear layer parameterized by the degradation matrix N to obtain the initial estimate x0, and then x0 is sent to the denoising module and the linear layer parameterized by the matrix W. Then, a weighted denoising signal d1 is added to the output of the linear layer N and W, and then the shortcut connection of the residual network is used to obtain the updated x1. Continuous iterations can obtain the corresponding noise signal x k .

[0060] The prior network f disclosed in this embodiment θ (·) No aspects are learned from data in advance, so this deep image prior has practically the same performance as hand-crafted priors, such as using the TV norm.

[0061] Step S2, by combining the neural network structures of DPDNN and U-Net, a denoising network PDSNet based on U-Net is constructed, wherein the denoising network PDSNet includes a feature encoder and a feature decoder. The feature encoder is composed of multiple convolutional layers and a nonlinear ReLU function, and an attention mechanism module is used to replace the pooling layer of the original U-Net network. The attention mechanism module includes a channel attention module and a spatial attention module, wherein the channel attention module is composed of a maximum pooling layer, an average pooling layer and multiple convolutional layers, and the spatial attention module is composed of a convolutional layer and an activation function; the feature decoder includes multiple convolutional layers grouped into four feature decoders, and an upsampling layer for fusing the feature map of the same spatial resolution generated in the encoding stage and the upsampled feature map generated in the decoding stage to obtain a new upsampled feature map.

[0062] In this embodiment, step S2 may specifically include: using a feature encoder composed of multiple 3×3 convolutional layers and a nonlinear ReLU function, using a scaling factor of 2 in the downsampling layer, and downsampling along two axes of the feature map.

[0063] Specifically, the denoiser structure is constructed, and the denoising network PDSNet based on U-Net is designed by combining DPDNN and U-Net neural network structures. The denoiser, namely the denoising network PDSNet, consists of 13 layers and is divided into two different parts: feature encoder and feature decoder.

[0064] In the encoding part, the feature encoder consists of a series of 3×3 convolutional layers and a nonlinear ReLU function. In this embodiment, the scaling factor is set to 0.5, and the downsampling is performed along the two axes of the feature map using a scaling factor of 2 in the downsampling layer. Considering that the pooling layer in the U-net network reduces the image size by simple scaling, it is impossible to determine which part needs to be retained for different images. Therefore, in this embodiment, the pooling layer of the original U-Net network is replaced. Specifically, the attention mechanism module is used to replace the pooling layer of the original U-Net network. The function of this module is to use the information of the current stage to guide the previous stage to learn the noise information, so that the network can focus on the significant information in the noisy image and ignore the unimportant information, thereby improving the efficiency and accuracy of information processing. There are two parts in the attention mechanism module: the channel attention module and the spatial attention module. The channel attention module consists of a maximum pooling layer, an average pooling layer and a series of convolutional layers; the spatial attention module consists of a convolutional layer and an activation function.

[0065] In the decoding part, the feature decoder contains a series of convolutional layers, which are grouped into four feature decoders. This is followed by an upsampling layer. Since the final extracted feature map will lose a lot of spatial information, directly using these features to reconstruct the image will result in insufficient image details. In order to compensate for this defect in the decoding stage, the feature map with the same spatial resolution generated in the encoding stage and the upsampled feature map generated in the decoding stage are fused together to obtain a new upsampled feature map. In this way, the lost spatial information can be supplemented on the basis of retaining the original features, and a more detailed image can be reconstructed. Finally, 6 such denoisers are used to help the algorithm convergence of the aforementioned denoising task algorithm, and we force all parameters to be shared to avoid overfitting. Instead of directly reconstructing the original image, the denoiser forces the denoising network to predict the residual, which has been verified to be more robust. To this end, we added a jump connection from the input to the reconstructed image in the denoising network.

[0066] Step S3, improve the loss function of the denoising network PDSNet, given N pairs of images The improved structural loss function is expressed as follows:

[0067]

[0068] Where N is the number of noise-free image pairs, Θ is the network parameter, and x noise Represents the noisy image input to the network, y ground represents ground truth, R(x noise ; Θ) represents the reconstructed image obtained by the PDSNet network, and C(·) represents the structural information of the image; the denoising network PDSNet with the improved structural loss function is jointly trained using the training dataset.

[0069] Specifically, currently, only using MSE loss may make the denoised image area smooth, and the visual effect of the image is not good. Therefore, in order to further improve the network's ability to retain edge information and alleviate the problem of image blur caused by upsampling. In the SAR image denoising optimization method of this embodiment, structural loss is introduced for joint training to make up for the defects of MSE loss function in image denoising tasks. The proposed structural loss function can not only guide the training direction of the network, making the network converge faster, but also ensure that the texture information in the restored image takes into account the quality and visual effect of the generated denoised image.

[0070] In this embodiment, step S3 may specifically include the following contents.

[0071] The Canny edge gradient algorithm is improved based on the theory of universal gravitation. The gravitational field strength replaces the original edge gradient algorithm. It is assumed that the total gravitational field strength generated at each point is a combination of the gravitational field strengths generated by the surrounding pixels. The edge point is the point with the highest gradient strength in the image. The total result field strength for each point can be calculated by the following formula:

[0072] Where E is the gravitational field strength, m is the gray value of the target pixel, G is the gravitational field number, is the distance between pixel 1 and pixel 2.

[0073] Specifically, in this embodiment, the Canny operator is selected and improved as a structural information extraction algorithm. The Canny edge detection algorithm is widely cited in practical engineering, but in the traditional algorithm, the first-order finite difference in the 2×2 field is used to calculate the gradient of the image. This method is simple to calculate, but it is sensitive to noise information. At the same time, since the deviation in the 45° and 135° directions is not added, it is easy to lose the real edge information. Therefore, for SAR images, which have rich edge information, the traditional edge detection algorithm is not ideal in adaptability and is easy to lose local feature edge information. Therefore, in this embodiment, the concept of gravity field strength can be introduced to replace the image gradient.

[0074] The new edge detection algorithm based on the theory of universal gravitation disclosed in this embodiment is not only superior to the Sobel and Prewitt operators in edge extraction with a single criterion, but also can retain more useful edge information and has a good suppression effect on noise. However, since the gray value of each pixel is equivalent to the mass to calculate the gravitational intensity, the performance of the gravity edge detection algorithm in bright areas (areas with larger gray values) and dark areas (areas with lower gray values) will be very different. When a pixel in a bright area and a pixel in a dark area have the same gradient, the total gravitational force generated on the pixel in the dark area is smaller than that of the pixel in the bright area. Therefore, the gravitational method is insensitive to the gradient changes of pixels in some dark areas, resulting in the loss of edge points. For this reason, this embodiment introduces the gravitational field intensity to overcome the difference between bright areas and dark areas. The gravitational field intensity can be calculated by the following formula:

[0075] Where E is the gravitational field strength, m is the gray value of the target pixel, G is the gravitational field number, is the distance between pixel 1 and pixel 2.

[0076] This method replaces the original edge gradient algorithm with the gravitational field strength, and assumes that the total gravitational field strength generated at each point is a combination of the gravitational field strengths generated by the surrounding pixels, so the edge point is the point with the highest gradient strength in the image. The total result field strength for each point can finally be calculated using the following formula:

[0077] In this embodiment, the pixel positions within the 3×3 neighborhood are shown in the following table:

[0078] I[i-1,j+1] I[i,j+1] I[i+1,j+1] I[i-1,j] I[i,j] I[i+1,j] I[i-1,j-1] I[i,j-1] I[i+1,j-1]

[0079] Assume that the distance between two horizontal or vertical pixels is 1, and the distance between two diagonal pixels is At the same time, the gravitational strength of the distant pixel points on the observed pixel point decreases rapidly as the distance increases, so the influence of the gravitational strength of the distant pixel points can be ignored. The gravitational strength in the 3×3 neighborhood can be calculated using the following formula:

[0080]

[0081] in and are the unit vectors in the horizontal and vertical directions respectively, and the new gradient size is:

[0082]

[0083] Let the constant G = 1, and we can get the 3×3 neighborhood operator as:

[0084]

[0085] The structural loss function proposed in this embodiment introduces the similarity loss of the texture feature space, which better captures and preserves the details and texture of the image during the image restoration process, making the restored image more consistent with human visual perception and reducing the over-smoothing effect caused by the mean square error loss. At the same time, it also clarifies the optimization direction of the network, allowing the network training to converge faster. This new loss function is very effective for image processing tasks such as image denoising, reconstruction, and deblurring.

[0086] During the training process, using MSE as the loss function has a faster speed in the early training, but it oscillates repeatedly and converges slowly in the subsequent training process, requiring 40 rounds of training to complete. The PDSNet trained using the structural loss function proposed in this embodiment rises steadily during the training process and has completed convergence after 25 rounds. Both the training speed and accuracy are better than the traditional MSE loss function. By jointly training the denoising network with a new loss function, the learning direction of the network can be determined more effectively, and the training speed of the network can be significantly accelerated while ensuring image quality. In addition, such a network can better preserve the structure of the original image during the training process, making the denoising effect more significant, and the corresponding indicators will also be improved.

[0087] Step S4, inputting the SAR image to be denoised into the trained denoising network PDSNet to obtain the denoised SAR image.

[0088] This SAR image denoising optimization method regards the SAR despeckling problem as a mathematical optimization problem and solves it iteratively through CNN to separate the noise from the image, so that the denoising network becomes a simple discriminant learning problem and can focus on the existing degradation model and prior information in the image. At the same time, in order to make the prior terms better help the network optimization, this method combines the denoiser network and the prior network, and expands the iterative process into a feedforward neural network. Through end-to-end learning, it not only achieves high precision and high computational efficiency of deep learning, but also combines prior knowledge to achieve excellent denoising performance. It overcomes the problem that the traditional CNN-based SAR despeckling model extracts high-dimensional features of the image through the network to restore the clean labeled data ground truth to obtain higher denoising accuracy, but often involves complex optimization problems in the test phase, which leads to a long denoising process of the model and it is difficult to achieve high performance without sacrificing computational efficiency.

[0089] The SAR image denoising optimization method disclosed in this embodiment proposes a deep CNN model PDSNet based on prior drive and structural loss function. By learning the implicit prior knowledge of remote sensing images and adding structural loss functions, the model can more accurately identify the key information in the image, thereby achieving accurate noise removal while retaining more image texture and detail information. In addition, by adding an integral term to the denoising network, the robustness of the network can be effectively guaranteed under a large amount of noise interference. This SAR image denoising optimization method has stronger noise suppression capabilities than existing methods on real images, both in terms of subjective visual quality evaluation and objective evaluation indicators, and can also well preserve the edge information of the original noisy image, which is helpful for applying synthetic aperture radar images to detection, classification and other related tasks.

[0090] In another embodiment, a SAR image denoising optimization system is also disclosed, including a training data acquisition module, a network component module, a loss function improvement module and an image denoising module. The training data acquisition module is used to select sensor image data as training data, grayscale process the training data, and then add multiplicative noise as a training data set for the network. The network component module is used to construct a denoising network PDSNet based on U-Net by combining the neural network structures of DPDNN and U-Net, and the denoising network PDSNet includes a feature encoder and a feature decoder; the feature encoder is composed of multiple convolutional layers and a nonlinear ReLU function, and an attention mechanism module is used to replace the pooling layer of the original U-Net network. The attention mechanism module includes a channel attention module and a spatial attention module, wherein the channel attention module is composed of a maximum pooling layer, an average pooling layer and multiple convolutional layers, and the spatial attention module is composed of a convolutional layer and an activation function; the feature decoder includes multiple convolutional layers grouped into four feature decoders, and an upsampling layer for fusing the feature map of the same spatial resolution generated in the encoding stage and the upsampled feature map generated in the decoding stage to obtain a new upsampled feature map. The loss function improvement module is used to improve the loss function of the denoising network PDSNet. Given N pairs of images The improved structural loss function is expressed as follows:

[0091] Where N is the number of noise-free image pairs, Θ is the network parameter, and x noise Represents the noisy image input to the network, y ground represents ground truth, R(x noise; Θ) represents the reconstructed image obtained by the PDSNet network, and C(·) represents the structural information of the image; the denoising network PDSNet with the improved structural loss function is jointly trained using the training data set. The image denoising module is used to input the SAR image to be denoised into the trained denoising network PDSNet to obtain the denoised SAR image.

[0092] In this embodiment, the feature encoder consists of multiple 3×3 convolutional layers and a nonlinear ReLU function, and a scaling factor of 2 is used in the downsampling layer to downsample along the two axes of the feature map.

[0093] In this embodiment, the loss function improvement module is configured to improve the Canny edge gradient algorithm based on the theory of universal gravitation, replace the original edge gradient algorithm with the gravitational field strength, and assume that the total gravitational field strength generated at each point is a combination of the gravitational field strengths generated by the surrounding pixels. The edge point is the point with the highest gradient strength in the image, and the total result field strength for each point can be calculated by the following formula: Where E is the gravitational field strength, m is the gray value of the target pixel, G is the gravitational field number, is the distance between pixel 1 and pixel 2.

[0094] The specific functions of the above-mentioned SAR image denoising optimization system correspond to the SAR image denoising optimization method disclosed in the previous embodiment, so it will not be described in detail here. For details, please refer to the various embodiments of the SAR image denoising optimization method disclosed in the previous embodiment. It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other.

[0095] In some other embodiments, a SAR image denoising and optimization device is also provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, each step of the SAR image denoising and optimization method described in the above embodiments is implemented.

[0096] The SAR image denoising and optimization device may include, but not limited to, a processor and a memory. The server may include, but not limited to, a processor and a memory. A person skilled in the art may understand that the schematic diagram is only an example of a server and does not constitute a limitation on the server device. The server device may include more or fewer components than shown in the figure, or may combine certain components, or different components. For example, the server device may also include input and output devices, network access devices, buses, etc.

[0097] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the server device, and uses various interfaces and lines to connect various parts of the entire server device.

[0098] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the server device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0099] If the SAR image denoising optimization method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0101] In conclusion, the above is only a preferred embodiment of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the patent of the present invention.

Claims

1. A SAR image denoising optimization method, It is characterized in that The steps include: S1, select the sensor image data as training data, perform grayscale processing on the training data, and then add multiplicative noise as the training data set of the network; S2, by combining the neural network structures of DPDNN and U-Net, a priori-driven feedforward residual denoising network PDSNet based on U-Net is constructed, where the denoising network PDSNet includes a feature encoder and a feature decoder; The feature encoder is composed of multiple convolutional layers and a nonlinear ReLU function, and an attention mechanism module is used to replace the pooling layer of the original U-Net network. The attention mechanism module includes a channel attention module and a spatial attention module, wherein the channel attention module is composed of a maximum pooling layer, an average pooling layer and multiple convolutional layers, and the spatial attention module is composed of a convolutional layer and an activation function; The feature decoder comprises a plurality of convolutional layers grouped into four feature decoders, and an upsampling layer for fusing a feature map of the same spatial resolution generated in the encoding stage with an upsampled feature map generated in the decoding stage to obtain a new upsampled feature map; S3, improve the loss function of the denoising network PDSNet, given N pairs of images The improved structural loss function is expressed as follows: Where N is the number of noise-free image pairs, Θ is the network parameter, and x noise Represents the noisy image input to the network, y ground represents ground truth, R(x noise ; Θ) represents the reconstructed image obtained by the PDSNet network, and C(·) represents the structural information of the image; Use the training dataset to jointly train the denoising network PDSNet that introduces the improved structural loss function; S4, inputting the SAR image to be denoised into the trained denoising network PDSNet to obtain the denoised SAR image.

2. The SAR image denoising optimization method according to claim 1, It is characterized in that The step S2 specifically includes: A feature encoder consisting of multiple 3×3 convolutional layers and a nonlinear ReLU function is used. A scaling factor of 2 is used in the downsampling layer to downsample along both axes of the feature map.

3. The SAR image denoising optimization method according to claim 2, It is characterized in that The step S3 specifically includes: The Canny edge gradient algorithm is improved based on the theory of universal gravitation. The gravitational field strength replaces the original edge gradient algorithm. It is assumed that the total gravitational field strength generated at each point is a combination of the gravitational field strengths generated by the surrounding pixels. The edge point is the point with the highest gradient strength in the image. The total result field strength for each point can be calculated by the following formula: in is the gravitational field strength, m is the gray value of the target pixel, G is the gravitational field number, is the distance vector between pixel 1 and pixel 2.

4. A SAR image denoising optimization system, It is characterized in that include A training data acquisition module is used to select sensor image data as training data, perform grayscale processing on the training data, and then add multiplicative noise as a training data set for the network; A network component module is used to construct a priori-driven feedforward residual denoising network PDSNet based on U-Net by combining the neural network structures of DPDNN and U-Net, wherein the denoising network PDSNet includes a feature encoder and a feature decoder; The feature encoder is composed of multiple convolutional layers and a nonlinear ReLU function, and an attention mechanism module is used to replace the pooling layer of the original U-Net network. The attention mechanism module includes a channel attention module and a spatial attention module, wherein the channel attention module is composed of a maximum pooling layer, an average pooling layer and multiple convolutional layers, and the spatial attention module is composed of a convolutional layer and an activation function; the feature decoder includes multiple convolutional layers grouped into four feature decoders, and an upsampling layer for fusing the feature map of the same spatial resolution generated in the encoding stage and the upsampled feature map generated in the decoding stage to obtain a new upsampled feature map; The loss function improvement module is used to improve the loss function of the denoising network PDSNet. Given N pairs of images The improved structural loss function is expressed as follows: Where N is the number of noise-free image pairs, Θ is the network parameter, and x noise Represents the noisy image input to the network, y ground represents ground truth, R(x noise ; Θ) represents the reconstructed image obtained by the PDSNet network, and C(·) represents the structural information of the image; the denoising network PDSNet with the improved structural loss function is jointly trained using the training dataset; The image denoising module is used to input the SAR image to be denoised into the trained denoising network PDSNet to obtain the denoised SAR image.

5. The SAR image denoising optimization system according to claim 4, Features: The feature encoder consists of multiple 3×3 convolutional layers and a nonlinear ReLU function, with a scaling factor of 2 used in the downsampling layer to downsample along both axes of the feature map.

6. The SAR image denoising optimization system according to claim 5, It is characterized in that The loss function improvement module is configured to improve the Canny edge gradient algorithm based on the theory of universal gravitation, replace the original edge gradient algorithm with the gravitational field strength, and assume that the total gravitational field strength generated at each point is a combination of the gravitational field strengths generated by the surrounding pixels, then the edge point is the point with the highest gradient strength in the image, and the total result field strength for each point can be calculated by the following formula: in is the gravitational field strength, m is the gray value of the target pixel, G is the gravitational field number, is the distance vector between pixel 1 and pixel 2.

7. A SAR image denoising and optimization device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, Features: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

8. A computer-readable storage medium storing a computer program. Features: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.