SAR image automatic optimization method and device

Abstract

The invention discloses an SAR image automatic optimization method and device, and relates to the field of synthetic aperture radar image processing, and the method comprises the steps: carrying out the optimization iteration of an inputted SAR image through a prior extraction module, and inputting an optimization iteration result as prior knowledge into a feature coding module; performing feature extraction, detail enhancement and down-sampling by using a feature coding module to obtain a first feature map; using a feature decoding module to perform up-sampling, feature extraction and detail enhancement on the first feature map passing through the convolutional layer to obtain a second feature map; performing feature fusion on the second feature map by using a feature fusion module, and performing residual connection with an input SAR image to obtain a reconstructed image; and taking the reconstructed image as the input of a prior extraction module, and guiding the convergence of an optimization iterative algorithm based on a structure loss function. According to the method, the problems that the current remote sensing speckle removing network based on deep learning lacks discrimination capability and the restored image details are lost are solved.

Description

Technical Field

[0001] The invention relates to the field of synthetic aperture radar image processing, and in particular to a SAR image automatic optimization method and device. Background Art

[0002] Synthetic aperture radar (SAR) plays an important role in various earth observation tasks such as disaster monitoring, environmental monitoring, and ocean monitoring due to its advantages of multi-band, strong penetration, and all-day and all-weather use. However, due to the coherence of SAR imaging, there are inevitable multiplicative coherent speckle noise and artifacts in SAR images. Speckle noise destroys the edges of targets in SAR images, affecting people's reading of image information and the subsequent application of SAR images, such as target recognition, image segmentation, and scene classification. Despeckle processing of SAR images can restore the overall structural information of SAR images, making it easier for subsequent advanced visual processing tasks to better reveal the biological information of SAR images and the essential characteristics of target categories.

[0003] In order to ensure that the network can learn the characteristic characteristics of the SAR image itself to improve the denoising performance, scholars began to explore the use of multi-temporal remote sensing images and self-supervised algorithms for research. The multi-temporal remote sensing image denoising method integrates images from multiple observation time periods, finds images with similarities, and uses these similar images to eliminate noise. Therefore, the denoising algorithm using multi-temporal remote sensing images can study real SAR images, and is more consistent with the intrinsic information of the SAR image itself in the denoising task. Typical self-supervised denoising algorithms include denoising methods based on sparse representation and denoising methods based on autoencoders. The former represents the SAR image as sparse coefficients to learn the denoising model, while the latter learns the denoising model by training a network. However, in the self-supervised algorithm, due to the presence of contamination information in the SAR image input to the network for training, the network cannot correctly extract the texture detail information in the image, resulting in the repaired image being prone to detail loss and image blur. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a SAR image automatic optimization method and device to solve the problem that the current remote sensing speckle removal network based on deep learning lacks discrimination ability and the problem of loss of image details after restoration.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a SAR image automatic optimization method, comprising the following steps:

[0006] S1: Use the prior extraction module to optimize and iterate the input SAR image, and input the optimization iteration result into the feature encoding module as prior knowledge;

[0007] S2: Using the feature encoding module to perform feature extraction, detail enhancement and downsampling on the optimized iterative SAR image to obtain a first feature map;

[0008] S3: using a feature decoding module to upsample, extract features, and enhance details of the first feature map after the convolution layer to obtain a second feature map;

[0009] S4: using a feature fusion module to perform feature fusion on the second feature map, and performing a residual connection with the input SAR image to obtain a reconstructed image;

[0010] S5: The reconstructed image is used as the input of the prior extraction module, and the optimization iterative algorithm is guided to converge based on the structural loss function to complete the automatic optimization of the SAR image.

[0011] Furthermore, in S1, the prior extraction module is used to optimize and iterate the input SAR image, and a conjugate gradient iteration algorithm is used, and the formula is:

[0012] ;

[0013] in, is the step size coefficient calculated in each iteration, which controls the step size of updating the solution in the current search direction. For the The residual error of the iteration, For the The residual error of the iteration, For the The search direction of the iteration, For the The search direction of the iteration, is a symmetric positive definite matrix, For the The current solution vector of iterations, For the The solution after iterations is For the The conjugate coefficients for the iteration.

[0014] Furthermore, the feature encoding module in S2 includes three sequentially connected feature extraction modules, detail enhancement modules and downsampling modules;

[0015] The feature extraction module includes two 3×3 convolutional layers and a ReLU activation function;

[0016] The detail enhancement module is a DEGA module.

[0017] Further, the DEGA module is represented as:

[0018] ;

[0019] in, is the output of the DEGA module, is the gated weight attention module, is a shallow feature extraction module. It is the output result of the input image after detail enhancement and connection with the activation function. It is the input of DEGA module;

[0020] ;

[0021] in, is the detail prior enhancement module, is the ReLU activation function.

[0022] Furthermore, the detail prior enhancement module is expressed as:

[0023] ;

[0024] ;

[0025] in, is the output of the detail prior enhancement module, To dynamically modify the weight parameters, For detail enhancement convolution, is the weight compression module, is the sigmoid function.

[0026] Furthermore, the gated weighted attention module is expressed as:

[0027] ;

[0028] ;

[0029] in, is the output of the gated weighted attention module, is the gating weight parameter, is the attention module, is the attention module, is the input of the gated weighted attention module, is the sigmoid function.

[0030] Furthermore, the feature decoding module in S3 includes three upsampling modules and feature extraction modules connected in sequence, and the image output by the feature extraction module is jump-connected with the image output by the corresponding DEGA module to achieve detail enhancement.

[0031] Furthermore, the structural loss function in S5 is for:

[0032] ;

[0033] in, are the training parameters of the network, is the number of image samples, For denoised images, is a real image, and To balance the hyperparameters of each loss, is the edge correction term, is the total variation regularization term;

[0034] ;

[0035] in, is the Laplace edge operator, Output image for the network at pixel location The pixel value at The input noisy image at pixel position The pixel value at ;

[0036] ;

[0037] in, Output image for the network at pixel location The pixel value at Output image for the network at pixel location The pixel value at .

[0038] The technical solution also adopted by the present invention is: a device for a SAR image automatic optimization method, characterized in that it comprises:

[0039] Prior extraction module: The prior extraction module optimizes and iterates the input SAR image, and inputs the optimization and iteration results as prior knowledge into the feature encoding module;

[0040] Feature coding module: the feature coding module optimizes the iterative SAR image to perform feature extraction, detail enhancement and downsampling to obtain a first feature map;

[0041] Feature decoding module: The feature decoding module performs upsampling, feature extraction and detail enhancement on the first feature map after the convolution layer to obtain a second feature map;

[0042] Feature fusion module: The feature fusion module performs feature fusion on the second feature map and performs residual connection with the input SAR image to obtain a reconstructed image, uses the reconstructed image as the input of the prior extraction module, and guides the convergence of the optimization iterative algorithm based on the structural loss function.

[0043] The beneficial effects of the present invention are as follows: the present invention combines the traditional model-based method of SAR images with deep learning to form an intelligent and automated optimization method for SAR image despeckle tasks, so that SAR images have image quality that can meet the requirements of subsequent processing in various applications of artificial intelligence, which is conducive to the high-performance work of autonomous unmanned systems such as disaster monitoring, environmental monitoring, and ocean monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of an automatic optimization method for SAR images.

[0045] Figure 2 This is a schematic diagram of the DEGA module structure.

[0046] Figure 3 A framework diagram of an automatic optimization method for SAR images. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0048] Embodiment 1, as Figure 1 As shown, a SAR image automatic optimization method comprises the following steps:

[0049] S1: Use the prior extraction module to optimize and iterate the input SAR image, and input the optimization iteration result into the feature encoding module as prior knowledge;

[0050] S2: Using the feature encoding module to perform feature extraction, detail enhancement and downsampling on the optimized iterative SAR image to obtain a first feature map;

[0051] S3: using a feature decoding module to upsample, extract features, and enhance details of the first feature map after the convolution layer to obtain a second feature map;

[0052] S4: using a feature fusion module to perform feature fusion on the second feature map, and performing a residual connection with the input SAR image to obtain a reconstructed image;

[0053] S5: The reconstructed image is used as the input of the prior extraction module, and the optimization iterative algorithm is guided to converge based on the structural loss function to complete the automatic optimization of the SAR image.

[0054] During the optimization iteration of the input SAR image, the constrained optimization problem can be transformed into two sub-problems, which can be described as:

[0055] ;

[0056] in, For SAR image The solution obtained by the despeckle network is For the The solution obtained by the despeckle network is is a real SAR image, is the degenerate matrix, is the regularization term, is the regularization parameter, which balances the data fitting term during the optimization process, is the regularization parameter, controlling degree of impact.

[0057] The above problem is a quadratic optimization problem. Since the dimension of the noise matrix is ​​usually high and the solution process involves matrix inversion operations, we want to directly calculate Therefore, this scheme proposes a conjugate gradient iteration algorithm (CGIA) to calculate subproblem offers a unique solution. First The sub-problem can be deduced to be equivalent to:

[0058] ;

[0059] in, is the identity matrix, is the gradient derivation of the objective function, which can be further expressed as:

[0060] ;

[0061] First, by initializing the initial solution, the residual vector and the search direction vector, As the initial search direction. Then in each iteration, calculate the step coefficient To update the solution along the search direction , then update the residual , and by calculating the conjugate coefficient To update the new search direction, thus accelerating convergence. The iterative process is repeated continuously, gradually approaching the optimal solution.

[0062] In S1, the prior extraction module is used to optimize and iterate the input SAR image, and the conjugate gradient iteration algorithm is adopted. The formula is:

[0063] ;

[0064] in, is the step size coefficient calculated in each iteration, which controls the step size of updating the solution in the current search direction. For the The residual error of the iteration indicates that The degree of approach to the direction of For the The residual error of the iteration, For the The search direction of the iteration is such that each new direction is related to the previous direction by the matrix Conjugation, For the The search direction of the iteration, is a symmetric positive definite matrix, For the The current solution vector of iterations, For the The solution after iterations is For the The conjugate coefficients for the iteration.

[0065] Compared with the gradient descent method (GNN), the conjugate gradient method converges faster. GNN only uses the current gradient information in each iteration, which may lead to a zigzag path and slow convergence, especially when the condition number is large. The conjugate gradient method constructs the conjugate direction of the Hessian matrix of the objective function and comprehensively utilizes the information of the previous iterations to make the search direction better, move more directly towards the optimal solution, and reduce the number of iterations. In addition, the conjugate gradient method is more accurate and stable in the selection of step size. The gradient descent method is sensitive to the choice of step size. Too large a step size may lead to divergence, and too small a step size may lead to slow convergence.

[0066] Compared with the Newton iteration method, the conjugate gradient method does not need to calculate and store the Hessian matrix, which reduces the computational and memory overhead. Although the Newton method has a quadratic convergence rate in theory, the calculation of the second-order derivative is expensive in high-dimensional problems and may introduce numerical instability. The conjugate gradient method avoids direct operations on the Hessian matrix by iteratively solving a system of linear equations, and is suitable for large sparse systems. In addition, the conjugate gradient method only needs to store a small number of vectors, with lower memory requirements, which is suitable for resource-constrained environments.

[0067] The present invention designs an alternating optimization strategy for the task of SAR image despeckle. In each round of iteration, CGIA processing can provide better preliminary denoising results under a fixed weight, while the neural network can make up for some image details lost in the despeckle process by the iterative algorithm. Through collaborative optimization, the stability and robustness of denoising are enhanced, making the final despeckle effect more natural and rich in details.

[0068] The feature encoding module in S2 includes three sequentially connected feature extraction modules, detail enhancement modules and downsampling modules;

[0069] The feature extraction module includes two 3×3 convolutional layers and a ReLU activation function;

[0070] The detail enhancement module is a DEGA module.

[0071] In the SAR image despeckling task, the design of the model mainly focuses on how to effectively retain the details and texture information of the image. Some traditional deep learning-based methods are committed to improving the network's ability to extract potential features by increasing the depth and width of the convolutional layer, and the learning ability of its structure remains to be explored. It has been proposed in the prior art to use DEConv to solve the problem that the normal convolutional layer searches a huge solution space without constraints, but the method of directly adding multiple different convolution weights will obviously introduce instability in the gradient descent process, resulting in unclear gradient update direction. Based on this, the present invention designs a DEGA module to stabilize the gradient descent process while further retaining the detail information, such as Figure 2 The implementation of the DEGA module is as follows.

[0072] The DEGA module is represented as:

[0073] ;

[0074] in, is the output of the DEGA module, is the gated weight attention module, is a shallow feature extraction module. It is the output result of the input image after detail enhancement and connection with the activation function. It is the input of DEGA module;

[0075] ;

[0076] in, is the detail prior enhancement module, is the ReLU activation function.

[0077] The detail prior enhancement module is expressed as:

[0078] ;

[0079] ;

[0080] in, is the output of the detail prior enhancement module, In order to dynamically correct the weight parameters, the feature information conflict caused by the direct addition of weights in DEConv can be adjusted in time to avoid detail loss and improve the convergence speed and effect of the model. For detail enhancement convolution, is the weight compression module, is the sigmoid function.

[0081] The gated weighted attention module is expressed as:

[0082] ;

[0083] ;

[0084] in, is the output of the gated weighted attention module, is the gate weight parameter, dynamically adjusts the weight of each feature, and extracts the potential features of the image. is the attention module, is the attention module, is the input of the gated weighted attention module, is the sigmoid function.

[0085] The feature decoding module in S3 includes three upsampling modules and feature extraction modules connected in sequence. The image output by the feature extraction module is jump-connected with the image output by the corresponding DEGA module to achieve detail enhancement.

[0086] In the image denoising task, the choice of loss function plays a crucial role in the quality of the network reconstructed image. Usually, Mean Squared Error (MSE) is widely used in the training process of denoising networks because it has the advantages of simple calculation and smooth gradient, and can effectively narrow the gap between the predicted image and the target image in most cases. However, studies have shown that using MSE alone as a loss function has obvious limitations. Specifically, MSE tends to minimize the global error, which may lead to over-smoothing of certain local areas of the image, thereby losing important detail features, especially in preserving edge and texture information.

[0087] Edge information is crucial to the perceived quality and visual effects of images because it usually contains the contours, structures, and detailed features of objects. In synthetic aperture radar (SAR) images, edge and texture information is the key to identifying ground features. Therefore, in the process of image denoising, how to effectively preserve and reconstruct edge information becomes a major challenge. To solve this problem, recent studies have attempted to introduce edge protection mechanisms into the loss function to enhance the network's sensitivity to detailed features. For example, some works use perceptual loss or adversarial loss to emphasize high-frequency details, but these methods may increase the complexity of the model and the training process is unstable.

[0088] Based on the above considerations, this study proposes a composite loss function that combines edge correction terms and regularization terms on the basis of the traditional MSE loss function. This loss function aims to make up for the shortcomings of MSE in edge and detail preservation, while preventing the network from overfitting noise patterns. Specifically, the present invention introduces an edge correction term based on the Laplace operator to enhance the network's attention to edge information. In addition, in order to suppress the impact of noise on model training, the present invention adds a total variation regularization term to promote the generated image to be smoother and more continuous.

[0089] The structural loss function in S5 for:

[0090] ;

[0091] in, are the training parameters of the network, is the number of image samples, For denoised images, is a real image, and To balance the hyperparameters of each loss, is the edge correction term, is the total variation regularization term;

[0092] ;

[0093] in, is the Laplace edge operator, Output image for the network at pixel location The pixel value at The input noisy image at pixel position The pixel value at ;

[0094] ;

[0095] in, Output image for the network at pixel location The pixel value at Output image for the network at pixel location The pixel value at .

[0096] The edge correction term calculates the difference in edge response between the denoised image and the noisy image, encouraging the network to retain and reconstruct edge details while denoising. This term allows the model to pay more attention to high-frequency information during training, thereby avoiding the over-smoothing problem caused by MSE loss. The total variation regularization term suppresses the discontinuity caused by noise by penalizing the image gradient, promoting local smoothness of the image without blurring important edge features.

[0097] The edge correction loss function proposed in the present invention can not only significantly improve the network's noise suppression effect, but also effectively retain important features in the image, especially edge details, thereby improving the overall quality of the image. By combining the edge features of the image, the edge correction term can better guide the network to retain the characteristic information of the image while denoising, reducing the over-smoothing effect caused by MSE.

[0098] Combining the optimization algorithm and DEGA module, the present invention proposes PDSNet based on the CGIA iterative algorithm and the alternating optimization of the SAR image denoising network. The structure of the model is as follows Figure 3 As shown, inspired by semantic segmentation and target segmentation, the present invention designs a SAR despeckling network based on U-Net, which is divided into four parts: prior extraction, feature encoding, feature decoding and feature fusion. Although the model-based optimization algorithm has been decoupled into two sub-problems, it requires multiple iterations to obtain the final result, which requires a lot of computational cost. Based on the above content, the present invention first performs an iterative optimization of the algorithm on the input image before feature encoding, and uses the obtained information as prior knowledge to assist in guiding the network to learn image features, reduce the interference of random spots in SAR images on the network, and improve the network's ability to learn image details. In the image encoding part, the feature encoder consists of three feature extraction, detail enhancement and downsampling modules. The feature extraction part is a combination of two 3×3 convolutional layers and a ReLU activation function. The image after feature extraction will be enhanced in detail, that is, the DEGA module, and its specific details can be found in. Figure 2 Finally, feature mapping is performed through a downsampling layer with a scale factor of 0.5. Repeat the above operation three times to obtain a feature map with a size of 1 / 8 of the original. This part aims to extract the original features of the input image while enhancing the details of the image to provide more effective feature information for feature decoding.

[0099] In order to obtain the potential features of the image, the output of the image feature encoding will first pass through a 3×3 convolution layer before entering the image decoding layer. Similar to image encoding, this part will first perform an upsampling layer with a scale factor of 2, followed by feature extraction, and the resulting image will be jump-connected with the image obtained by the corresponding feature enhancement layer, and the details will be enhanced. The above operation is repeated three times, and finally, the features are fused through a 1×1 convolution layer and connected to the input image residual to obtain a more fine-grained reconstructed image. The reconstructed image obtained by the neural network as a denoiser will be used as the input of the CGIA iterative algorithm, thereby assisting in guiding the algorithm to converge quickly and realizing the alternating filtering operations performed by the network and iterative algorithm.

[0100] Embodiment 2, a device for a SAR image automatic optimization method, characterized by comprising:

[0101] Prior extraction module: The prior extraction module optimizes and iterates the input SAR image, and inputs the optimization and iteration results as prior knowledge into the feature encoding module;

[0102] Feature coding module: the feature coding module optimizes the iterative SAR image to perform feature extraction, detail enhancement and downsampling to obtain a first feature map;

[0103] Feature decoding module: The feature decoding module performs upsampling, feature extraction and detail enhancement on the first feature map after the convolution layer to obtain a second feature map;

[0104] Feature fusion module: The feature fusion module performs feature fusion on the second feature map and performs residual connection with the input SAR image to obtain a reconstructed image, uses the reconstructed image as the input of the prior extraction module, and guides the convergence of the optimization iterative algorithm based on the structural loss function.

[0105] The present invention proposes a model PDSNet based on prior drive and structural loss function, so that a new SAR image filter or functional module can be formed based on this method. The method model can more accurately identify the key information in the image by learning the implicit prior knowledge of the remote sensing image and adding the structural loss function, thereby achieving accurate noise removal while retaining more image texture and detail information. In addition, the present invention adds an integral term to the denoising network, so that the robustness of the network can be effectively guaranteed under a large amount of noise interference.

[0106] The method proposed by the present invention has good visual effects and objective evaluation indicators in suppressing speckle noise in SAR images and noise in real SAR images. Therefore, the method disclosed by the present invention is helpful to apply synthetic aperture radar images to detection, classification and other related work.

[0107] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the invention.

Claims

1. A SAR image automatic optimization method, characterized in that: The following steps are involved: S1: Use the prior extraction module to optimize and iterate the input SAR image, and input the optimization iteration result into the feature encoding module as prior knowledge; S2: Using the feature encoding module to perform feature extraction, detail enhancement and downsampling on the optimized iterative SAR image to obtain a first feature map; S3: using a feature decoding module to upsample, extract features, and enhance details of the first feature map after the convolution layer to obtain a second feature map; S4: using a feature fusion module to perform feature fusion on the second feature map, and performing a residual connection with the input SAR image to obtain a reconstructed image; S5: The reconstructed image is used as the input of the prior extraction module, and the optimization iterative algorithm is guided to converge based on the structural loss function to complete the automatic optimization of the SAR image.

2. The SAR image automatic optimization method according to claim 1, characterized in that: In S1, the prior extraction module is used to optimize and iterate the input SAR image, and the conjugate gradient iteration algorithm is adopted. The formula is: ; in, is the step size coefficient calculated in each iteration, which controls the step size of updating the solution in the current search direction. For the The residual error of the iteration, For the The residual error of the iteration, For the The search direction of the iteration, For the The search direction of the iteration, is a symmetric positive definite matrix, For the The current solution vector of iterations, For the The solution after iterations is For the The conjugate coefficient for iteration .

3. The SAR image automatic optimization method according to claim 1, characterized in that: The feature encoding module in S2 includes three sequentially connected feature extraction modules, detail enhancement modules and downsampling modules; The feature extraction module includes two 3×3 convolutional layers and a ReLU activation function; The detail enhancement module is a DEGA module.

4. The SAR image automatic optimization method according to claim 3, characterized in that: The DEGA module is represented as: ； in, is the output of the DEGA module, is the gated weight attention module, is a shallow feature extraction module. It is the output result of the input image after detail enhancement and connection with the activation function. It is the input of DEGA module; ； in, is the detail prior enhancement module, is the ReLU activation function.

5. The SAR image automatic optimization method according to claim 4, characterized in that: The detail prior enhancement module is expressed as: ； ； in, is the output of the detail prior enhancement module, To dynamically modify the weight parameters, For detail enhancement convolution, is the weight compression module, is the sigmoid function.

6. The SAR image automatic optimization method according to claim 5, characterized in that: The gated weighted attention module is expressed as: ； ； in, is the output of the gated weighted attention module, is the gating weight parameter, is the attention module, is the attention module, is the input of the gated weighted attention module, is the sigmoid function.

7. The SAR image automatic optimization method according to claim 6, characterized in that: The feature decoding module in S3 includes three upsampling modules and feature extraction modules connected in sequence. The image output by the feature extraction module is jump-connected with the image output by the corresponding DEGA module to achieve detail enhancement.

8. The SAR image automatic optimization method according to claim 1, characterized in that: The structural loss function in S5 for: ； in, are the training parameters of the network, is the number of image samples, For denoised images, is a real image, and To balance the hyperparameters of each loss, is the edge correction term, is the total variation regularization term; ； in, is the Laplace edge operator, Output image for the network at pixel location The pixel value at The input noisy image at pixel position The pixel value at ; ； in, Output image for the network at pixel location The pixel value at Output image for the network at pixel location The pixel value at .

9. A device for the SAR image automatic optimization method according to any one of claims 1 to 8, characterized in that: include: Prior extraction module: The prior extraction module optimizes and iterates the input SAR image, and inputs the optimization and iteration results as prior knowledge into the feature encoding module; Feature coding module: the feature coding module optimizes the iterative SAR image to perform feature extraction, detail enhancement and downsampling to obtain a first feature map; Feature decoding module: The feature decoding module performs upsampling, feature extraction and detail enhancement on the first feature map after the convolution layer to obtain a second feature map; Feature fusion module: The feature fusion module performs feature fusion on the second feature map and performs residual connection with the input SAR image to obtain a reconstructed image, uses the reconstructed image as the input of the prior extraction module, and guides the convergence of the optimization iterative algorithm based on the structural loss function.

Images