A method and apparatus for automatic optimization of SAR images

Abstract

The application discloses a kind of SAR image automatic optimization method and device, it is related to synthetic aperture radar image processing field, method includes: using prior extraction module to the input SAR image is optimized iteration, and optimization iteration result is input as prior knowledge to feature coding module;Using feature coding module carries out feature extraction, detail enhancement and down-sampling, obtains first feature map;Using feature decoding module is after first feature map to the convolution layer is up-sampled, feature extraction and detail enhancement are carried out, and second feature map is obtained;Using feature fusion module carries out feature fusion to second feature map, and with the residual connection of input SAR image, obtains reconstruction image;Reconstruction image is as the input of prior extraction module, and based on structure loss function guides optimization iteration algorithm convergence.The application solves the problem that current deep learning-based remote sensing despeckling network lacks discriminant ability and the details of repaired image are lost.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of synthetic aperture radar image processing, in particular to a SAR image automatic optimization method and device. BACKGROUND

[0002] Synthetic aperture radar (SAR) plays an important role in various tasks of earth observation such as disaster monitoring, environmental monitoring, ocean monitoring, etc. due to its advantages of multi-band, strong penetration and all-weather use. However, due to the coherence of SAR imaging, there are unavoidable multiplicative speckle noise and artifacts in SAR images. Speckle noise destroys the edges of targets in SAR images, affecting the reading of image information and the application of SAR images in later stages, such as target recognition, image segmentation and scene classification. De-speckling processing of SAR images can restore the overall structural information of SAR images, making it easier for subsequent advanced visual processing tasks to better reveal the biological information and essential characteristics of target categories of SAR images.

[0003] In order to ensure that the network can learn the characteristics of the SAR image itself to improve the denoising performance, scholars have begun to explore the use of multi-temporal remote sensing images and self-supervised algorithms for research. The multi-temporal remote sensing image denoising method integrates images from multiple observation time periods together, finds images with similarities among them, and uses these similar images to eliminate noise. Therefore, the denoising algorithm using multi-temporal remote sensing images can study real SAR images, and is more consistent with the intrinsic information of SAR images in the denoising task. Typical self-supervised denoising algorithms include sparse representation-based denoising methods and autoencoder-based denoising methods. The former represents SAR images as sparse coefficients to learn a denoising model, while the latter trains a network to learn a denoising model. However, in the self-supervised algorithm, due to the presence of contaminated information in the SAR images input into the network for training, the network cannot correctly extract the texture detail information in the images, resulting in the repaired images being prone to detail loss and image blurring problems. SUMMARY

[0004] In view of the above deficiencies in the prior art, the SAR image automatic optimization method and device provided by the present application solve the problems of lack of discrimination ability in the current deep learning-based remote sensing de-speckling network and loss of details in the repaired images.

[0005] In order to achieve the above-mentioned application purposes, the technical scheme adopted by the present application is as follows: a SAR image automatic optimization method, comprising the following steps:

[0006] S1: using a prior extraction module to optimize and iterate the input SAR image, and inputting the optimization and iteration result as prior knowledge into a feature encoding module;

[0007] S2: feature extraction, detail enhancement and down-sampling are performed on the SAR image after optimization iteration by using a feature encoding module to obtain a first feature map;

[0008] S3: up-sampling, feature extraction and detail enhancement are performed on the first feature map after convolution layer by using a feature decoding module to obtain a second feature map;

[0009] S4: feature fusion is performed on the second feature map by using a feature fusion module, and residual connection is performed with the input SAR image to obtain a reconstructed image;

[0010] S5: the reconstructed image is taken as the input of the prior extraction module, and the optimization iteration algorithm is guided to converge based on a structure loss function, and SAR image automatic optimization is completed.

[0011] Further, the prior extraction module is used to perform optimization iteration on the input SAR image in S1, and a conjugate gradient iteration algorithm is adopted, and the formula is:

[0012] ;

[0013] wherein, is a step length coefficient calculated in each iteration, which controls the step size of updating the solution in the current search direction, is the residual error of the i th iteration, is the residual error of the i th iteration, is the search direction of the i th iteration, is the search direction of the i th iteration, is a symmetric positive definite matrix, is the current solution vector of the i th iteration, is the solution after the i th iteration, is the conjugate coefficient of the i th iteration.

[0014] Further, the feature encoding module in S2 includes three sequentially connected feature extraction modules, detail enhancement modules and down-sampling modules;

[0015] The feature extraction module includes two 3x3 convolution layers and ReLU activation functions;

[0016] The detail enhancement module is a DEGA module.

[0017] Further, the DEGA module is represented as:

[0018] ;​​​​​​​

[0019] wherein, is the output of the DEGA module, is the gated weight attention module, is the shallow feature extraction module, is the output result of the input image after detail enhancement and the activation function connection, is the input of the DEGA module;

[0020] ;

[0021] wherein, is the detail prior enhancement module, is the ReLU activation function.

[0022] Further, the detail prior enhancement module is expressed as:

[0023] ;

[0024] ;

[0025] wherein, is the output of the detail prior enhancement module, is the dynamic correction weight parameter, is the detail enhancement convolution, is the weight compression module, is the sigmoid function.

[0026] Further, the gated weight attention module is expressed as:

[0027] ;

[0028] ;

[0029] wherein, is the output of the gated weight attention module, is the gated weight parameter, is the attention module, is the attention module, is the input of the gated weight attention module, is the sigmoid function.

[0030] Further, the feature decoding module in S3 includes three sequentially connected up-sampling modules and feature extraction modules, and the image output by the feature extraction module is connected with the image output by the corresponding DEGA module to realize detail enhancement.

[0031] Further, the structure loss function in S5 is For:

[0032] ;

[0033] wherein, is a training parameter of the network, is a number of image samples, is a denoised image, is a real image, and is a hyperparameter for balancing the losses, is an edge correction term, is a total variation regularization term;

[0034] ;

[0035] wherein, is a Laplace edge operator, is a pixel value of the network output image at pixel position , is a pixel value of the input noisy image at pixel position ;

[0036] ;

[0037] wherein, is a pixel value of the network output image at pixel position , is a pixel value of the network output image at pixel position .

[0038] The technical scheme also adopted by the application is: a device of a SAR image automatic optimization method, characterized by comprising:

[0039] A priori extraction module: the a priori extraction module optimizes and iterates the input SAR image, and inputs the optimization and iteration result as a priori knowledge to a feature coding module;

[0040] Feature coding module: the feature coding module optimizes and iterates the SAR image to obtain a first feature map through feature extraction, detail enhancement and down-sampling;

[0041] Feature decoding module: the feature decoding module performs up-sampling, feature extraction and detail enhancement on the first feature map after the convolution layer to obtain a second feature map;

[0042] Feature fusion module: the feature fusion module performs feature fusion on the second feature map, and performs residual connection with the input SAR image to obtain a reconstructed image, takes the reconstructed image as the input of the a priori extraction module, and guides the convergence of the optimization and iteration algorithm based on a structural loss function.

[0043] The present application combines the traditional model-based method of SAR image with deep learning to form an intelligent and automated optimization method for SAR image despeckling task. The SAR image has image quality that can meet the requirements of subsequent processing in various applications of artificial intelligence, which is beneficial to the development of high-performance autonomous unmanned systems for disaster monitoring, environmental monitoring, ocean monitoring, etc. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 It is a flow chart of a SAR image automatic optimization method.

[0045] Figure 2 It is a schematic diagram of DEGA module structure.

[0046] Figure 3 It is a framework diagram of a SAR image automatic optimization method. DETAILED DESCRIPTION

[0047] The present application will be further described below in combination with the drawings and specific embodiments.

[0048] Embodiment 1, as shown, a SAR image automatic optimization method, comprising the following steps: Figure 1

[0049] S1: using a priori extraction module to optimize and iterate the input SAR image, and inputting the optimization and iteration result as priori knowledge to a feature encoding module;

[0050] S2: using the feature encoding module to perform feature extraction, detail enhancement and down-sampling on the SAR image after optimization and iteration, to obtain a first feature map;

[0051] S3: using the feature decoding module to perform up-sampling, feature extraction and detail enhancement on the first feature map after convolution layer, to obtain a second feature map;

[0052] S4: using the feature fusion module to perform feature fusion on the second feature map, and connecting in residual with the input SAR image, to obtain a reconstructed image;

[0053] S5: taking the reconstructed image as the input of the priori extraction module, and guiding the optimization and iteration algorithm to converge based on a structure loss function, to complete the SAR image automatic optimization.

[0054] In the process of optimizing and iterating the input SAR image, the constrained optimization problem can be converted into two sub-problems, which can be described as:

[0055] ;

[0056] wherein, is the SAR image​ The solution obtained by the despeckle network is For the The solution obtained by the despeckle network is is a real SAR image, is the degenerate matrix, is the regularization term, is the regularization parameter, which balances the data fitting terms during the optimization process, is the regularization parameter, controlling degree of impact.

[0057] The above problem is a quadratic optimization problem. Since the dimension of the noise matrix is ​​usually high and the solution involves matrix inversion operations, we want to directly calculate Therefore, this scheme proposes a conjugate gradient iteration algorithm (CGIA) to calculate The subproblem offers a unique solution. First The subproblem can be deduced to be equivalent to:

[0058] ;

[0059] in, is the identity matrix, is the gradient derivation of the objective function, which can be further expressed as:

[0060] ;

[0061] First, by initializing the initial solution, the residual vector and the search direction vector, with As the initial search direction. Then in each iteration, calculate the step coefficient To update the solution along the search direction , then update the residual , and calculate the conjugate coefficient by To update the new search direction, thus accelerating convergence. The iterative process is repeated continuously, gradually approaching the optimal solution.

[0062] In S1, the prior extraction module is used to optimize and iterate the input SAR image using a conjugate gradient iterative algorithm. The formula is:

[0063] ;

[0064] in, The step size coefficient calculated in each iteration controls the step size of updating the solution in the current search direction. For the The residual error of the iteration indicates that The degree of approach to the direction of is the residual of the first iteration, is the search direction of the first iteration, is the residual of the first iteration, is the search direction of the first iteration, such that each new direction is conjugate to the previous direction with respect to the matrix is the search direction of the first iteration, is the search direction of the first iteration, is the search direction of the first iteration, is a symmetric positive definite matrix, is the current solution vector of the first iteration, is the solution after the first iteration, is the solution after the first iteration, is the solution after the first iteration, is the solution after the first iteration, is the conjugate coefficient of the first iteration.

[0065] Compared with the gradient descent method (GNN), the conjugate gradient method has faster convergence speed. GNN only uses the current gradient information in each iteration, which may lead to a zigzag path, slow convergence, especially in the case of large condition number. The conjugate gradient method uses the information of the previous iterations by constructing the conjugate direction with respect to the Hessian matrix of the objective function, making the search direction more optimal and enabling the method to directly approach the optimal solution, reducing the number of iterations. In addition, the conjugate gradient method is more accurate and stable in step size selection. The gradient descent method is sensitive to the selection of step size, which may lead to divergence if the step size is too large, or slow convergence if the step size is too small.

[0066] Compared with the Newton iteration method, the conjugate gradient method does not need to calculate and store the Hessian matrix, reducing the computational and memory overhead. Although the Newton method has a quadratic convergence speed in theory, the calculation of the second-order derivative is expensive in high-dimensional problems, and may introduce numerical instability. The conjugate gradient method solves linear equations by iteration, avoiding direct manipulation of the Hessian matrix, and is suitable for large sparse systems. In addition, the conjugate gradient method only needs to store a small number of vectors, requiring less memory, and is suitable for resource-limited environments.

[0067] The present application designs an alternating optimization strategy for the task of SAR image despeckling. In each iteration, the CGIA process can provide better preliminary denoising results under a fixed weight, while the neural network can make up for the loss of image details in the despeckling process. Through collaborative optimization, the stability and robustness of denoising are enhanced, making the final despeckling effect more natural and detailed.

[0068] The feature encoding module in S2 includes three sequentially connected feature extraction modules, detail enhancement modules and down-sampling modules;

[0069] The feature extraction module includes two 3x3 convolution layers and ReLU activation functions;

[0070] The detail enhancement module is a DEGA module.

[0071] In the task of SAR image debanding, the design of the model mainly focuses on how to effectively preserve the details and texture information of the image. Some traditional deep learning-based methods strive to improve the ability of the network to extract potential features by increasing the depth and width of the convolutional layer, and the learning ability of its structure remains to be explored. In the prior art, DEConv is used to solve the problem that normal convolutional layers search a huge solution space without constraints, but the direct addition of multiple different convolutional weights obviously introduces instability in the gradient descent process, resulting in an unclear gradient update direction. Based on this, the present application designs a DEGA module, which allows the network to further preserve detail information while stabilizing the gradient descent process, as shown in Figure 2 The implementation of the DEGA module is as follows.

[0072] The DEGA module is represented as:

[0073] ;

[0074] Among them, is the output of the DEGA module, is a gating weight attention module, is a shallow feature extraction module, is the output result of the connection of the detail enhancement of the input image and the activation function, is the input of the DEGA module;

[0075] ;

[0076] Among them, is a detail prior enhancement module, is a ReLU activation function.

[0077] The detail prior enhancement module is represented as:

[0078] ;

[0079] ;

[0080] Among them, is the output of the detail prior enhancement module, is a dynamic correction weight parameter, which can adjust the feature information conflict caused by the direct addition of weights in DEConv in a timely manner, avoid detail loss, and improve the convergence speed and effect of the model, is a detail enhancement convolution, is a weight compression module, is a sigmoid function.

[0081] The gating weight attention module is represented as:

[0082] ;

[0083] ;

[0084] wherein, is the output of the gating weight attention module, is a gating weight parameter that dynamically adjusts the weight of each feature and extracts the latent features of the image, is an attention module, is an attention module, is the input of the gating weight attention module, is a sigmoid function.

[0085] The feature decoding module in S3 includes three sequentially connected up-sampling modules and a feature extraction module. The image output by the feature extraction module is connected to the image output by the corresponding DEGA module to achieve detail enhancement.

[0086] In the image denoising task, the choice of loss function plays a crucial role in the quality of the reconstructed image by the network. Under normal circumstances, Mean Squared Error (MSE) is widely used in the training process of denoising networks because of its simplicity in calculation, smoothness in gradient, and other advantages, and it can effectively reduce the gap between the predicted image and the target image in most cases. However, research shows that using MSE alone as a loss function has obvious limitations. Specifically, MSE tends to minimize global error, which can lead to excessive smoothing of some local regions of the image, thereby losing important detail features, especially in the preservation of edge and texture information.

[0087] Edge information is crucial for the perceptual quality and visual effect of an image, as it often contains the outlines, structures, and detail features of objects. In synthetic aperture radar (SAR) images, edge and texture information are even more critical for identifying ground features. Therefore, in the image denoising process, how to effectively preserve and reconstruct edge information has become a major challenge. To address this issue, recent research has attempted to introduce edge protection mechanisms into the loss function to improve the sensitivity of the network to detail features. For example, some works use perceptual loss or adversarial loss to emphasize high-frequency details, but these methods can increase the complexity of the model and make the training process unstable.

[0088] Based on the above considerations, this study proposes a composite loss function that combines an edge correction term and a regularization term, building on the traditional MSE loss function. This loss function aims to compensate for the shortcomings of MSE in preserving edges and details, while preventing the network from overfitting to noisy patterns. Specifically, this paper introduces an edge correction term based on the Laplacian operator to enhance the network's focus on edge information. Furthermore, to suppress the impact of noise on model training, this paper adds a total variation regularization term to promote smoother and more continuous generated images.

[0089] The structural loss function in S5 for:

[0090] ;

[0091] in, are the training parameters of the network, is the number of image samples, For denoised images, is a real image, and To balance the hyperparameters of each loss, is the edge correction term, is the total variation regularization term;

[0092] ;

[0093] in, is the Laplace edge operator, Output image for the network at pixel location The pixel value at For the input noisy image at pixel position The pixel value at ;

[0094] ;

[0095] in, Output image for the network at pixel location The pixel value at Output image for the network at pixel location The pixel value at .

[0096] The edge correction term calculates the difference in edge response between the denoised image and the noisy image, encouraging the network to preserve and reconstruct edge details while denoising. This term allows the model to focus more on high-frequency information during training, thus avoiding the oversmoothing problem caused by the MSE loss. The total variation regularization term, by penalizing image gradients, suppresses discontinuities caused by noise and promotes local smoothness in the image without blurring important edge features.

[0097] The edge correction loss function provided by the present application can not only significantly improve the noise suppression effect of the network, but also effectively retain important features in the image, especially edge details, thereby improving the overall quality of the image. The edge correction term can better guide the network to retain feature information of the image while denoising, and reduce the over-smoothing effect caused by MSE, by combining the edge features of the image.

[0098] In combination with the optimization algorithm and the DEGA module, the present application provides a PDSNet based on the CGIA iterative algorithm and the SAR image denoising network alternating optimization. The structure of the model is shown in Figure 3 Based on the model optimization algorithm, although the algorithm has been decoupled into two sub-problems, a large amount of calculation cost is required for multiple iterations to obtain the final result. Based on the above, the present application performs an iterative optimization of the input image before feature encoding, and uses the obtained information as prior knowledge to assist the network in learning image features, thereby reducing the interference of random speckles in the SAR image on the network and improving the network's ability to learn image details. In the image encoding part, the feature encoder is composed of three feature extraction, detail enhancement and down-sampling modules. The feature extraction part is composed of two 3x3 convolution layers and ReLU activation function combination, respectively. After feature extraction, the image will be enhanced in detail, that is, the DEGA module, for details, refer to Figure 2 Finally, the feature mapping is performed through the down-sampling layer with a scale factor of 0.5. The above operation is repeated three times to obtain a feature map with a size of 1 / 8 of the original image. This part aims to extract the original features of the input image while enhancing the details of the image to provide more effective feature information for feature decoding.

[0099] In order to obtain the latent features of the image, the output of the image feature encoding will first pass through a 3x3 convolution layer and then enter the image decoding layer. Similar to image encoding, this part will first perform an up-sampling layer with a scale factor of 2, then perform feature extraction, and then perform a jump connection between the obtained image and the image obtained by the corresponding feature enhancement layer, and then perform detail enhancement. The above operation is repeated three times, and finally a 1x1 convolution layer is used for feature fusion and residual connection with the input image to obtain a reconstructed image with finer granularity. The reconstructed image obtained by the neural network as the denoiser will be used as the input of the CGIA iterative algorithm, thereby assisting the algorithm to converge quickly and realizing the alternating filtering operation of the network and the iterative algorithm.

[0100] Embodiment 2, a device for automatic optimization of SAR images, characterized in that it comprises:

[0101] Priori extraction module: the priori extraction module optimizes iteration on the input SAR image, and inputs the optimization iteration result as priori knowledge to the feature coding module;

[0102] Feature coding module: the feature coding module optimizes iteration on the SAR image to obtain the first feature map through feature extraction, detail enhancement and down sampling;

[0103] Feature decoding module: the feature decoding module performs up sampling, feature extraction and detail enhancement on the first feature map after convolution layer to obtain the second feature map;

[0104] Feature fusion module: the feature fusion module performs feature fusion on the second feature map, and performs residual connection with the input SAR image to obtain a reconstructed image, takes the reconstructed image as the input of the priori extraction module, and guides the optimization iteration algorithm to converge based on the structural loss function.

[0105] The present application proposes a model PDSNet based on priori driving and structural loss function, so that a new SAR image filter or functional module can be formed based on the method. The method model can more accurately identify the key information in the image by learning the implicit priori knowledge of remote sensing image and adding the structural loss function, so as to realize accurate noise removal while retaining more image texture and detail information. In addition, the present application adds an integral term in the denoising network, so that the robustness of the network can be effectively guaranteed under a large amount of noise interference.

[0106] The method proposed in the present application has good visual effect and objective evaluation index in the aspect of SAR image speckle noise and real SAR image noise suppression. Therefore, the method disclosed in the present application is helpful to apply synthetic aperture radar image to detection, classification and other related work.

[0107] Those skilled in the art will appreciate that the embodiments described herein are intended to aid the reader in understanding the principles of the present application and should not be understood to limit the scope of the present application to such specific embodiments and examples. Those skilled in the art can make various other specific modifications and combinations according to the technical inspiration disclosed in the present application without departing from the spirit of the present application, and these modifications and combinations are still within the scope of protection of the present application.

Claims

1. A method for automatic optimization of SAR images, characterized in that, The method comprises the following steps: S1: using a prior extraction module to perform optimization iteration on an input SAR image, and inputting the optimization iteration result as prior knowledge into a feature encoding module; S2: using the feature encoding module to perform feature extraction, detail enhancement and down-sampling on the SAR image after optimization iteration, to obtain a first feature map; The feature encoding module in S2 comprises three sequentially connected feature extraction modules, a detail enhancement module and a down-sampling module; The feature extraction module comprises two 3*3 convolution layers and a ReLU activation function; The detail enhancement module is a DEGA module; The DEGA module is expressed as: ； wherein, is the output of the DEGA module, is the gated weight attention module, is the shallow feature extraction module, is the output result of the connection of the detail enhancement on the input image and the activation function, is the input of the DEGA module; ； wherein, is a details prior enhancement module, is a ReLU activation function, is an input to the DEGA module; S3: using a feature decoding module to perform up-sampling, feature extraction and detail enhancement on the first feature map after convolution, to obtain a second feature map; S4: using a feature fusion module to perform feature fusion on the second feature map, and performing residual connection with the input SAR image, to obtain a reconstructed image; S5: taking the reconstructed image as the input of the prior extraction module, and guiding the optimization iteration algorithm to converge based on a structural loss function, to complete automatic optimization of the SAR image.

2. The method of claim 1, wherein, In S1, the conjugate gradient iteration algorithm is adopted to perform optimization iteration on the input SAR image, and the formula is: ； in, The step size coefficient calculated in each iteration controls the step size of updating the solution in the current search direction. For the The residual error of the iteration, For the The residual error of the iteration, For the The search direction of the iteration, For the The search direction of the iteration, is a symmetric positive definite matrix, For the The current solution vector of the iteration, For the The solution after iterations is For the The conjugate coefficient for the iteration.

3. The method of claim 1, wherein, The detail prior enhancement module is expressed as: ； ； wherein, is the output of the detail prior enhancement module, is the dynamic correction weight parameter, is the detail enhancement convolution, is the weight compression module, is the sigmoid function, is the input of the DEGA module.

4. The method of claim 1, wherein, The gating weight attention module is expressed as: ； ； wherein, is an output of a gated weight attention module, is a gated weight parameter, is an attention module, is an attention module, is an input of a gated weight attention module, is a sigmoid function.

5. The method of claim 1, wherein, The feature decoding module in S3 comprises three sequentially connected up-sampling modules and feature extraction modules, and the image output by the feature extraction module is connected with the image output by the corresponding DEGA module in a skip connection manner, to realize detail enhancement.

6. The method of claim 1, wherein, The structure loss function in S5 is: is: ； wherein, is a training parameter of the network, is a number of image samples, is a denoised image, is a real image, and is a hyperparameter balancing the losses, is an edge correction term, is a total variation regularization term; ； wherein is a Laplacian edge operator, is a pixel value of the network output image at pixel position , is a pixel value of the input noisy image at pixel position ; ； wherein, is the pixel value of the network output image at pixel position , is the pixel value of the network output image at pixel position .

7. A device for automatic optimization of SAR images according to any one of claims 1-6, characterized in that, The method comprises: The prior extraction module performs optimization iteration on the input SAR image, and inputs the optimization iteration result as prior knowledge into the feature encoding module; The feature encoding module performs feature extraction, detail enhancement and down-sampling on the SAR image after optimization iteration, to obtain a first feature map; The feature decoding module performs up-sampling, feature extraction and detail enhancement on the first feature map after convolution, to obtain a second feature map; The feature fusion module performs feature fusion on the second feature map, and performs residual connection with the input SAR image, to obtain a reconstructed image, takes the reconstructed image as the input of the prior extraction module, and guides the optimization iteration algorithm to converge based on a structural loss function.

Images