Optical image guided SAR image denoising method and system based on gradient consistency constraint

Abstract

The invention relates to a remote sensing image processing technology, and discloses an optical image guided SAR image denoising method and system based on gradient consistency constraint, and the system comprises a cross-modal reference image generation module, a blind spot network denoising module and an omnidirectional gradient loss calculation module. A cross-domain reference image is generated through non-local mean filtering and a conditional diffusion model, the cross-domain reference image and an SAR image are fused on the basis of a double-branch blind spot network, priori knowledge is provided for SAR image denoising by fully utilizing the cross-domain reference image, and under the guidance of omnidirectional gradient loss, the system ensures global smoothness, and meanwhile, the SAR image denoising accuracy is improved. According to the method, the detail texture expression capability and the visual quality of the denoised image are remarkably improved, an efficient and accurate solution is provided for SAR remote sensing image understanding, and the method is particularly suitable for a high-quality SAR image denoising task under strong SAR image noise.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and in particular relates to an optical image-guided SAR image denoising method and system based on gradient consistency constraints. Background Art

[0002] Synthetic aperture radar is an active remote sensing technology. During imaging, the radar itself emits an electromagnetic wave beam, and then receives the signal reflected by the target object, thereby detecting the characteristics of the target object. Compared with optical remote sensing, SAR has the characteristics of all-weather and all-day, and is therefore widely used in military, agriculture, rescue and other aspects. However, due to the coherent scattering of surface targets, SAR images inevitably contain coherent speckle noise formed by the coherent combination of scatterers existing within the resolution unit. The presence of this type of noise seriously affects the performance of downstream imaging tasks such as detection, segmentation and classification. In addition, since coherent speckle noise is multiplicative noise, it is more difficult to remove than additive noise. Therefore, suppressing coherent speckle noise while restoring boundary and contrast information with high fidelity is very critical for understanding SAR images.

[0003] SAR image denoising, as a fundamental preprocessing step for understanding SAR images, has garnered extensive attention over the past few decades. In particular, extensive research in the past two decades has demonstrated the importance of this area. Currently, the main approaches for suppressing speckle noise include: 1) spatial domain methods, 2) transform domain methods, 3) non-local methods, and 4) deep learning. Spatial domain methods rely on the spatial distribution of noise by calculating a weighted average of pixels within a local window to achieve noise suppression. However, because these filters employ a local window approach to suppress noise, edge and texture details in SAR images are often lost. These filters perform well in homogeneous regions but poorly in non-homogeneous regions due to the abrupt changes between adjacent pixels. Transform domain methods achieve noise separation by converting the image from the spatial domain to the transform domain, where noise and texture can be clearly distinguished. Compared to spatial domain denoising methods, transform domain methods better preserve texture details, but due to the need for transformations and inverse transformations, they can produce additional artifacts when processing edges. Non-local filtering weights pixel regions with similar characteristics across the entire image, enabling denoising while preserving image detail to the greatest extent possible. However, these methods will still identify similar features in areas where they don't exist, resulting in some annoying and strange phenomena such as ghosting and brushstrokes. The shortcomings of these methods pose a challenge to the noise suppression capabilities of real SAR images and hinder the further promotion of the technology in high-precision application scenarios. Optical remote sensing images, with their high spatial resolution, strong temporal continuity, and rich surface information coverage, play an irreplaceable role in surface monitoring and earth observation. They are less susceptible to noise and can provide better prior knowledge for noise suppression in SAR images. Therefore, by combining optical prior knowledge, it is expected to further improve the quality of SAR images and provide more reliable technical support for seamless spatiotemporal monitoring of the surface. Summary of the Invention

[0004] Aiming at the problem of noise in real SAR image caused by the superposition of coherent speckles in backscatter, the present invention provides an optical image-guided SAR image denoising method based on gradient consistency constraint, combining the advantages of high resolution and clear ground object types of optical images to provide valuable gain information for SAR image denoising.

[0005] The technical solutions for implementing the present invention are as follows: In a first aspect, the present invention provides a method for SAR image denoising guided by optical imagery based on gradient consistency constraints, comprising the following steps: Step 1: Construct a cross-modal reference image generation module: Combine the input SAR image and optical image to generate a cross-domain reference image through non-local mean filtering and conditional diffusion model; Step 2: Construct a blind spot network denoising module: Use a dual-branch structure to fuse the SAR image and the cross-domain reference image. After disrupting the noise correlation through pixel shuffling and downsampling, use the blind spot convolution and dilated convolution modules to restore the image, achieve global noise suppression, and output the denoising result. Step 3: Construct an omnidirectional gradient loss calculation module: calculate the omnidirectional gradient map of the cross-domain reference image, and use the omnidirectional gradient map to calculate the loss of the denoising result to complete the training; In step 4, the prediction stage, the trained cross-modal reference image generation module and blind spot network denoising module are used to obtain the final denoising result.

[0006] Furthermore, the specific implementation of non-local mean filtering includes: The input optical remote sensing image is and SAR images are , first of all Divide the window and block, then calculate The similarity weight of the current block center with each block within the defined window , its formula is expressed as:

[0007] Where, represents the normalization coefficient, is represented as the smoothing parameter, and Represent the current block center and other block centers respectively, represents the Euclidean distance between the center point of the block and other blocks in the window, and Represent the current block center and other block centers respectively; The obtained similarity block weights are compared with the SAR image The corresponding pixel value in Multiply and calculate to generate the cross-domain reference map of the current SAR image , its formula is expressed as: .

[0008] Furthermore, the specific processing process of the conditional diffusion model is as follows: First, the generated cross-domain reference graph Input diffusion model for training, in the forward process, the conditional diffusion model is fed to the SAR image In progress Add noise in the stage; Then the generated noise superimposed image is subjected to the inverse diffusion process and the cross-domain reference image is used to As a conditional image for supervision, Start iteratively reconstructing the clean image, where express Noise image at the moment; the final generation and input Corresponding output reference image .

[0009] Furthermore, the specific implementation of step 2 is as follows: 2.1) Downsampling the image by pixel shuffling Processing is performed to obtain several output images, which are then stitched together to form a single image with the same size as the input image. ; 2.2) Construct a dual-branch blind-spot network. The processing process is as follows: first, the feature dimension is expanded through 1×1 convolution. Then, a dual-branch structure is used to extract features with different convolution kernel sizes. Each branch uses a blind-spot convolution with the center pixel blinded to process the features. Then, multiple dilated convolution blocks are used to extract and restore the features. Finally, the results of the two branches are spliced and a 1×1 convolution is used to output the final denoised image. 2.3) Use the same sampling coefficient as step 2.1) to perform pixel shuffling upsampling on the denoised image and reconstruct the low-resolution result into the original resolution , where H, W, and C represent the height, width, and number of channels of the image respectively.

[0010] Furthermore, the specific implementation of step 2.2) is as follows: 2.2.1) Image after pixel shuffling and downsampling Perform 1×1 convolution operation to generate channel context code; 2.2.2) The features after channel expansion are fed into a dual-branch structure, where one branch uses a 3×3 blind spot convolution and the other uses a 5×5 blind spot convolution. After the convolution is completed, two 1×1 convolutions are used to adjust the features. 2.2.3) Texture information is restored by stacking multiple dilated convolution modules, each of which is a combination of a 3×3 dilated convolution and a 1×1 convolution; 2.2.4) Integrate the features from the two branches and stack multiple 1×1 convolutions to recover the features, thus obtaining the final denoised image.

[0011] Furthermore, the specific implementation of step 3 is as follows: 3.1) Cross-domain reference image Fill the edges; 3.2) Shift the cross-domain reference image by one pixel in each of the eight directions: upper left, upper, upper right, right, lower right, lower, lower left, and left, and calculate the pixel difference between the image and the original image. 3.3) Stack the obtained gradient maps in the eight directions and calculate their maximum and minimum values pixel by pixel. If the maximum and minimum values are equal, mark the pixel as having no gradient. 3.4) Superimpose the obtained gradient-free pixels and the gradient maps in the eight directions, calculate the maximum index in the channel dimension, generate a one-dimensional omnidirectional gradient map, and obtain the direction and magnitude of the gradient map; 3.5) Based on the original SAR image and denoising results Loss calculation is performed to ensure the stability of the overall modality and features. Secondly, the homogeneous region mask is extracted through the omnidirectional gradient map generated in step 3.4), and the homogeneous region of the denoising result is compared with the cross-domain reference map. homogeneous regions Loss calculation is used to ensure smoothness of homogeneous regions; finally, by adding The loss is used to ensure that the checkerboard effect caused by the dilated convolution is alleviated.

[0012] Further, The loss is calculated as follows:

[0013] in represents the pixel position, Respectively represent the height, width and number of channels of the image, and Respectively represents the calculation of the difference between adjacent pixels in the horizontal and vertical directions of the image.

[0014] Furthermore, the overall loss is shown in the following formula:

[0015] in, Indicates the output result, represents the input image, represents a cross-domain reference graph, Indicates the output result and the L1 loss between the input image, Indicates the output result Cross-domain reference graph In homogeneous areas L1 loss, Indicates the output result own loss; , and The weight of each loss.

[0016] In the second aspect, the present application also provides an optical image-guided SAR image denoising system based on gradient consistency constraints, which mainly includes a cross-modal reference image generation module, a blind spot network denoising module and an omnidirectional gradient loss calculation module; The cross-modal reference image generation module integrates the non-local means method with the conditional diffusion model to achieve semantic reconstruction of SAR images through modal guidance from the optical image, providing reliable knowledge priors for phase noise suppression in the blind spot network. Specifically, the similarity between each block in the optical image and the blocks in the search window is calculated pixel by pixel, and a distance weight is generated. The generated distance weight is applied to the corresponding SAR image pixels to generate a cross-domain reference image. Using the SAR image and the cross-domain reference image as image pair inputs, the conditional diffusion model generates a mapping relationship between the two, reducing the reliance on the optical image in the subsequent SAR image denoising process.

[0017] The blind spot network denoising module has a dual-branch structure. Each branch includes a pixel-shuffled downsampling layer, a 3×3 blind spot convolution layer with a stride of 1, and multiple residual dilated convolution blocks. The residual dilated convolution block consists of a 3×3 dilated convolution and a 1×1 convolution layer. Without increasing complexity, it expands the receptive field of the convolution to restore the features of the central blind spot. Specifically, the SAR image and the cross-domain reference image are pixel-shuffled downsampled to remove the correlation of SAR image noise, which conforms to the blind spot network's assumption of noise independence. The blind spot network removes noise through a process of blinding blind spot convolution and restoring dilated convolution. Finally, pixel-shuffled upsampling is used to restore the original resolution.

[0018] The omnidirectional gradient loss calculation module is designed as an efficient gradient calculation structure based on the integral map to efficiently calculate the difference between adjacent pixels. It is used to quickly distinguish homogeneous areas and heterogeneous areas in SAR images, and supervise their restoration results respectively to ensure the smoothness of homogeneous areas and the texture structure of heterogeneous areas, and combine Loss is used to ensure the continuity of the global image.

[0019] The result output module is used in the prediction stage to obtain the final denoising result by using the trained cross-modal reference image generation module and blind spot network denoising module.

[0020] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the optical image-guided SAR image denoising method based on gradient consistency constraints as described in the above technical solution is implemented.

[0021] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the computer program that can be run on the processor as described above. When the processor executes the program, it implements the optical image-guided SAR image denoising method based on gradient consistency constraints as described in the above technical solution.

[0022] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the computer program as described above and can be run on the processor, and when the processor executes the program, implements the optical image-guided SAR image denoising method based on gradient consistency constraints as described in the above technical solution.

[0023] Compared with the prior art, the present invention has the following advantages: 1) This invention combines non-local mean filtering techniques with matching optical pixel weights to generate cross-domain reference images, aligning SAR image modalities with optical texture features. These cross-domain reference images serve as auxiliary information to enhance the denoising process. Furthermore, a conditional diffusion model is used to fit the mapping between SAR images and cross-domain reference images, avoiding the need for paired data during the training and testing phases of the denoising network. This method effectively overcomes modality discrepancies by guiding SAR image reconstruction through optical imagery, providing reliable prior knowledge for SAR image denoising.

[0024] 2) The multi-scale blind spot despeckle network designed in this paper achieves semi-supervised speckle reduction by integrating cross-domain reference image features. This network initially reduces speckle correlation by performing pixel shuffling and downsampling on the SAR image and the cross-domain reference image. Subsequently, a multi-scale blind spot sub-network is employed to mitigate potential information loss caused by identical blind spot pixels. While maintaining the advantages of end-to-end training, this network utilizes feature priors from the cross-domain reference image to guide the SAR image denoising process, achieving high-fidelity noise reduction.

[0025] 3) Existing SAR denoising methods typically use the L1 loss for information reconstruction, neglecting structural information. By constructing an omnidirectional gradient loss, we distinguish between homogeneous and heterogeneous regions based on gradient map evaluation, ensuring appropriate supervision of different regions to achieve homogeneous smoothing and heterogeneous detail refinement. Furthermore, the use of the TV loss mitigates checkerboard artifacts common in blind spot networks. By integrating these losses, we achieve a balance between smoothness and texture detail in the denoised image, significantly improving the overall image reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 This is an overall framework diagram of an optical image-guided SAR image denoising method based on gradient consistency constraints provided by an embodiment of the present invention; Figure 2 Schematic diagram of non-local mean filtering of SAR images guided by optical imagery provided by an embodiment of the present invention; Figure 3 This is a schematic diagram of generating a conditional diffusion model provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of a dual-branch blind spot network provided by an embodiment of the present invention; Figure 5 : These are sample images of SAR image denoising provided by an embodiment of the present invention; (a) represents an optical image, (b) represents a SAR image, (c) represents the result of comparison method 1, (d) represents the result of comparison method 2, (e) represents the result of comparison method 3, (f) represents the result of comparison method 4, (g) represents the result of comparison method 5, (h) represents the result of comparison method 6, (i) represents the result of comparison method 7, and (j) represents the result of the present invention.

[0028] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention; Reference numerals: Among them: 810 - processor, 820 - communication interface, 830 - memory, 840 - communication bus. DETAILED DESCRIPTION

[0029] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0030] This embodiment provides an optical image-guided SAR image denoising method based on gradient consistency constraints. The method includes a cross-modal reference image generation module, a blind spot network denoising module and an omnidirectional gradient loss calculation module. The cross-domain reference image is generated by non-local mean filtering and a conditional diffusion model, and the cross-domain reference image and the SAR image are fused based on a dual-branch blind spot network. The cross-domain reference image is fully utilized to provide prior knowledge for SAR image denoising. Under the guidance of the omnidirectional gradient loss, the method significantly improves the detail texture expression ability and visual quality of the denoised image while ensuring global smoothness, providing an efficient and accurate solution for SAR remote sensing image understanding. Figure 1 As shown, the specific implementation includes: 1) Construct a cross-modal reference image generation module, combining the input SAR image and optical image through non-local mean filtering and conditional diffusion model to generate a cross-domain reference image. The specific steps include the following: 1.1) If Figure 2 As shown, input optical remote sensing image and SAR images First calculate The similarity weight between the current block center and each block within the defined window is expressed as:

[0031] Where, represents the normalization coefficient, is represented as the smoothing parameter, and Represent the current block center and other block centers respectively, Indicates the Euclidean distance between the center point of the block and other blocks in the window. For the present invention, the block size is set to 3, the search window size is 15, Set it to 10 to control the attenuation of the Gaussian function. It is the inverse of the Euclidean distance between the center point of the current block and other blocks in the window. If they are more similar, the weight is greater; otherwise, the weight is smaller.

[0032] 1.2) If Figure 2 As shown, the similarity block weights obtained in step 1.1 are compared with the SAR image The corresponding pixel value in Multiply and calculate to generate the cross-domain reference map of the current SAR image , its formula is expressed as:

[0033] 1.3) If Figure 3 As shown in the figure, the matching SAR-cross-domain reference image pair generated in step 1.2 is the cross-domain reference image The function of this model is to further reduce the dependence on matching optical images by learning the mapping relationship between SAR images and corresponding cross-domain reference images. In progress The added noise in the stage is expressed as:

[0034] in, , is a predefined hyperparameter that represents the The noise variance is , and we set is monotonically increasing, which satisfies < In fact, any step size of Can be directly based on the original data To take a sample: : ),in , This way of expression makes It can be expressed as The linear combination of the noise and the additive noise ensures the naturalness and computational efficiency of the diffusion process.

[0035] 1.4) If Figure 3 As shown, the noise superposition image generated in step 1.3 is subjected to the inverse diffusion process and the reference image is used As a conditional image for supervision, Start iteratively reconstructing the clean image, where express The noise image at the moment. The final generation and input Corresponding output reference image The process can be expressed as:

[0036] in ,in, represents the time step, the current noise image, and the conditional image. , represents the mean and variance of the prior distribution of the model prediction. In our method, the input and conditions The channels are concatenated channel by channel to form the input of the two-layer channel. The diffusion model further enhances detail preservation and generation stability by gradually reconstructing the target distribution through a multi-step inverse denoising process based on a Markov chain.

[0037] 2) Construct a blind spot network denoising module, which consists of pixel shuffling downsampling / upsampling and a dual-branch blind spot network. Each branch of the dual-branch blind spot network includes a blind spot convolution for blinding and a multi-layer dilated convolution for feature recovery of a large receptive field. Finally, the final denoising result is output by fusing the features of the two branches and using multiple stacked 1×1 convolution layers. Figure 4 As shown, it specifically includes the following steps: 2.1) In SAR images, noise exhibits strong correlation, making it difficult to distinguish from real textures. Pixel shuffling downsampling combines a high-resolution image into multiple low-resolution images by sampling at intervals. This part controls the ability to suppress correlated noise through the downsampling coefficient. Given an input image , the downsampling factor is 5, then the 25 output images are obtained , and then stitch them together to form a consistent size with the input image .

[0038] 2.2) The dual-branch blind spot network achieves semi-supervised noise suppression by fusing cross-domain reference image features. The network uses a multi-scale blind spot sub-network to prevent potential information loss caused by the same blind spot pixels. The network first expands the dimension of the feature through 1×1 convolution, and then uses a dual-branch structure to extract features with different convolution kernel sizes. Each branch uses a blind spot convolution with the center pixel blinded to process the feature. The features are then extracted and restored through multiple dilated convolution blocks. Finally, the results of the two branches are spliced and the final denoised image is output using 1×1 convolution. Figure 4 As shown, the specific steps include: 2.2.1) Input image after pixel shuffling and downsampling Perform 1×1 convolution operations to generate channel context encoding to extract the potential feature information of the image.

[0039] 2.2.2) The expanded features are fed into a two-branch structure, with one branch using a 3×3 blind convolution and the other using a 5×5 blind convolution to avoid missing information about specific pixels during the blind convolution. After the convolution is complete, two 1×1 convolutions are used to adjust the features.

[0040] 2.2.3) Texture information is restored through the stacking of nine dilated convolutional modules. Each dilated convolutional module is a combination of a 3×3 dilated convolution and a 1×1 convolution. Furthermore, the use of residual connections in the dilated convolutional modules reduces feature weakening during the convolution process.

[0041] 2.2.4) Integrate the features from the two sub-networks and stack multiple 1×1 convolutions to recover the features, thus obtaining the final denoised image.

[0042] 2.3) The denoising result output by the network is still at low resolution. Use the same sampling coefficient as step 2.1 to perform pixel shuffling upsampling on the denoised image. The low-resolution result is reconstructed into the original resolution .

[0043] 3) The omnidirectional gradient loss calculation module is designed to calculate gradient differences in eight directions and stack the eight-directional features. Pixels in the features below a set threshold are set to 0. The maximum and minimum values along the channel dimension of the features are calculated. If the maximum value equals the minimum value, the pixel is marked as having no gradient. Finally, a gradient-free dimension is generated. This gradient-free dimension is stacked with the eight-directional features along the channel dimension. The maximum index of the final nine-dimensional feature along the channel dimension is calculated to generate a one-dimensional omnidirectional gradient map. This omnidirectional gradient map is then used to calculate the loss of the denoising results.

[0044] 3.1) The cross-domain reference image obtained in step 1.4 Fill the edges with a coefficient of 1 to avoid value out-of-bounds problems caused by translation.

[0045] 3.2) Shift the cross-domain reference image by one pixel in each of the eight directions: upper left, upper, upper right, right, lower right, lower, lower left, and left, and calculate the pixel difference between it and the original image.

[0046] 3.3) Stack the obtained gradient maps in the eight directions and calculate their maximum and minimum values pixel by pixel. If the maximum and minimum values are equal, mark the pixel as having no gradient.

[0047] 3.4) Superimpose the obtained gradient-free pixels with the original gradient maps in the eight directions, calculate the maximum index in the channel dimension, and generate a one-dimensional omnidirectional gradient map. Obtain the direction and magnitude of the gradient.

[0048] 3.5) Based on the original SAR image and the output of the dual-branch blind spot network in step 2.3) conduct Loss calculation is performed to ensure the stability of the overall modality and features. Secondly, the homogeneous region mask is extracted through the omnidirectional gradient map generated in step 3.4), and the result of the double-branch blind spot network output in step 2.3) is obtained. Homogeneous regions and cross-domain reference maps homogeneous regions Loss calculation is used to ensure smoothness of homogeneous regions. Finally, by adding The loss is used to ensure that the checkerboard effect caused by the dilated convolution is alleviated. The calculation method is as follows, where represents the pixel position, Respectively represent the height, width and number of channels of the image, and Respectively represent the calculation of the difference between adjacent pixels in the horizontal and vertical directions of the image:

[0049] Finally, the overall loss is shown in the following formula. Indicates the output result (In the formula, denoted) and the input image (denoted by represents the L1 loss between Indicates the output result Cross-domain reference graph (In the formula, In homogeneous areas L1 loss, Indicates the output result own loss:

[0050] , and The weights of each loss are respectively set in this method. , and Take the sum of 0.8, 0.2 To obtain the best denoising results.

[0051] In step 4, the prediction stage, the trained cross-modal reference image generation module and blind spot network denoising module are used to obtain the final denoising result.

[0052] The experiments in this embodiment were carried out in the hardware environment of NVIDIA RTX 3090 and the software environment of Python.

[0053] For the conditional diffusion model, we use Weather-diffusion as the backbone to learn the mapping between SAR images and cross-domain reference images. The model utilizes a U-Net architecture based on WideResNet. The network integrates group normalization and 16×16 resolution self-attention blocks, and injects temporal conditional embeddings into each residual block. We configure the diffusion process to T=1000 and use a linearly increasing noise table, where from Increment to , corresponding to and At the same time, the exponential moving average (EMA) method is used for the model parameters, starting from the first iteration of training, and updated at a decay rate of 0.999 after each training iteration. Adam is used to optimize the model training, and the initial learning rate is , the weight decays to 0.

[0054] For the multi-scale blind spot network, the Adam optimizer is used to optimize the SAR image denoising system based on the gradient consistency constraint and the initial learning rate is set to The first-order moment estimate and the second-order moment estimate decay rates were 0.9 and 0.999, respectively, for a total of 20 training iterations. A learning rate adjustment strategy was also implemented, reducing the learning rate to 0.1 of its original value every 8 batches to explore local minima of the loss function. For pixel shuffling downsampling / upsampling, the number of downsampling steps was 5 during training and 1 during testing.

[0055] This example uses the QXSLAB_SAROPT dataset, which matches Gaofen-3 SAR images with Google Earth optical images. This dataset contains 20,000 image pairs, each with a size of 256×256, and a resolution of 1 meter for both optical and SAR images. The training, validation, and test sets are divided in a 9:1:1 ratio.

[0056] The SAR denoising effects of the present invention are compared with those of four existing methods published in authoritative journals, namely: [C. -A. Deledalle, L. Denis, F. Tupin, 2009. Iterative Weighted MaximumLikelihood Denoising With Probabilistic Patch-Based Weights. IEEE Trans.Image Process. 18, 2661–2672.] (Comparison Method 1), [S. Parrilli, M. Poderico, CVAngelino, L. Verdoliva, 2012. A Nonlocal SAR Image Denoising Algorithm Basedon LLMMSE Wavelet Shrinkage. IEEE Trans. Geosci. Remote Sens. 50, 606–616.] (Comparison Method 2), [Vitale, S., Cozzolino, D., Scarpa, G., Verdoliva, L., Poggi,G., 2019. Guided patchwise nonlocal SAR despeckling. IEEE Trans. Geosci. Remote Sens. 57, 6484–6498.] (Comparison method 3), [Perera, MV, Bandara, WGC, Valanarasu, JMJ, Patel, VM, 2022. Transformer-based SAR image despeckling, in: IGARSS 2022-2022 IEEE International Geoscience and RemoteSensing Symposium. IEEE, pp. 751–754.] (Comparison method 4), [Ko, J., Lee, S., 2021. SAR image despeckling using continuous attention module. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 15, 3–19.] (Comparison method 5), [Lee, W., Son, S., Lee, KM,2022.Ap-bsn: Self-supervised denoising for real-world images via asymmetricpd and blind-spot network, in: Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. pp. 17725–17734.] (Comparison method 6), [Molini, AB, Valsesia, D., Fracastoro, G., Magli, E., 2021. Speckle2Void: Deep self-supervised SAR despeckling with blind-spot convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 60, 1–17.] (Compare method 7). The corresponding repair results are as follows Figure 5 shown. Figure 5 (a) represents the optical image, (b) represents the SAR image, (c) represents the results of Comparison Method 1, (d) represents the results of Comparison Method 2, (e) represents the results of Comparison Method 3, (f) represents the results of Comparison Method 4, (g) represents the results of Comparison Method 5, (h) represents the results of Comparison Method 6, (i) represents the results of Comparison Method 7, and (j) represents the results of the present invention. The comparison results show that compared with other methods, the optical image-guided SAR image denoising system designed in this invention, based on gradient consistency constraints, performs best in terms of detail reconstruction and homogeneous region smoothness in the denoised results. While other methods suffer from varying degrees of detail loss and texture blurring during reconstruction, the present invention significantly improves these deficiencies, effectively enhancing the visual quality and structural integrity of the image.

[0057] The embodiment of the present invention also provides an optical image-guided SAR image denoising system based on gradient consistency constraints, including the following modules: The cross-modal reference image generation module is used to combine the input SAR image and optical image to generate a cross-domain reference image through non-local mean filtering and conditional diffusion model; The blind spot network denoising module is used to fuse SAR images and cross-domain reference images using a dual-branch structure. After disrupting noise correlation through pixel shuffling and downsampling, it uses blind spot convolution and dilated convolution modules to restore the image, suppressing global noise and outputting the denoised results. An omnidirectional gradient loss calculation module is used to calculate the omnidirectional gradient map of the cross-domain reference image and use the omnidirectional gradient map to calculate the loss of the denoising result to complete the training; The result output module is used in the prediction stage to obtain the final denoising result by using the trained cross-modal reference image generation module and blind spot network denoising module.

[0058] The specific implementation method of each module is the same as that of each step and will not be described in detail in this invention.

[0059] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call logic instructions in the memory 830 to execute the optical image-guided SAR image denoising method based on gradient consistency constraints.

[0060] Furthermore, the logic instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0061] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the methods of the optical image-guided SAR image denoising system based on gradient consistency constraints provided by the above methods.

[0062] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method of the optical image-guided SAR image denoising system based on gradient consistency constraints provided by the above methods.

[0063] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0064] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An optical image-guided SAR image denoising method based on gradient consistency constraint, characterized by: The steps include: Step 1: Construct a cross-modal reference image generation module: Combine the input SAR image and optical image to generate a cross-domain reference image through non-local mean filtering and conditional diffusion model; Step 2: Construct a blind spot network denoising module: Use a dual-branch structure to fuse the SAR image and the cross-domain reference image. After disrupting the noise correlation through pixel shuffling and downsampling, use the blind spot convolution and dilated convolution modules to restore the image, achieve global noise suppression, and output the denoising result. Step 3: Construct an omnidirectional gradient loss calculation module: calculate the omnidirectional gradient map of the cross-domain reference image, and use the omnidirectional gradient map to calculate the loss of the denoising result to complete the training; In step 4, the prediction stage, the trained cross-modal reference image generation module and blind spot network denoising module are used to obtain the final denoising result.

2. The optical image-guided SAR image denoising method based on gradient consistency constraint according to claim 1, wherein: The specific implementation of non-local mean filtering includes: The input optical remote sensing image is and SAR images are , first of all Divide the window and block, then calculate The similarity weight of the current block center with each block within the defined window , its formula is expressed as: Where, represents the normalization coefficient, is represented as the smoothing parameter, and Represent the current block center and other block centers respectively, represents the Euclidean distance between the center point of the block and other blocks in the window, and Represent the current block center and other block centers respectively; The obtained similarity block weights are compared with the SAR image The corresponding pixel value in Multiply and calculate to generate the cross-domain reference map of the current SAR image , its formula is expressed as: 。 3. The optical image-guided SAR image denoising method based on gradient consistency constraint according to claim 2, characterized in that: The specific processing process of the conditional diffusion model is as follows: First, the generated cross-domain reference graph Input diffusion model for training, in the forward process, the conditional diffusion model is fed to the SAR image In progress Add noise in the stage; Then the generated noise superimposed image is subjected to the inverse diffusion process and the cross-domain reference image is used to As a conditional image for supervision, Start iteratively reconstructing the clean image, where express Noise image at the moment; final generation and input The corresponding output cross-domain reference image .

4. The optical image-guided SAR image denoising method based on gradient consistency constraint according to claim 1, wherein: The specific implementation of step 2 is as follows: 2.1) Downsampling the image by pixel shuffling Processing is performed to obtain several output images, which are then stitched together to form a single image with the same size as the input image. ; 2.2) Construct a dual-branch blind spot network. The processing process is as follows: first, the feature dimension is expanded through 1×1 convolution. Then, a dual-branch structure is used to extract features with different convolution kernel sizes. Each branch uses a blind spot convolution with the center pixel blinded to process the features. Then, multiple dilated convolution blocks are used to extract and restore the features. Finally, the results of the two branches are spliced and 1×1 convolution is used to output the denoised image. 2.3) Use the same sampling coefficient as step 2.1) to perform pixel shuffling upsampling on the denoised image and reconstruct the low-resolution result into the original resolution result , where H, W, and C represent the height, width, and number of channels of the image respectively.

5. The optical image-guided SAR image denoising method based on gradient consistency constraint according to claim 4, characterized in that: The specific implementation of step 2.2) is as follows: 2.2.1) Image after pixel shuffling and downsampling Perform 1×1 convolution operation to generate channel context code; 2.2.2) The features after channel expansion are fed into a dual-branch structure, where one branch uses a 3×3 blind spot convolution and the other uses a 5×5 blind spot convolution. After the convolution is completed, two 1×1 convolutions are used to adjust the features. 2.2.3) Texture information is restored by stacking multiple dilated convolution modules, each of which is a combination of a 3×3 dilated convolution and a 1×1 convolution; 2.2.4) Integrate the features from the two branches and stack multiple 1×1 convolutions to recover the features, thus obtaining the final denoised image.

6. The optical image-guided SAR image denoising method based on gradient consistency constraint according to claim 1, wherein: The specific implementation of step 3 is as follows: 3.1) Cross-domain reference image Fill the edges; 3.2) Shift the cross-domain reference image by one pixel in each of the eight directions: upper left, upper, upper right, right, lower right, lower, lower left, and left, and calculate the pixel difference between the image and the original image. 3.3) Stack the obtained gradient maps in the eight directions and calculate their maximum and minimum values pixel by pixel. If the maximum and minimum values are equal, mark the pixel as having no gradient. 3.4) Superimpose the obtained gradient-free pixels and the gradient maps in the eight directions, calculate the maximum index in the channel dimension, generate a one-dimensional omnidirectional gradient map, and obtain the direction and magnitude of the gradient map; 3.5) Based on the original SAR image and denoising results Loss calculation is performed to ensure the stability of the overall modality and features. Secondly, the homogeneous region mask is extracted through the omnidirectional gradient map generated in step 3.4), and the homogeneous region of the denoising result is compared with the cross-domain reference image. homogeneous regions Loss calculation is used to ensure smoothness of homogeneous regions; finally, by adding The loss is used to ensure that the checkerboard effect caused by the dilated convolution is alleviated.

7. The optical image-guided SAR image denoising method based on gradient consistency constraint according to claim 6, characterized in that: The loss is calculated as follows: in represents the pixel position, Respectively represent the height, width and number of channels of the image, and Respectively represents the calculation of the difference between adjacent pixels in the horizontal and vertical directions of the image.

8. The optical image-guided SAR image denoising method based on gradient consistency constraint according to claim 7, wherein: The overall loss is shown in the following formula: in, Indicates the output result, represents the input image, represents a cross-domain reference image, Indicates the output result and the L1 loss between the input image, Indicates the output result Cross-domain reference graph In homogeneous areas L1 loss, Indicates the output result own loss; , and The weight of each loss.

9. An optical image-guided SAR image denoising system based on gradient consistency constraints, characterized by: Includes the following modules: The cross-modal reference image generation module is used to combine the input SAR image and optical image to generate a cross-domain reference image through non-local mean filtering and conditional diffusion model; The blind spot network denoising module is used to fuse SAR images and cross-domain reference images using a dual-branch structure. After disrupting noise correlation through pixel shuffling and downsampling, it uses blind spot convolution and dilated convolution modules to restore the image, suppressing global noise and outputting the denoised results. An omnidirectional gradient loss calculation module is used to calculate the omnidirectional gradient map of the cross-domain reference image and use the omnidirectional gradient map to calculate the loss of the denoising result to complete the training; The result output module is used in the prediction stage to obtain the final denoising result by using the trained cross-modal reference image generation module and blind spot network denoising module.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the optical image-guided SAR image denoising method based on gradient consistency constraint according to any one of claims 1 to 8 is implemented.

Images