A SAR image debanding and optical image fusion method, system, device and medium

Abstract

The application relates to the technical field of remote sensing image processing, in particular to a SAR image despeckling and optical image fusion method, system, device and medium, the method realizes mutual reinforcement optimization of two tasks by constructing a closed-loop iterative mutual promotion mechanism for SAR despeckling and optical image fusion, and realizes end-to-end training through a weighted combined joint loss function. The method realizes mutual reinforcement optimization of SAR despeckling and optical fusion, experiments on two large-scale SAR-optical data sets show that the method achieves optimal performance in structure preservation and noise suppression, and can be widely applied to remote sensing image processing scenes such as land cover analysis, target detection and environmental monitoring.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing technology, and in particular to a method, system, device and medium for SAR image despegging and optical image fusion. Background Technology

[0002] Synthetic Aperture Radar (SAR) and optical images are core remote sensing data sources for Earth observation, and the two are highly complementary: optical images contain rich spectral, texture, and color information, which can intuitively reflect the surface features of ground objects; SAR images can capture the unique scattering structure of ground objects and are not affected by weather or lighting conditions, enabling all-day and all-weather observation. The integration of the two is of great significance for improving the comprehensive analysis capabilities of remote sensing images.

[0003] However, SAR images are inevitably contaminated by multiplicative speckle noise during imaging. This noise distorts the ground structure pattern and weakens cross-modal structural consistency, becoming a core technical challenge in SAR-optical image fusion. Existing fusion methods mainly fall into two paradigms, both of which have significant drawbacks due to their failure to effectively handle the entanglement between speckle noise and structural information, and their neglect of the inherent interaction between SAR despeccing and fusion: First, single-modal despeccing is performed before fusion. The lack of cross-modal structural prior guidance during despeccing makes it easy to over-smooth subtle scattering structures, losing core complementary SAR information and resulting in insufficient structural integrity of the fusion result; Second, direct two-stream fusion does not explicitly model the statistical characteristics of speckle noise, allowing it to easily propagate into the fusion features, causing structural artifacts in the fused image and reducing its visual quality and application value.

[0004] Furthermore, existing speckle removal methods that combine frequency domain processing mostly employ fixed threshold wavelet shrinkage strategies, which cannot adaptively suppress speckle noise. At the same time, they lack a pixel-level evaluation mechanism for the reliability of SAR features, making it difficult to avoid residual noise propagation in high uncertainty regions. Moreover, there is no method to integrate SAR speckle removal and optics into a closed-loop iterative and mutually reinforcing system, nor is there a targeted joint loss function to achieve end-to-end collaborative optimization of the two tasks, resulting in both failing to achieve optimal performance.

[0005] To address the aforementioned technical shortcomings, there is an urgent need to propose an algorithm architecture that can jointly optimize and mutually reinforce SAR speckle removal and optical fusion. By constructing a closed-loop iterative mutual promotion mechanism, combined with frequency-decoupled adaptive speckle removal, uncertainty-aware cross-modal fusion, and joint loss constraints, this approach can solve problems such as oversmoothing, noise propagation, and structural distortion in existing methods, thereby improving the SAR speckle removal effect and the quality of SAR-optical image fusion. Summary of the Invention

[0006] This application provides a method, system, device, and medium for SAR image despewing and optical image fusion. This solution can achieve efficient despewing of noisy SAR images and high-quality fusion of SAR-optical images, thereby improving the information utilization rate and subsequent processing accuracy of remote sensing images.

[0007] To address the aforementioned technical problems, in a first aspect, embodiments of this application provide a method for SAR image despeccing and optical image fusion, comprising the following steps: Step S1: Input the original noisy SAR image and the registered optical image, convert the original noisy SAR image to the logarithmic domain to process multiplicative speckle noise, and simultaneously initialize the fusion latent representation, uncertainty map, iteration number t=1, and total iteration number T; wherein, the joint loss function combines uncertainty-aware pixel-level loss with structural similarity loss to construct a despeccing loss, constructs a fusion loss from three dimensions: intensity consistency, gradient structure, and chromaticity fidelity, and constructs an iterative mutual promotion loss with the final iteration's fusion latent representation as a reference; Step S2: Perform wavelet domain near-end update processing for the t-th iteration to complete SAR image despeccing and calculate pixel-level uncertainty, obtaining the despecced SAR image and uncertainty map for this iteration; Step S3: Based on the despecced SAR image obtained in Step S2... The image and uncertainty map are processed sequentially, and uncertainty-guided fusion update and data consistency update are performed to obtain the fused latent representation and corrected SAR image after this iteration. Among them, the data consistency update is based on the gradient-based correction strategy, and uses a learnable step size to correct the speckled SAR image of this iteration, using the original noisy SAR image as a reference to maintain the fidelity between the speckled SAR image and the original observation data. Step S4: Determine whether the number of iterations satisfies t < T. If so, t = t + 1, and use the fused latent representation, uncertainty map and corrected SAR image of this iteration as the prior information for the next iteration, and return to step S2 to continue the iteration; otherwise, the iteration is completed. Step S5: The corrected SAR image obtained in the final iteration is restored to the original domain through exponential mapping to obtain a clean SAR image. The fused latent representation of the final iteration is reconstructed with structure preservation residuals to obtain the SAR-optical fusion image in the RGB domain.

[0008] In some exemplary embodiments, step S1 specifically includes: inputting the original noisy SAR image. and the registered optical image ; the original noisy SAR image Transform to the logarithmic domain to handle multiplicative speckle while initializing the fusion latent representation. Uncertainty diagram Current iteration number Total number of iterations Among them, the joint loss function Despeculiarization loss is constructed by combining pixel-level loss with uncertainty perception and structural similarity loss. The fusion loss is constructed from three dimensions: intensity consistency, gradient structure, and chromaticity fidelity. The iterative mutual promotion loss is constructed with reference to the fusion latent representation of the final iteration. The expression for the joint loss function is:

[0009] In the formula, , , These are preset weighted hyperparameters used to adjust the contribution weights of the three loss terms in joint training.

[0010] In some exemplary embodiments, the specific processing steps of the wavelet domain near-end update in step S2 are as follows: Step S201: Perform discrete wavelet transform on the logarithmic domain corrected SAR image to decompose it into a low-frequency subband LL and a high-frequency subband HF; Step S202: Combine the fusion latent representation and uncertainty map from the previous iteration, sequentially perform uncertainty-aware modulation, residual injection, and long-range dependency modeling of Mamba basis modules on the low-frequency subband LL to complete the low-frequency macrostructure reconstruction; Step S203: Calculate the local statistical features of the high-frequency subband HF and combine them with the fusion latent representation from the previous iteration. The gradient features shown are concatenated and input into the Kolmogorov-Arnold network to learn the adaptive shrinkage threshold of the space and perform soft shrinkage processing on the high-frequency subband HF to complete high-frequency speckle noise suppression; Step S204: Perform inverse wavelet transform on the reconstructed low-frequency subband LL and the denoised high-frequency subband HF to obtain the speckled SAR image of this iteration; Step S205: Calculate the pixel-level uncertainty map of this iteration based on the residual between the original noisy SAR image and the current speckled SAR image, as well as the structural deviation between the speckled SAR image and the previous round of fusion latent representation.

[0011] In some exemplary embodiments, after the low-frequency macrostructure reconstruction is completed in step S202, the modulated and reconstructed low-frequency subband is obtained. The expression for uncertainty-aware modulation is:

[0012] In the formula, , For the fusion potential representation Affine parameters extracted after projection onto LL space. , Uncertainty diagram The generated uncertainty-aware correction parameters It is the Sigmoid activation function. The modulation intensity scale parameters are to be learned. These are residual injection coefficients used to preserve the energy characteristics of the original SAR image. This indicates element-wise multiplication.

[0013] In some exemplary embodiments, in step S203, the high-frequency sub-band is calculated. Local mean and local standard deviation Local statistical characteristics, and their fusion potential representation with the previous round. gradient features After concatenation, the data is input into the Kolmogorov-Arnold network (KAN), and the learning space adaptively shrinks to a threshold. The high-frequency subband (HF) is then subjected to soft contraction to suppress high-frequency speckle noise; among which, the spatial adaptive contraction threshold is... The calculation expression is:

[0014] The expression for soft shrinkage processing is:

[0015] In the formula, For the feature mapping function of the Kolmogorov-Arnold network, For symbolic functions, For absolute value operations, To perform the maximum value operation, This is the high-frequency subband after noise reduction.

[0016] In some exemplary embodiments, the specific processing steps of the uncertainty-guided fusion update in step S3 are as follows: Step S301: Using a 3-level U-Net as a shared encoder, multi-scale hierarchical features are extracted from the speckled SAR image and optical image of this iteration, respectively. Step S302: Using the pixel-level uncertainty map of this iteration, confidence modulation is performed on the SAR layered features at each scale, and adaptive attenuation is performed on the SAR feature regions with high uncertainty. Step S303: Using the multi-scale hierarchical features of the optical image as the context prior, the modulated SAR features and optical features are fused in a multi-scale two-stage process to obtain the fusion latent representation at each scale. Step S304: Using a bottom-up fusion approach combined with skip connections, the fusion latent representations at each scale are aggregated to obtain the global fusion latent representation for this iteration, while preserving high-resolution spatial structure information.

[0017] In some exemplary embodiments, the specific steps of the multi-scale dual-stage fusion in step S303 are as follows: Step S3031: Perform spatial gating fusion, selectively filter the modulated SAR features through a spatial gating mechanism, and only integrate the SAR features that match the optical feature structure into the optical features to obtain the gating fused features; Step S3032: Perform channel adaptive injection, generate channel-level adaptive weighting coefficients from the optical features through a channel excitation mechanism, use these coefficients to weight the gating fused features and combine them with the optical features, and perform feature optimization through residual blocks to obtain the fusion latent representation at this scale.

[0018] Secondly, this application also provides a SAR image despeccing and optical image fusion system. This system implements the SAR image despeccing and optical image fusion method described in the above embodiments. The system includes: an iterative mutual-promotion loss construction module, an iteration module, an update module, a judgment module, and an output module connected sequentially. The iterative mutual-promotion loss construction module takes the original noisy SAR image and the registered optical image as input, converts the original noisy SAR image to the logarithmic domain to process multiplicative speckle, and initializes the fusion latent representation, uncertainty map, iteration number t=1, and total iteration number T. The joint loss function combines uncertainty-aware pixel-level loss with structural similarity loss to construct a despeccing loss, constructs a fusion loss from three dimensions: intensity consistency, gradient structure, and chromaticity fidelity, and constructs the iterative mutual-promotion loss with the final iterative fusion latent representation as a reference. The iteration module performs wavelet domain near-end update processing in the t-th iteration, completes SAR image despeccing, calculates pixel-level uncertainty, and obtains the despeccing S of this iteration. The update module is used to sequentially perform uncertainty-guided fusion update and data consistency update based on the despeculiar SAR image and uncertainty map output by the iteration module, to obtain the fused latent representation and corrected SAR image after this iteration. Among them, the data consistency update is based on the gradient-type correction strategy, which uses a learnable step size to correct the despeculiar SAR image of this iteration, and uses the original noisy SAR image as a reference to maintain the fidelity between the despeculiar SAR image and the original observation data. The judgment module is used to determine whether the number of iterations satisfies t < T. If so, t = t + 1, and the fused latent representation, uncertainty map and corrected SAR image of this iteration are used as the prior information for the next iteration, and the iteration is returned to the iteration module to continue the iteration. Otherwise, the iteration is completed. The output module is used to restore the corrected SAR image obtained in the final iteration to the original domain through exponential mapping to obtain a clean SAR image. The structure-preserving residual reconstruction is performed on the fused latent representation of the final iteration to obtain the SAR-optical fusion image in the RGB domain.

[0019] In addition, this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the SAR image despeccing and optical image fusion method described in the above embodiments.

[0020] In addition, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the SAR image despeccing and optical image fusion method described in the above embodiments.

[0021] The technical solution provided in this application has at least the following advantages: This application provides a method, system, device, and medium for SAR image despeccing and optical image fusion. The method includes the following steps: Step S1: Input the original noisy SAR image and the registered optical image, convert the original noisy SAR image to the logarithmic domain to process multiplicative speckle noise, and initialize the fusion latent representation, uncertainty map, iteration number t=1, and total iteration number T; wherein, the joint loss function combines uncertainty-aware pixel-level loss with structural similarity loss to construct a despeccing loss, constructs a fusion loss from three dimensions: intensity consistency, gradient structure, and color fidelity, and constructs an iterative mutual promotion loss with the final iteration's fusion latent representation as a reference; Step S2: Perform wavelet domain near-end update processing for the t-th iteration to complete SAR image despeccing and calculate pixel-level uncertainty, obtaining the despecced SAR image and uncertainty map for this iteration; Step S3: Based on the despecced SAR image obtained in Step S2 and The uncertainty map is used to sequentially perform uncertainty-guided fusion update and data consistency update to obtain the fused latent representation and corrected SAR image after the current iteration. Among them, the data consistency update is based on the gradient-based correction strategy, which uses a learnable step size to correct the speckled SAR image of the current iteration, and uses the original noisy SAR image as a reference to maintain the fidelity between the speckled SAR image and the original observation data. Step S4: Determine whether the number of iterations satisfies t < T. If so, t = t + 1, and use the fused latent representation, uncertainty map and corrected SAR image of the current iteration as the prior information for the next iteration, and return to step S2 to continue the iteration; otherwise, the iteration is completed. Step S5: The corrected SAR image obtained in the final iteration is restored to the original domain through exponential mapping to obtain a clean SAR image. The fused latent representation of the final iteration is reconstructed with structure preservation residuals to obtain the SAR-optical fusion image in the RGB domain.

[0022] The SAR image despeccing and optical image fusion method provided in this application constructs a closed-loop iterative mutual promotion mechanism for SAR despeccing and optical image fusion, breaking the technical drawback of decoupling the two tasks in existing technologies. It realizes mutual reinforcement and optimization of despeccing and fusion, effectively solving the problems of over-smoothing and noise propagation in the fused image. At the same time, it adopts a frequency-decoupled wavelet domain near-end update strategy, combining Mamba base modules and KAN networks to complete long-range dependency modeling of low-frequency macrostructure and spatial adaptive suppression of high-frequency speckle noise, respectively, taking into account both the preservation of ground structure and accurate suppression of speckle noise. It also introduces a pixel-level random uncertainty estimator to realize uncertainty-guided cross-modal fusion, avoiding the propagation of residual noise to the fusion result and improving the fidelity of the fused image. In addition, a multi-scale two-stage fusion strategy is designed to effectively preserve the high-resolution spatial structure details of the fused image, and end-to-end training is achieved through a weighted combination of joint loss functions, stabilizing the iterative optimization training process and ensuring the convergence and robustness of the algorithm.

[0023] The algorithm architecture of this application is lightweight, containing only 0.86M parameters and 78.42G FLOPs. While ensuring the effect of SAR despeccing and optical image fusion processing, it has high computational efficiency and can be easily deployed in actual engineering scenarios of remote sensing image processing. It can provide high-quality image data support for various remote sensing applications such as land cover analysis, target detection, and environmental monitoring, and has good practical application value and engineering feasibility. Attached Figure Description One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments, and unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0024] Figure 1 This is a schematic flowchart of a SAR image despegging and optical image fusion method provided in an embodiment of this application.

[0025] Figure 2 This is a schematic diagram showing the low-frequency and high-frequency architecture design details of a SAR speckle removal module provided in an embodiment of this application.

[0026] Figure 3 A comparison diagram of fusion results provided in an embodiment of this application.

[0027] Figure 4 A comparison chart of quantitative indicators of fusion results provided in an embodiment of this application.

[0028] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] As can be seen from the background technology, existing speckle removal methods that combine frequency domain processing mostly adopt fixed threshold wavelet shrinkage strategies, which cannot adaptively suppress speckle noise; at the same time, they lack a pixel-level evaluation mechanism for the reliability of SAR features, making it difficult to avoid residual noise propagation in high uncertainty areas; and there is no method to integrate SAR speckle removal and optics into a closed-loop iterative and mutually reinforcing system, nor is there a targeted joint loss function to achieve end-to-end collaborative optimization of the two tasks, resulting in both failing to achieve optimal performance.

[0030] To address the aforementioned technical problems, this application provides a method, system, device, and medium for SAR image speckle removal and optical image fusion. The method includes the following steps: Step S1: Input the original noisy SAR image and the registered optical image. Convert the original noisy SAR image to the logarithmic domain to process multiplicative speckle noise. Simultaneously initialize the fusion latent representation, uncertainty map, iteration number t=1, and total iteration number T. The joint loss function combines uncertainty-aware pixel-level loss with structural similarity loss to construct a speckle removal loss. The fusion loss is constructed from three dimensions: intensity consistency, gradient structure, and chromaticity fidelity. The iterative mutual promotion loss is constructed using the final iteration's fusion latent representation as a reference. Step S2: Perform wavelet domain near-end update processing for the t-th iteration to complete SAR image speckle removal and calculate pixel-level uncertainty, obtaining the speckled SAR image and uncertainty map for this iteration. Step S3: Based on the speckled SAR image obtained in Step S2... The method sequentially performs uncertainty-guided fusion update and data consistency update on the R image and uncertainty map to obtain the fused latent representation and corrected SAR image after the current iteration. The data consistency update is based on a gradient-based correction strategy, using a learnable step size to correct the despeculiarized SAR image of the current iteration, maintaining the fidelity between the despeculiarized SAR image and the original noisy SAR image as a reference. Step S4: Determine if the iteration number satisfies t < T. If so, t = t + 1, and use the fused latent representation, uncertainty map, and corrected SAR image of the current iteration as prior information for the next iteration, returning to step S2 to continue iteration; otherwise, the iteration is completed. Step S5: Restore the corrected SAR image obtained in the final iteration to the original domain through exponential mapping to obtain a clean SAR image. Perform residual reconstruction with structure preservation on the fused latent representation of the final iteration to obtain the RGB domain SAR-optical fusion image. This method achieves mutual enhancement and optimization of SAR despeculiarization and optical fusion. Experiments on two large-scale SAR-optical datasets show that this application achieves optimal performance in both structure preservation and noise suppression, and can be widely applied to remote sensing image processing scenarios such as land cover analysis, target detection, and environmental monitoring.

[0031] The embodiments of this application will now be described in detail with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0032] See Figure 1 This application provides a method for SAR image despegging and optical image fusion, including the following steps: Step S1: Input the original noisy SAR image and the registered optical image. Convert the original noisy SAR image to the logarithmic domain to process multiplicative speckle noise. At the same time, initialize the fusion latent representation, uncertainty map, iteration number t=1 and total iteration number T. Among them, the joint loss function combines the uncertainty-aware pixel-level loss with the structural similarity loss to construct the speckle removal loss. The fusion loss is constructed from three dimensions: intensity consistency, gradient structure, and color fidelity. The iterative mutual promotion loss is constructed with the fusion latent representation of the final iteration as a reference.

[0033] Step S2: Perform wavelet domain near-end update processing for the t-th iteration to complete SAR image speckle removal and calculate pixel-level uncertainty, obtaining the speckled SAR image and uncertainty map for this iteration.

[0034] Step S3: Based on the despeculiar SAR image and uncertainty map obtained in step S2, perform uncertainty-guided fusion update and data consistency update in sequence to obtain the fused latent representation and corrected SAR image after this iteration. Among them, the data consistency update is based on the gradient-based correction strategy, which uses a learnable step size to correct the despeculiar SAR image of this iteration, and uses the original noisy SAR image as a reference to maintain the fidelity between the despeculiar SAR image and the original observation data.

[0035] Step S4: Determine whether the number of iterations satisfies t < T. If so, then t = t + 1, and use the fused latent representation, uncertainty map, and corrected SAR image of this iteration as prior information for the next iteration, and return to step S2 to continue iterating; otherwise, complete the iteration.

[0036] Step S5: Restore the corrected SAR image obtained from the final iteration to the original domain through exponential mapping to obtain a clean SAR image; perform residual reconstruction with structure preservation on the fusion latent representation of the final iteration to obtain a SAR-optical fusion image in the RGB domain.

[0037] This application provides a method for SAR image despeccing and optical image fusion, which solves the technical problems of excessive image smoothing, structural distortion, and noise propagation caused by decoupling despeccing and fusion in existing SAR-optical image fusion methods. First, this method models SAR despeccing and SAR-optical fusion as a joint maximum a posteriori probability problem, and achieves expansion-based iterative optimization through semi-quadratic splitting, thus constructing a closed-loop mutually reinforcing mechanism for despeccing and fusion. Figure 2 The diagram shows detailed architectural design of the SAR despeckle module at low and high frequencies. Figure 2 Figure (a) shows the architecture of the structure-aware low-frequency repairer, namely the low-frequency macrostructure reconstruction unit (LL-DeMamba). The input is the modulated SAR low-frequency subband, which is normalized, modeled by four-way SSM long distance, and fused with adaptive directional weights. The low-frequency subband is then reconstructed by combining the residual connection output. Figure 2 Figure (b) illustrates the architecture of the spatial adaptive high-frequency contraction phase, namely the high-frequency speckle suppression unit (HF-DeKan). It integrates a dual-input latent representation with the SAR high-frequency subband, extracts features through block statistical analysis, learns a nonlinear mapping via a KAN network, and outputs a pixel-level adaptive contraction threshold. The core of this application is a wavelet domain frequency decoupling and mutual-promotion denoiser. It combines Mamba base modules to complete long-range dependency modeling and reconstruction of the low-frequency structure, and uses a Kolmogorov-Arnold network (KAN) base module to achieve adaptive contraction suppression of high-frequency speckle. Simultaneously, it integrates a random uncertainty estimator based on dynamically weighted cross-modal guidance information with local reliability. Furthermore, this application designs an adaptive cross-guidance fusion head to selectively fuse SAR and optical features based on uncertainty, and uses a joint loss function to complete end-to-end training of the framework.

[0038] This application aims to achieve high-quality SAR speckle removal and SAR-optical cross-modal image fusion. The proposed solution is a unified algorithm architecture for SAR speckle removal and optical image fusion based on iterative mutual promotion. This algorithm achieves mutual reinforcement and optimization of the two tasks by constructing a closed-loop iterative mutual promotion mechanism for SAR speckle removal and optical image fusion, and achieves end-to-end training through a weighted joint loss function. The main process of this algorithm includes the following steps: Step S1: Input the original noisy SAR image and the registered optical image The original noisy SAR image Transform to the logarithmic domain to handle multiplicative speckle while initializing the fusion latent representation. Uncertainty diagram Current iteration number Total number of iterations Among them, the joint loss function Despeculiarization loss is constructed by combining pixel-level loss with uncertainty perception and structural similarity loss. The fusion loss is constructed from three dimensions: intensity consistency, gradient structure, and chromaticity fidelity. The iterative mutual promotion loss is constructed with reference to the fusion latent representation of the final iteration. The expression for the joint loss function is:

[0039] In the formula, , , These are preset weighted hyperparameters used to adjust the contribution weights of the three loss terms in joint training.

[0040] Step S2: Perform wavelet domain near-end update processing for the t-th iteration to complete SAR image speckle removal and calculate pixel-level uncertainty, obtaining the speckled SAR image for this iteration. Uncertainty diagram .

[0041] Step S3: Despeckled SAR image obtained in step S2 and uncertainty diagram The uncertainty-guided fusion update and data consistency update are performed sequentially to obtain the fusion latent representation after this iteration. Corrected SAR image Among them, data consistency updates are based on gradient-based correction strategies, utilizing learnable step sizes. For the speckle-reduced SAR images in this iteration Correction is performed to obtain the original noisy SAR image. To maintain the fidelity between the despeculiarized SAR image and the original observation data, the expression for data consistency update is:

[0042] In the formula, For the first The learnable step size for each iteration is obtained through adaptive learning during network training and is used to stabilize the structural restoration process of SAR images.

[0043] Step S4: Determine if the number of iterations is satisfied. If so, then let The fusion latent representation of this iteration Uncertainty diagram Corrected SAR image If the information is used as prior information for the next iteration, return to step S2 to continue the iteration; otherwise, complete the iteration.

[0044] Step S5: Obtain the corrected SAR image from the final iteration. Through exponential mapping Restored to the original domain, a clean SAR image is obtained. ; Fusion latent representation for the final iteration By performing structure-preserving residual reconstruction, a SAR-optical fusion image in the RGB domain is obtained. .

[0045] Furthermore, the specific processing steps for the wavelet domain near-end update in step S2 are as follows: Step S201: Correct the SAR image in the logarithmic domain. Perform Discrete Wavelet Transform (DWT) to decompose into low-frequency subbands. and high-frequency subband Among them, the low-frequency subband Includes large-scale macroscopic structural information of the scene, high-frequency subband It mainly includes speckle noise and detailed information on the edges of ground features.

[0046] Step S202: Combine the fusion latent representation from the previous iteration. and uncertainty diagram Uncertainty-aware modulation, residual injection, and long-range dependency modeling of Mamba base modules are sequentially performed on the low-frequency subband $LL$ to complete the low-frequency macrostructure reconstruction, resulting in the modulated and reconstructed low-frequency subband. The expression for uncertainty-aware modulation is:

[0047] In the formula, , For the fusion potential representation Affine parameters extracted after projection onto LL space. , Uncertainty diagram The generated uncertainty-aware correction parameters It is the Sigmoid activation function. The modulation intensity scale parameters are to be learned. These are residual injection coefficients used to preserve the energy characteristics of the original SAR image. This indicates element-wise multiplication.

[0048] Step S203: Calculate the high-frequency subband Local mean and local standard deviation Local statistical characteristics, and their fusion potential representation with the previous round. gradient features After concatenation, the data is input into the Kolmogorov-Arnold network (KAN), and the learning space adaptively shrinks to a threshold. The high-frequency subband (HF) is then subjected to soft contraction to suppress high-frequency speckle noise; among which, the spatial adaptive contraction threshold is... The calculation expression is:

[0049] The expression for soft shrinkage processing is:

[0050] In the formula, For the feature mapping function of the Kolmogorov-Arnold network, For symbolic functions, For absolute value operations, To perform the maximum value operation, This is the high-frequency subband after noise reduction.

[0051] Step S204: Reconstruct the low-frequency subband and the denoised high-frequency subband Perform inverse wavelet transform (IDWT) to obtain the speckled SAR image for this iteration. .

[0052] Step S205: Based on the original noisy SAR image Compared with this speckle-reduced SAR image residual and despeccable SAR images Compared with the previous round of integration potential The structural deviation was calculated using probabilistic statistical methods to obtain the pixel-level uncertainty map for this iteration. The uncertainty diagram Used to characterize the speckle removal reliability of each pixel location in a SAR image.

[0053] Furthermore, the specific processing steps for the uncertainty-guided fusion update described in step S3 are as follows: Step S301: Using a 3-level U-Net as a shared encoder, extract the speckle-reduced SAR images from the current iteration. and optical images Extract multi-scale hierarchical features from the data, denoted as... ,,in For feature scale level, For the overall scale level, For the first Scale-based SAR layering characteristics, For the first Scale-based optical layering features.

[0054] Step S302: Utilize the pixel-level uncertainty map from this iteration SAR layered features at various scales Confidence modulation is performed, and adaptive attenuation is applied to SAR feature regions with high uncertainty to obtain modulated SAR layered features. The expression for confidence modulation is:

[0055] In the formula, Uncertainty diagram Project to the Scale-based feature map. Modal projection is a 1×1 convolution used to achieve feature dimension matching.

[0056] Step S303: Using the multi-scale hierarchical features of optical images As a contextual prior, modulated SAR features With optical characteristics Multi-scale two-stage fusion is performed to obtain the fusion latent representation at each scale. .

[0057] Step S304: Using a bottom-up fusion approach combined with skip connections, aggregate the fusion latent representations at various scales. The global fusion latent representation of this iteration is obtained. At the same time, it preserves high-resolution spatial structure information and avoids the loss of details during the fusion process.

[0058] Furthermore, the specific steps of the multi-scale dual-stage fusion described in step S303 are as follows: Step S3031: Perform spatial gating fusion through the spatial gating mechanism. Modulated SAR features Perform selective screening, only those with optical characteristics Structure-matched SAR features are incorporated into optical features to obtain gated fusion features. The expression is:

[0059] In the formula, The optical feature reference potential is represented by the l-th scale, which is composed of optical layered features. Obtained through feature enhancement; Step S3032: Perform channel adaptive injection, using a channel excitation mechanism to extract the latent representation from the optical feature reference. Generate channel-level adaptive weighting coefficients Using this coefficient to perform gating fusion features After weighting, it is compared with the optical feature reference latent representation. Combined, and through residual blocks Feature optimization is performed to obtain the fusion latent representation at this scale. The expression is:

[0060] In the formula, For the first The channel-level adaptive weighting coefficients at the scale are determined by the channel excitation mechanism. The channel features are obtained through statistical learning and used to balance the fusion weights of cross-modal features.

[0061] In this application, the loss of blemish removal This is a combination of uncertainty-aware pixel-level loss and structural similarity loss, used to suppress speckle noise in SAR images while ensuring structural fidelity. Its expression is:

[0062] In the formula, This represents the total number of pixels in the SAR image. For pixel index, Let $i$ be the gray value of the $i$-th pixel in the speckle-reduced SAR image. For the clean SAR ground truth image, the first grayscale value of each pixel. For the uncertainty graph The value of each pixel, For uncertainty weighting coefficients, The weighting coefficients for structural similarity loss are... is the structural similarity (SSIM) loss function.

[0063] Fusion loss By constraining the fusion results from three dimensions—intensity consistency, gradient structure, and color fidelity—complementary information preservation between SAR and optical images is achieved. The expression for this is:

[0064] In the formula, , , These are the weighting coefficients for the losses in each dimension. For the Charbonnier penalty function, , , These are the brightness components of the fused image, optical image, and SAR image, respectively. For gradient calculation, For the target gradient features, It is an L1 norm. To fuse the RGB components of an image, For the RGB components of an optical image, This is a local constant used to avoid a denominator of 0.

[0065] Iterative mutual promotion loss The fusion latent representation of the final iteration For reference, the fusion latent representation of intermediate iterations The training process, which involves regularization and stable iterative optimization, is expressed as follows:

[0066] In the formula, For the final iterative fusion of latent representations Features after gradient separation It is an L2 norm.

[0067] The effectiveness of the SAR image despeccing and optical image fusion method provided in this application is verified below with reference to the accompanying drawings.

[0068] Figure 3 The fusion results are shown in the comparison chart. Tests were conducted on the WHU-OPT-SAR dataset, and the results were compared with eight leading image fusion methods. The fusion results of the method in this application showed the best visual performance.

[0069] Figure 4 A comparison chart of quantitative metrics for the fusion results is shown. Tests were conducted on the WHU-OPT-SAR dataset, and compared with eight leading methods in the field of image fusion. The method in this application significantly outperforms the comparative methods in all core quantitative metrics.

[0070] See Figure 5 Another embodiment of this application provides an electronic device, including: at least one processor 110; and a memory 111 communicatively connected to the at least one processor; wherein the memory 111 stores instructions executable by the at least one processor 110, the instructions being executed by the at least one processor 110 to enable the at least one processor 110 to perform any of the above method embodiments.

[0071] The memory 111 and processor 110 are connected via a bus, which may include any number of interconnecting buses and bridges, connecting various circuits of one or more processors 110 and memory 111. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 110 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 110.

[0072] Processor 110 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 111 can be used to store data used by processor 110 during operation.

[0073] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0074] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0075] Based on the above technical solutions, this application provides a method, system, device, and medium for SAR image speckle removal and optical image fusion. The method includes the following steps: Step S1: Input the original noisy SAR image and the registered optical image, convert the original noisy SAR image to the logarithmic domain to process multiplicative speckle, and initialize the fusion latent representation, uncertainty map, iteration number t=1, and total iteration number T; wherein, the joint loss function combines uncertainty-aware pixel-level loss with structural similarity loss to construct the speckle removal loss, constructs the fusion loss from three dimensions: intensity consistency, gradient structure, and chromaticity fidelity, and constructs the iterative mutual promotion loss with the fusion latent representation of the final iteration as a reference; Step S2: Perform wavelet domain near-end update processing for the t-th iteration to complete SAR image speckle removal and calculate pixel-level uncertainty, obtaining the speckled SAR image and uncertainty map of this iteration; Step S3: Based on the speckled SAR obtained in Step S2... The R image and uncertainty map are sequentially subjected to uncertainty-guided fusion update and data consistency update to obtain the fused latent representation and corrected SAR image after this iteration. Among them, the data consistency update is based on the gradient-based correction strategy, which uses a learnable step size to correct the speckled SAR image of this iteration, and uses the original noisy SAR image as a reference to maintain the fidelity between the speckled SAR image and the original observation data. Step S4: Determine whether the number of iterations satisfies t < T. If so, t = t + 1, and use the fused latent representation, uncertainty map and corrected SAR image of this iteration as the prior information for the next iteration, and return to step S2 to continue the iteration; otherwise, complete the iteration. Step S5: The corrected SAR image obtained in the final iteration is restored to the original domain through exponential mapping to obtain a clean SAR image. The fused latent representation of the final iteration is reconstructed with structure preservation residuals to obtain the SAR-optical fusion image in the RGB domain.

[0076] The SAR image despeccing and optical image fusion method provided in this application constructs a closed-loop iterative mutual promotion mechanism for SAR despeccing and optical image fusion, breaking the technical drawback of decoupling the two tasks in existing technologies. It realizes mutual reinforcement and optimization of despeccing and fusion, effectively solving the problems of over-smoothing and noise propagation in the fused image. At the same time, it adopts a frequency-decoupled wavelet domain near-end update strategy, combining Mamba base modules and KAN networks to complete long-range dependency modeling of low-frequency macrostructure and spatial adaptive suppression of high-frequency speckle noise, respectively, taking into account both the preservation of ground structure and accurate suppression of speckle noise. It also introduces a pixel-level random uncertainty estimator to realize uncertainty-guided cross-modal fusion, avoiding the propagation of residual noise to the fusion result and improving the fidelity of the fused image. In addition, a multi-scale two-stage fusion strategy is designed to effectively preserve the high-resolution spatial structure details of the fused image, and end-to-end training is achieved through a weighted combination of joint loss functions, stabilizing the iterative optimization training process and ensuring the convergence and robustness of the algorithm.

[0077] The algorithm architecture of this application is lightweight, containing only 0.86M parameters and 78.42G FLOPs. While ensuring the effect of SAR despeccing and optical image fusion processing, it has high computational efficiency and can be easily deployed in actual engineering scenarios of remote sensing image processing. It can provide high-quality image data support for various remote sensing applications such as land cover analysis, target detection, and environmental monitoring, and has good practical application value and engineering feasibility.

[0078] Those skilled in the art will understand that the above-described embodiments are specific examples of implementing this application, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of this application. Any person skilled in the art can make their own modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.

Claims

1. A method for SAR image speckle removal and optical image fusion, characterized in that, Includes the following steps: Step S1: Input the original noisy SAR image and the registered optical image. Convert the original noisy SAR image to the logarithmic domain to process multiplicative speckle noise. At the same time, initialize the fusion latent representation, uncertainty map, iteration number t=1 and total iteration number T. Among them, the joint loss function combines the uncertainty-aware pixel-level loss with the structural similarity loss to construct the speckle removal loss. The fusion loss is constructed from three dimensions: intensity consistency, gradient structure, and color fidelity. The iterative mutual promotion loss is constructed with the fusion latent representation of the final iteration as a reference. Step S2: Perform wavelet domain near-end update processing for the t-th iteration to complete SAR image speckle removal and calculate pixel-level uncertainty, obtaining the speckled SAR image and uncertainty map for this iteration; Step S3: Based on the despeculiar SAR image and uncertainty map obtained in step S2, perform uncertainty-guided fusion update and data consistency update in sequence to obtain the fused latent representation and corrected SAR image after this iteration. Among them, the data consistency update is based on the gradient-based correction strategy, and uses a learnable step size to correct the despeculiar SAR image of this iteration, using the original noisy SAR image as a reference to maintain the fidelity between the despeculiar SAR image and the original observation data. Step S4: Determine whether the number of iterations satisfies t < T. If so, then t = t + 1, and use the fused latent representation, uncertainty map, and corrected SAR image of this iteration as prior information for the next iteration, and return to step S2 to continue iterating; otherwise, complete the iteration. Step S5: Restore the corrected SAR image obtained from the final iteration to the original domain through exponential mapping to obtain a clean SAR image; perform residual reconstruction with structure preservation on the fusion latent representation of the final iteration to obtain a SAR-optical fusion image in the RGB domain.

2. The SAR image despegging and optical image fusion method according to claim 1, characterized in that, Step S1 specifically includes: Input raw noisy SAR image and the registered optical image ; Original noisy SAR image Transform to the logarithmic domain to handle multiplicative speckle while initializing the fusion latent representation. Uncertainty diagram Current iteration number Total number of iterations ; Among them, the joint loss function Despeculiarization loss is constructed by combining pixel-level loss with uncertainty perception and structural similarity loss. The fusion loss is constructed from three dimensions: intensity consistency, gradient structure, and chromaticity fidelity. The iterative mutual promotion loss is constructed with reference to the fusion latent representation of the final iteration. The expression for the joint loss function is: In the formula, , , These are preset weighted hyperparameters used to adjust the contribution weights of the three loss terms in joint training.

3. The SAR image despegging and optical image fusion method according to claim 1, characterized in that, The specific processing steps for the wavelet domain near-end update in step S2 are as follows: Step S201: Perform discrete wavelet transform on the logarithmic domain corrected SAR image to decompose it into low-frequency subband LL and high-frequency subband HF. Step S202: Combining the fusion latent representation and uncertainty map from the previous iteration, perform uncertainty-aware modulation, residual injection, and long-distance dependency modeling of Mamba base modules sequentially on the low-frequency subband LL to complete the low-frequency macrostructure reconstruction. Step S203: Calculate the local statistical features of the high-frequency subband HF, concatenate them with the gradient features of the previous round of fusion latent representation, and input them into the Kolmogorov-Arnold network. Learn the space adaptive shrinkage threshold and perform soft shrinkage processing on the high-frequency subband HF to complete high-frequency speckle noise suppression. Step S204: Perform inverse wavelet transform on the reconstructed low-frequency subband LL and the denoised high-frequency subband HF to obtain the speckle-free SAR image of this iteration. Step S205: Based on the residual between the original noisy SAR image and the current despecced SAR image, and the structural deviation between the despecced SAR image and the previous round of fusion latent representation, the pixel-level uncertainty map of this iteration is calculated.

4. The SAR image despegging and optical image fusion method according to claim 3, characterized in that, After completing the low-frequency macrostructure reconstruction in step S202, the modulated and reconstructed low-frequency subband is obtained. The expression for uncertainty-aware modulation is: In the formula, , For the fusion potential representation Affine parameters extracted after projection onto LL space. , Uncertainty diagram The generated uncertainty-aware correction parameters It is the Sigmoid activation function. The modulation intensity scale parameters are to be learned. These are residual injection coefficients used to preserve the energy characteristics of the original SAR image. This indicates element-wise multiplication.

5. The SAR image despegging and optical image fusion method according to claim 3, characterized in that, In step S203, the high-frequency sub-band is calculated. Local mean and local standard deviation Local statistical characteristics, and their fusion potential representation with the previous round. gradient features After concatenation, the data is input into the Kolmogorov-Arnold network (KAN), and the learning space adaptively shrinks to a threshold. The high-frequency subband (HF) is then subjected to soft contraction to suppress high-frequency speckle noise; among which, the spatial adaptive contraction threshold is... The calculation expression is: The expression for soft shrinkage processing is: In the formula, For the feature mapping function of the Kolmogorov-Arnold network, For symbolic functions, For absolute value operations, To perform the maximum value operation, This is the high-frequency subband after noise reduction.

6. The SAR image despegging and optical image fusion method according to claim 1, characterized in that, The specific processing steps for the uncertainty-guided fusion update in step S3 are as follows: Step S301: Using a 3-level U-Net as a shared encoder, multi-scale hierarchical features are extracted from the speckled SAR image and optical image of this iteration, respectively. Step S302: Using the pixel-level uncertainty map of this iteration, confidence modulation is performed on the SAR layered features at each scale, and adaptive attenuation is performed on the SAR feature regions with high uncertainty. Step S303: Using the multi-scale hierarchical features of the optical image as the context prior, the modulated SAR features and optical features are fused in a multi-scale two-stage process to obtain the fusion latent representation at each scale. Step S304: Using a bottom-up fusion approach combined with skip connections, the fusion latent representations at each scale are aggregated to obtain the global fusion latent representation for this iteration, while preserving high-resolution spatial structure information.

7. The SAR image despegging and optical image fusion method according to claim 6, characterized in that, The specific steps of the multi-scale two-stage fusion described in step S303 are as follows: Step S3031: Perform spatial gating fusion. Selectively screen the modulated SAR features through the spatial gating mechanism, and only integrate the SAR features that match the optical feature structure into the optical features to obtain the gating fused features. Step S3032: Perform channel adaptive injection. Generate channel-level adaptive weighting coefficients from optical features through a channel excitation mechanism. Use these coefficients to weight the gated fusion features and combine them with the optical features. Then, perform feature optimization through residual blocks to obtain the fusion latent representation at this scale.

8. A SAR image despeculiaration and optical image fusion system, the system being used to implement the SAR image despeculiaration and optical image fusion method as described in any one of claims 1 to 7, characterized in that, The system includes: an iterative mutual promotion loss construction module, an iteration module, an update module, a judgment module, and an output module, connected sequentially; wherein, The iterative mutual promotion loss construction module is used to input the original noisy SAR image and the registered optical image, transform the original noisy SAR image to the logarithmic domain to process multiplicative speckle noise, and initialize the fusion latent representation, uncertainty map, iteration number t=1 and total iteration number T. Among them, the joint loss function combines uncertainty-aware pixel-level loss with structural similarity loss to construct speckle removal loss, constructs fusion loss from three dimensions: intensity consistency, gradient structure, and color fidelity, and constructs iterative mutual promotion loss with the final iterative fusion latent representation as a reference. The iterative module is used to perform wavelet domain near-end update processing for the t-th iteration, complete SAR image despeccing and calculate pixel-level uncertainty, and obtain the despecced SAR image and uncertainty map for this iteration; The update module is used to perform uncertainty-guided fusion update and data consistency update in sequence according to the despeculiar SAR image and uncertainty map output by the iteration module, so as to obtain the fused latent representation and the corrected SAR image after this iteration update; wherein, the data consistency update is based on the gradient-type correction strategy, and uses a learnable step size to correct the despeculiar SAR image of this iteration, and uses the original noisy SAR image as a reference to maintain the fidelity between the despeculiar SAR image and the original observation data. The judgment module is used to determine whether the number of iterations satisfies t < T. If so, then t = t + 1, and the fused latent representation, uncertainty map, and corrected SAR image of this iteration are used as prior information for the next iteration, and the iteration is returned to the iteration module to continue the iteration; otherwise, the iteration is completed. The output module is used to restore the corrected SAR image obtained from the final iteration to the original domain through exponential mapping to obtain a clean SAR image; and to perform structure-preserving residual reconstruction on the fusion latent representation of the final iteration to obtain a SAR-optical fusion image in the RGB domain.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the SAR image despeccing and optical image fusion method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the SAR image despiking and optical image fusion method according to any one of claims 1 to 7.

Images