A method and system for SAR and optical image fusion based on morphological feature decomposition

Abstract

The application discloses a SAR and optical image fusion method and system based on morphological feature decomposition, and belongs to the field of image processing. First, the optical image and the SAR image are preprocessed, then, a differentiated multi-layer feature decomposition strategy is adopted to decompose the image into main structure information, morphological feature information and edge detail information, wherein the main structure information is extracted by a local total variation method, the morphological feature information is obtained by a generalized total variation model and residual suppression, and the edge detail information is decomposed by jointing bilateral filtering and a least square model. Then, different feature layers are respectively fused by a saliency guided fusion, a local weighted fusion and a fusion strategy combining edge preservation and sparse detail compensation. Finally, color information is restored by IHS inverse transformation to obtain a fused image with clear structure, complete morphology and rich details, so that fine fusion from structure to morphology and then to details is realized.

Description

Technical Field

[0001] This invention belongs to the field of image processing, specifically relating to a method and system for fusing SAR and optical images based on morphological feature decomposition. Background Technology

[0002] The fusion technology of Synthetic Aperture Radar (SAR) and optical imagery leverages the complementary advantages of these two types of remote sensing data to achieve a more comprehensive and accurate representation of surface targets. SAR imagery offers all-weather, all-time acquisition capabilities, enabling stable imaging in complex terrain and adverse weather conditions, and providing clear structural and geometric information. Optical imagery, on the other hand, directly reflects the color, texture, and detailed features of ground objects, offering excellent interpretability. In practical applications, optical imagery is susceptible to environmental factors such as clouds, fog, and shadows, leading to missing or obscured target information. SAR imagery, with its strong penetrating power, effectively compensates for this deficiency. As SAR imaging resolution continues to improve, ultra-high-resolution SAR imagery is gradually exhibiting more complex three-dimensional structural features, such as the three-dimensional outlines and geometric shapes of buildings. However, this complex structural information can also increase the difficulty of image interpretation to some extent, and even interfere with the accurate identification and extraction of targets. In contrast, the rich texture and morphological information contained in optical imagery helps enhance target identification capabilities. Therefore, by finely fusing SAR and optical images, we can not only fully leverage the advantages of SAR images in structural representation and penetration imaging, but also effectively preserve the color, texture, and detail information of optical images, thereby significantly improving the accuracy and reliability of remote sensing images in applications such as land cover classification, environmental monitoring, and urban change detection.

[0003] Despite significant progress in SAR and optical image fusion technology in recent years, numerous challenges remain. Existing fusion methods can be broadly categorized into traditional methods and deep learning-based methods. Traditional fusion methods, such as Principal Component Analysis (PCA), IHS Transform, and Laplacian Pyramid, can achieve structural information fusion to a certain extent, but struggle to achieve an effective balance between detail preservation and noise suppression. This often results in the loss of color and texture information in optical images or the introduction of speckle noise into SAR images. Deep learning-based fusion methods have a clear advantage in feature representation capabilities, particularly in target extraction and change detection in complex scenes. However, they are heavily reliant on large-scale, high-quality training data and still have limitations in ultra-high-resolution image fusion and SAR speckle noise suppression. Therefore, how to effectively suppress noise while maximizing the preservation and recovery of detail information during the fusion of SAR and optical images at different resolutions remains a critical technical problem that urgently needs to be solved. Summary of the Invention

[0004] To address the problems existing in the background technology, this invention proposes a SAR and optical image fusion method based on morphological feature decomposition, aiming to achieve an effective balance between noise suppression and detail preservation during the fusion of images of different resolutions. First, the input optical and SAR images are preprocessed. The optical image uses IHS transform to separate brightness and color information to reduce the destruction of color information during fusion; the SAR image employs a combination of grayscale equalization and side-window filtering to effectively suppress speckle noise while enhancing structural information. Second, this invention proposes a differentiated multi-layer feature decomposition strategy, decomposing the preprocessed image information into three categories: principal structure information, morphological feature information, and edge detail information. Principal structure information is extracted using the local total variation method to characterize the overall structure and stable regions of the image; morphological feature information is obtained through a generalized total variation model combined with a residual suppression strategy to enhance the shape and geometric features of the target; edge detail information is decomposed using a combination of bilateral filtering and a least squares model to accurately extract detailed edges and reduce noise interference. Based on this, corresponding fusion strategies were designed for different feature layers: the main structure layer adopted a saliency-guided fusion method to highlight key structural information; the morphological feature layer adopted a local weighted fusion strategy to achieve adaptive integration of morphological information; and the edge detail layer combined an edge preservation mechanism with a sparse detail compensation strategy to achieve high-fidelity detail restoration. Finally, the fused brightness information and the original color information were reconstructed through IHS inverse transform to obtain a fused image with clear structure, complete morphology, and rich details, achieving refined fusion from the structural layer, morphological layer to the detail layer.

[0005] Through innovative methods, the SAR and optical fusion method based on morphological feature decomposition effectively suppresses noise while preserving sufficient details and achieving high-fidelity detail restoration, providing strong technical support for the fine fusion and collaborative processing and optimization of multimodal images.

[0006] This invention proposes a SAR and optical image fusion method based on morphological feature decomposition to solve the problem of fusing multimodal images of SAR and optical images at different resolutions.

[0007] The technical solution adopted in this invention is: a method for fine fusion of SAR and optical images at different resolutions based on morphological feature decomposition and sparse detail compensation, comprising the following steps: Preprocessing of optical and SAR images; Extract main structure information, morphological feature information, and edge detail information from optical images and SAR images; The main structure information is fused using a saliency-guided fusion method to obtain the main structure fusion result; The morphological feature information is fused using a local adaptive weighted fusion method to obtain the morphological feature fusion result; An edge detail fusion method combining edge preservation enhancement and sparse detail compensation is used to fuse edge detail information to obtain the edge detail fusion result; The main structure fusion result, morphological feature fusion result, and edge detail fusion result are weighted and superimposed to obtain the final fusion result; The fusion result is combined with the color components of the optical image and then subjected to an inverse IHS transformation to output the final SAR and optical image fusion result.

[0008] Furthermore, the preprocessing includes: the optical image undergoes IHS transformation, the SAR image is first subjected to grayscale equalization processing, and then the side window filtering is used to suppress speckle noise and enhance structural edges, thereby obtaining a preprocessed SAR image with a clear structure; the preprocessed optical image and SAR image are spatially aligned and scaled to ensure that the two types of images maintain consistency in spatial resolution and pixel grid.

[0009] Furthermore, a multi-scale decomposition method based on morphological feature decomposition is used to extract morphological feature information from optical images and SAR images; The extraction of morphological feature information includes two sub-processes: initial structure map extraction and structural residual signal construction followed by Gaussian guided suppression. The initial structure map is obtained by structural decomposition using a total variation model with Gaussian weights. F， The initial structure diagram The following optimization model was used to obtain:

[0010] in, and They are x and y Directional difference operator; and It is a diagonal matrix that weights the gradient, reflecting the adaptive smoothing weights at each position; It is the identity matrix; Regularization intensity factor I This represents the original image.

[0011] Furthermore, in the process of constructing the structural residual signal and Gaussian-guided morphological suppression, the original image is subtracted from the initial structure map to construct the structural residual signal. RThis process extracts subtle differences not captured by the initial structure diagram; subsequently, Gaussian smoothing is applied to the structural residual signal to extract background trends and suppress high-frequency noise; and morphological feature information is obtained by suppressing background trends in the structural residual signal.

[0012] in, Represents the morphological features of an image. This represents the initial structure diagram of the image. This indicates that for each point in the residual signal R The result after Gaussian filtering.

[0013] Furthermore, edge detail information is extracted using the BLF-LS method, which combines a bilateral filter and a least squares model. First, pixel saliency maps and structural saliency maps are calculated separately. Then, combined with a total variational model, the fusion is completed under the dual guidance of the pixel domain and the gradient domain to obtain the fusion result of the main structure information.

[0014] Furthermore, the morphological feature information fusion adopts a local adaptive weighted fusion strategy that combines structural similarity and image average gradient, specifically including: First, the image is locally segmented using a sliding window, and the structural similarity index SSIM and the average gradient index AG are calculated. Based on these, the local fusion weights are obtained to achieve local weighted fusion of morphological feature information. AG is used to calculate the average gradient of an image patch. The formulas for calculating the AG value of SAR and optical image patches are as follows:

[0015] in, X Represents the input image patch. N Indicates the size of the image patch; and Indicates the image in and Partial derivatives in the direction, i Indicates the index of the image patch. Represents the calculation of SAR image patches AG value, Represents the computation of optical image blocks AG value; The structural similarity index and the average gradient index of the image are normalized to eliminate the dimensional differences between the different indices, and the local adaptive fusion weights are calculated based on the normalized indices; the calculation method of the local fusion weights is expressed as follows:

[0016] in, This indicates normalization processing. and These represent the local weighting coefficients of the SAR image and the optical image, respectively. Indicators representing the structural similarity of SAR images Indicators representing the structural similarity of optical images; Finally, based on the aforementioned local adaptive fusion weights, the morphological feature information of the SAR image and the morphological feature information of the optical image are locally weighted and fused to obtain the fused morphological feature information, the expression of which is:

[0017] in, This represents the morphological features after fusion. This represents the morphological features of SAR. It represents the morphological features of optical images.

[0018] Furthermore, the edge detail information fusion adopts a strategy that combines edge preservation enhancement and sparse detail compensation, including edge preservation enhancement based on sub-window standard deviation filter and detail compensation based on sparse representation; During edge-preserving enhancement, the statistical characteristics of pixel grayscale distribution within the sub-window are analyzed to achieve adaptive enhancement of edges, corners, and textured regions; for any pixel in the image... In its neighborhood window Internal calculation of gray standard deviation:

[0019] in, Represents the neighborhood window The standard deviation of the grayscale of an inner pixel; the larger the standard deviation, the stronger the contrast and texture of that area. The value represents the average value of the pixels within the window, and N represents the neighborhood window. The number of inner pixels, Represents the neighborhood window Inner i The intensity value of each pixel; The neighborhood window is divided into multiple sub-windows, and the standard deviation of each sub-window is calculated to enhance structural sensitivity. Based on this, an edge preservation factor is defined to achieve local edge enhancement. Multi-scale response analysis of the image is performed based on the sub-window standard deviation filter (SSF) to obtain detailed features at different scales, resulting in an edge-preserving enhanced detail layer, the expression of which is:

[0020]

[0021] in,D Indicates edge detail information, Represents the SSF operator. and These represent fine-scale detail layers and small-scale detail layers, respectively. This represents the enhanced detail layer after multi-scale image processing. and These are the weight coefficients for the fine-scale detail layer and the small-scale detail layer, respectively. and These represent the enhanced detail layers for SAR and optical images, respectively. This indicates that the final edge retains the enhancement layer.

[0022] Furthermore, during the sparse detail compensation process, robust principal component analysis is performed on the edge detail information, and K-SVD dictionary learning and orthogonal matching pursuit are used to sparsely reconstruct the sparse residual terms to recover the true high-frequency detail information; its expression is:

[0023]

[0024] in, and Indicates optical image blocks With SAR image patches OMP encoding is performed to obtain sparse coefficients; Represents the image block dictionary. This represents the sparsity coefficients after fusion. This represents the detail block of the final sparse reconstruction. It corresponds to the image block dictionary. Part of Represents the local mean of an image patch. x and y Represents the global pixel coordinates on the image. i Indicates the index number of the image patch. Indicates the first i Each reconstruction block in pixels Detailed location information; Indicates the first i The weight of each block at that pixel position. Representing coordinates Detail compensation layer at the location; Keep the edges reinforced. With detail compensation layer The fusion process yields a fused portion containing high-precision edge details and realistic texture information. .

[0025] Furthermore, the main structure information fusion results morphological feature fusion results Blending results with edge details Pixel-by-pixel weighted overlay is performed to obtain the final SAR-optical fusion image. F Its mathematical expression is as follows:

[0026] in, This indicates the final fusion result. W M The weights of the morphological feature information components are represented. This indicates the result of morphological feature fusion. W D Indicates the weight of the edge detail information components. This indicates the result of edge detail blending.

[0027] The present invention also provides a SAR and optical image fusion system based on morphological feature decomposition, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the SAR and optical image fusion method based on morphological feature decomposition as described in the above technical solution.

[0028] Compared with the prior art, the present invention has the following advantages and beneficial effects: This invention proposes a refined fusion method for SAR-optical images of different resolutions. Based on morphological feature decomposition and sparse detail compensation, this method constructs a fusion framework that perceives different resolution features. This framework includes differentiated multi-layer feature decomposition and a hierarchical complementary fusion strategy, introducing the concept of "morphological feature information" as a mid-frequency feature. In the fusion stage, an adaptive weighting mechanism based on structural similarity (SSIM) and average gradient (AG) is designed to effectively balance the contradiction between noise suppression and detail preservation during the fusion process. Furthermore, this invention proposes an edge information fusion strategy combining edge preservation enhancement and sparse detail compensation. An edge preservation factor is constructed by introducing a sub-window standard deviation filter (SSF), and sparse detail compensation is achieved by combining robust principal component analysis (RPCA) and K-SVD algorithms, effectively suppressing high-frequency noise while restoring realistic and continuous texture details. Experimental results show that the proposed method can significantly improve the structural consistency, detail fidelity, and visual naturalness of the fused images in SAR-optical image fusion tasks of different resolutions, exhibiting good robustness and adaptability in complex scenes. Attached Figure Description

[0029] Figure 1 : Flowchart of the method of this invention; Figure 2: Schematic diagram of image morphological feature information decomposition and extraction; Figure 3 : Schematic diagram of image morphological feature information fusion; Figure 4 : Schematic diagram of image edge detail information fusion; Figure 5 Optical-SAR image fusion dataset, where A is a partial display of the Sentinel-1 image dataset and B is a partial display of the Capella Space image dataset. (a) is an urban scene, (b) is a farmland scene, (c) is a forest scene, (d) is a suburb, and (e) is a water scene. Figure 6 The images are optical-SAR images fusion results, where A is the fusion result of the Sentinel-1 image dataset, B is the fusion result of the Capella Space image dataset, (a) is the fusion result of urban scenes, (b) is the fusion result of farmland scenes, (c) is the fusion result of forest scenes, (d) is the fusion result of suburban scenes, (e) is the fusion result of water areas, and (f) is the fusion result of roads. Detailed Implementation To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0030] Please see Figure 1 The flowchart illustrates a SAR-optical image fusion method based on morphological feature decomposition provided by this invention, comprising the following steps: Step 1: Preprocess the optical and SAR images. The optical images are transformed by IHS, and the SAR images are processed by grayscale equalization and side window filtering to suppress speckle noise and enhance structural information. Spatial alignment and scale unification are then performed on the preprocessed optical and SAR images.

[0031] Preferably, in step 1, the SAR image is first subjected to grayscale equalization to improve grayscale distribution, and then side-window filtering is used to suppress speckle noise and enhance structural edges, thereby obtaining a preprocessed SAR image with clear structure. The preprocessed optical image and SAR image are then spatially aligned and scaled to ensure that the two types of images maintain consistency in spatial resolution and pixel grid, so as to avoid spatial misalignment during feature decomposition and fusion.

[0032] Step 2: Employ a structure decomposition method based on local total variation to extract principal structure information from optical and SAR images, which is used to characterize low-frequency stable structure features.

[0033] Preferably, in step 2, the main structure information is obtained by evaluating pixel stability using the pixel grayscale change rate, and by combining the original image with its smoothed image, stable structural regions in the image are extracted.

[0034] Step 3: Employ a multi-scale decomposition method based on morphological feature decomposition to extract morphological feature information from optical and SAR images, which is used to characterize the mid-frequency geometric structure and target morphological features.

[0035] As a preferred option, such as Figure 2 As shown, step 3, the extraction of morphological feature information includes two sub-processes: initial structure map extraction and structural residual signal construction followed by Gaussian-guided morphological suppression. Specifically, in the initial structure map extraction process, Gaussian weighting is introduced into the total variation regularization to adaptively constrain the image gradient, suppressing texture and detail interference and retaining only the main structural components in the image, thereby obtaining the initial structure map. F .

[0036] Furthermore, the initial structural diagram This can be obtained through the following optimization model:

[0037] in, and They are x and y Directional difference operator; and It is a diagonal matrix that weights the gradient, reflecting the adaptive smoothing weights at each position; It is the identity matrix; Regularization intensity factor I This represents the original image.

[0038] In the process of constructing structural residual signals and Gaussian-guided morphological suppression, the original image is subtracted from the initial structure map to construct the structural residual signal. R This process extracts subtle differences not captured by the initial structure diagram; subsequently, the structural residual signal is Gaussian smoothed to extract background trends and suppress high-frequency noise. The Gaussian smoothing process can be represented as:

[0039] in, Let represent a two-dimensional Gaussian function, where The standard deviation of the Gaussian function determines the smoothness of the Gaussian filter. This indicates the point where the Gaussian function is applied in the image. This indicates that for each point in the residual signal R The result after Gaussian filtering.i and j This represents the local spatial coordinates within the Gaussian filter kernel (sliding window).

[0040] By suppressing the background trend of the structural residual signal, morphological feature information is obtained:

[0041] in, It represents the morphological features of the image. This represents the initial structural information of the image.

[0042] Step 4: An edge-preserving decomposition of details method is used to extract edge detail information from optical and SAR images to characterize high-frequency edges and texture details.

[0043] As a preferred option, in step 4, the edge detail information is obtained by using the BLF-LS method, which combines a bilateral filter (BLF) and a least squares (LS) model, to preserve edge details and achieve fine decomposition.

[0044] Step 5: The main structure information is fused using a saliency-guided fusion method to obtain the main structure fusion result.

[0045] Preferably, in step 5, the main structure information fusion process first calculates the pixel saliency map and the structure saliency map separately, and then combines the total variation model to complete the fusion under the dual guidance of the pixel domain and the gradient domain, thus obtaining the main structure information fusion result. .

[0046] Step 6: The morphological feature information is fused using a local adaptive weighted fusion method to obtain the morphological feature fusion result.

[0047] Preferably, in step 6, such as Figure 3 As shown, the morphological feature information fusion adopts a local adaptive weighted fusion strategy that combines structural similarity and image average gradient to enhance the local morphological and detail representation capabilities while maintaining structural consistency. First, a sliding window strategy is used to locally divide the morphological feature information of the SAR image, the optical image, and the pre-fused image, dividing the image into multiple corresponding image blocks to obtain morphological feature information within local regions. Second, a structural similarity index and an image average gradient index are calculated for each pair of corresponding image blocks. Structural similarity measures the structural consistency between the SAR image and the optical image within a local region, while the image average gradient characterizes edge variations and detail intensity within the local region. The structural similarity index can be expressed as:

[0048] in, Represents two image blocks X and Y The structural similarity evaluation value between them. and Representing image blocks X and Y The mean value of all pixel values ​​within the range; and Representing image blocks X and Y The variance; Represents image blocks X and Y Covariance between them; , A constant term is used to prevent the denominator from being zero. , and The first segment representing the SAR, optical, and preliminary fused images i Image blocks.

[0049] AG (Average Gradient) is used to calculate the average gradient of an image patch. This metric is sensitive to image texture and edge details, and plays a crucial role in improving the sharpness and edge contrast of the fused image. The formulas for calculating the AG value of SAR and optical image patches are as follows:

[0050] in, X Represents the input image patch. N Indicates the size of the image patch; and Indicates the image in and Partial derivatives in the direction, i Indicates the index of the image patch. Represents the calculation of SAR image patches AG value, Represents the computation of optical image blocks The AG value.

[0051] The structural similarity index and the average gradient index of the image are normalized to eliminate the dimensional differences between the different indices, and local adaptive fusion weights are calculated based on the normalized indices. The calculation method of the local fusion weights can be expressed as:

[0052] in, This indicates normalization processing. and These represent the local weighting coefficients for SAR and optical images, respectively.

[0053] Finally, based on the aforementioned local adaptive fusion weights, the morphological feature information of the SAR image and the morphological feature information of the optical image are locally weighted and fused to obtain the fused morphological feature information, the expression of which is:

[0054] in, This represents the morphological features after fusion. This represents the morphological features of SAR. It represents the morphological features of optical images.

[0055] Step 7: The edge detail information is fused using a fusion method that combines edge preservation enhancement and sparse detail compensation to obtain the edge detail fusion result.

[0056] As a preferred option, such as Figure 4 As shown, step 7 employs a fusion strategy combining edge-preserving enhancement and sparse detail compensation to address the detail loss and edge breakage issues present in traditional detail fusion methods. This includes two sub-processes: edge-preserving enhancement based on a sub-window standard deviation filter and sparse representation-driven detail compensation.

[0057] During edge-preserving enhancement, adaptive enhancement of edges, corners, and textured regions is achieved by analyzing the statistical characteristics of pixel grayscale distribution within a sub-window. For any pixel in the image... In its neighborhood window Internal calculation of gray standard deviation:

[0058] in, Represents the neighborhood window The standard deviation of the grayscale of an inner pixel; the larger the standard deviation, the stronger the contrast and texture of that area. The value represents the average value of the pixels within the window, and N represents the neighborhood window. The number of inner pixels, Represents the neighborhood window Inner i The intensity value of each pixel.

[0059] The neighborhood window is divided into multiple sub-windows, and the standard deviation of each sub-window is calculated to enhance structural sensitivity. Based on this, an edge preservation factor is defined to achieve local edge enhancement, the expression of which is:

[0060] in, , , , These represent the standard deviations of the four sub-windows into which the local neighborhood window containing the center pixel is divided. This represents the maximum standard deviation among the four sub-windows. This represents the minimum standard deviation among the four sub-windows. This represents the enhanced pixel value. Represents pixels Edge preservation weights, image average strength, It is a tiny positive value to prevent the denominator from being 0.

[0061] Multi-scale response analysis of the image is performed based on the sub-window standard deviation filter (SSF) to obtain detailed features at different scales, resulting in an edge-preserving enhanced detail layer, the expression of which is:

[0062]

[0063] in, D It indicates edge detail information. Represents the SSF operator. and These represent fine-scale detail layers and small-scale detail layers, respectively. This represents the enhanced detail layer after multi-scale image processing. and These are the weight coefficients for the fine-scale detail layer and the small-scale detail layer, respectively. and These represent the enhanced detail layers for SAR and optical images, respectively. This indicates that the final edge retains the enhancement layer.

[0064] To compensate for high-frequency detail information that may be lost during the fusion process, a sparse representation-driven detail compensation strategy is introduced for edge detail information. First, for edge detail information... Robust Principal Component Analysis (RPCA) was performed to decompose the components into low-rank principal terms. With sparse residuals Its optimization model is expressed as:

[0065] in, D This represents the edge detail information obtained during the eigenvalue decomposition stage. The nuclear norm of a matrix is ​​denoted as . Represents the matrix Norm, L It is a low-rank term. S For sparse residual terms, λ>0 is used to balance the two terms.

[0066] Subsequently, the sparse residual terms Sliding block extraction is performed, a redundant dictionary is constructed using the K-SVD method, and orthogonal matching pursuit (OMP) is used to sparsely encode SAR image blocks and optical image blocks, achieving maximum sparse coding. The energy criterion is used to select coefficients, resulting in sparsely reconstructed detail blocks. The compensated detail image is then recovered using an overlap-weighted averaging method, the expression of which is:

[0067]

[0068] in, and Indicates optical image blocks With SAR image patches OMP encoding is performed to obtain sparse coefficients. Represents the image block dictionary. This represents the sparsity coefficients after fusion. This represents the detail block of the final sparse reconstruction. It corresponds to the image block dictionary. Part of Represents the local mean of an image patch. x and y Represents the global pixel coordinates on the image. i Indicates the index number of the image patch. Indicates the first i Each reconstruction block in pixels Detailed location information; Indicates the first i The weight of each block at that pixel position. Representing coordinates Detailed compensation layer at the location.

[0069] Keep the edges reinforced. With detail compensation layer The fusion process yields a fused portion containing high-precision edge details and realistic texture information. .

[0070] Step 8: Weight and superimpose the main structure fusion result, morphological feature fusion result, and edge detail fusion result to obtain the fusion result.

[0071] As a preferred option, step 8 will fuse the main structure information results. morphological feature fusion results Blending results with edge details Pixel-by-pixel weighted stacking is performed to obtain the final SAR-optical fused image F. Its mathematical expression is as follows:

[0072] in, This indicates the final fusion result. W M The weights of the morphological feature information components are represented. This indicates the result of morphological feature fusion. W D Indicates the weight of the edge detail information components. This indicates the result of edge detail blending.

[0073] Step 9: Perform an inverse IHS transformation on the fusion result and the color components of the optical image to output the final SAR and optical image fusion result.

[0074] Preferably, the IHS inverse transform described in step 9 introduces the fused brightness information into the color space while maintaining the original color relationships of the optical image. This enhances structural and detail information while avoiding damage to the color information of the optical image during the fusion process. The output fused image possesses both the structural expressiveness of SAR imagery and the color and visual interpretability of optical imagery, improving the overall visual quality and practical value of the fused image.

[0075] Step 10: Evaluate the fusion effect of optical-SAR images using eight indicators: information entropy (EN), standard deviation (SD), spatial frequency (SF), average gradient (AG), correlation coefficient (CC), sum of differential correlations (SCD), peak signal-to-noise ratio (PSNR), and visual fidelity (VIF). This invention uses nine sets of optical-SAR images to test the algorithm's performance; the dataset is available at [link to dataset]. Figure 5 The final fusion result is as follows Figure 6 As shown in the figure. For each image pair, a quantitative test was performed using eight indicators. The proposed fusion method is named the MSFusion algorithm and is compared with several optimal image matching methods (LP, DTCWT, Hybrid-MSD, VSFF, IFCNN, U2Fusion, MMIF-EMMA, ITFuse, and MulFS-CAP). The comparison results are shown in Table 1.

[0076] Table 1 compares the average values ​​of six evaluation metrics for 10 methods on the Opt-Sar-Fusion dataset. The best results are underlined, and the second-best results are indicated in italics.

[0077]

[0078] As shown in Table 1, based on the quantitative results, MSFusion excels in key visual dimensions such as information entropy, contrast, sharpness, and texture in optical-SAR image data. It exhibits good structural consistency and perceptual fidelity, and performs well in multi-source heterogeneous image fusion tasks. This verifies the method's advancement, applicability, and potential for widespread application, making it suitable for high-precision remote sensing interpretation scenarios such as urban modeling, disaster monitoring, and land cover classification.

[0079] On the other hand, embodiments of the present invention also provide a SAR and optical image fusion system based on morphological feature decomposition, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the SAR and optical image fusion method based on morphological feature decomposition as described in the above technical solution.

[0080] It should be understood that any parts not described in detail in this specification belong to the prior art.

[0081] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A SAR and optical image fusion method based on morphological feature decomposition, characterized in that, include: Preprocessing of optical and SAR images; Extract main structure information, morphological feature information, and edge detail information from optical images and SAR images; The main structure information is fused using a saliency-guided fusion method to obtain the main structure fusion result; The morphological feature information is fused using a local adaptive weighted fusion method to obtain the morphological feature fusion result; An edge detail fusion method combining edge preservation enhancement and sparse detail compensation is used to fuse edge detail information to obtain the edge detail fusion result; The main structure fusion result, morphological feature fusion result, and edge detail fusion result are weighted and superimposed to obtain the final fusion result; The fusion result is combined with the color components of the optical image and then subjected to an inverse IHS transformation to output the final SAR and optical image fusion result.

2. The SAR and optical image fusion method based on morphological feature decomposition according to claim 1, characterized in that: The preprocessing includes: optical images undergoing IHS transformation, SAR images undergoing grayscale equalization, and then side-window filtering to suppress speckle noise and enhance structural edges, thereby obtaining a preprocessed SAR image with a clear structure; spatial alignment and scale unification are performed on the preprocessed optical images and SAR images to ensure that the two types of images maintain consistency in spatial resolution and pixel grid.

3. The SAR and optical image fusion method based on morphological feature decomposition according to claim 1, characterized in that: A multi-scale decomposition method based on morphological feature decomposition is used to extract morphological feature information from optical images and SAR images; The extraction of morphological feature information includes two sub-processes: initial structure map extraction and structural residual signal construction followed by Gaussian guided suppression. The initial structure map is obtained by structural decomposition using a total variation model with Gaussian weights. F， The initial structure diagram The following optimization model was used to obtain: in, and They are x and y Directional difference operator; and It is a diagonal matrix that weights the gradient, reflecting the adaptive smoothing weights at each position; It is the identity matrix; Regularization intensity factor I This represents the original image.

4. The SAR and optical image fusion method based on morphological feature decomposition according to claim 3, characterized in that: In the process of constructing structural residual signals and Gaussian-guided morphological suppression, the original image is subtracted from the initial structure map to construct the structural residual signal. R This process extracts subtle differences not captured by the initial structure diagram; subsequently, Gaussian smoothing is applied to the structural residual signal to extract background trends and suppress high-frequency noise; and morphological feature information is obtained by suppressing background trends in the structural residual signal. in, Represents the morphological features of an image. This represents the initial structure diagram of the image. This indicates that for each point in the residual signal R The result after Gaussian filtering.

5. The SAR and optical image fusion method based on morphological feature decomposition according to claim 1, characterized in that: Edge detail information is extracted using the BLF-LS method, which combines bilateral filters and least squares models. First, pixel saliency maps and structural saliency maps are calculated separately. Then, the fusion is completed under the dual guidance of the pixel domain and gradient domain by combining the total variation model to obtain the fusion result of the main structure information.

6. The SAR and optical image fusion method based on morphological feature decomposition according to claim 1, characterized in that: The morphological feature information fusion adopts a local adaptive weighted fusion strategy that combines structural similarity and image average gradient, specifically including: First, the image is locally segmented using a sliding window, and the structural similarity index SSIM and the average gradient index AG are calculated. Based on these, the local fusion weights are obtained to achieve local weighted fusion of morphological feature information. AG is used to calculate the average gradient of an image patch. The formulas for calculating the AG value of SAR and optical image patches are as follows: in, X Represents the input image patch. N Indicates the size of the image patch; and Indicates the image in and Partial derivatives in the direction, i Indicates the index of the image patch. Represents the calculation of SAR image patches AG value, Represents the computation of optical image blocks AG value; The structural similarity index and the average gradient index of the image are normalized to eliminate the dimensional differences between the different indices, and the local adaptive fusion weights are calculated based on the normalized indices; the calculation method of the local fusion weights is expressed as follows: in, This indicates normalization processing. and These represent the local weighting coefficients of the SAR image and the optical image, respectively. Indicators representing the structural similarity of SAR images Indicators representing the structural similarity of optical images; Finally, based on the aforementioned local adaptive fusion weights, the morphological feature information of the SAR image and the morphological feature information of the optical image are locally weighted and fused to obtain the fused morphological feature information, the expression of which is: in, This represents the morphological features after fusion. This represents the morphological features of SAR. It represents the morphological features of optical images.

7. The SAR and optical image fusion method based on morphological feature decomposition according to claim 1, characterized in that: Edge detail information fusion adopts a strategy that combines edge preservation enhancement and sparse detail compensation, including edge preservation enhancement based on sub-window standard deviation filter and detail compensation based on sparse representation; During edge-preserving enhancement, the statistical characteristics of pixel grayscale distribution within the sub-window are analyzed to achieve adaptive enhancement of edges, corners, and textured regions; for any pixel in the image... In its neighborhood window Internal calculation of gray standard deviation: in, Represents the neighborhood window The standard deviation of the grayscale of an inner pixel; the larger the standard deviation, the stronger the contrast and texture of that area. The value represents the average value of the pixels within the window, and N represents the neighborhood window. The number of inner pixels, Represents the neighborhood window Inner i The intensity value of each pixel; The neighborhood window is divided into multiple sub-windows, and the standard deviation of each sub-window is calculated to enhance structural sensitivity. Based on this, an edge preservation factor is defined to achieve local edge enhancement. Multi-scale response analysis of the image is performed based on the sub-window standard deviation filter (SSF) to obtain detailed features at different scales, resulting in an edge-preserving enhanced detail layer, the expression of which is: in, D Indicates edge detail information, Represents the SSF operator. and These represent fine-scale detail layers and small-scale detail layers, respectively. This represents the enhanced detail layer after multi-scale image processing. and These are the weight coefficients for the fine-scale detail layer and the small-scale detail layer, respectively. and These represent the enhanced detail layers for SAR and optical images, respectively. This indicates that the final edge retains the enhancement layer.

8. The SAR and optical image fusion method based on morphological feature decomposition according to claim 7, characterized in that: In the sparse detail compensation process, robust principal component analysis is used to decompose the edge detail information, and K-SVD dictionary learning and orthogonal matching pursuit are applied to the sparse residual terms for sparse reconstruction to recover the true high-frequency detail information; its expression is: in, and Indicates optical image blocks With SAR image patches OMP encoding is performed to obtain sparse coefficients; Represents the image block dictionary. This represents the sparsity coefficients after fusion. This represents the detail block of the final sparse reconstruction. It corresponds to the image block dictionary. Part of Represents the local mean of an image patch. x and y Represents the global pixel coordinates on the image. i Indicates the index number of the image patch. Indicates the first i Each reconstruction block in pixels Detailed location information; Indicates the first i The weight of each block at that pixel position. Representing coordinates Detail compensation layer at the location; Keep the edges reinforced. With detail compensation layer The fusion process yields a fused portion containing high-precision edge details and realistic texture information. .

9. The SAR and optical image fusion method based on morphological feature decomposition according to claim 1, characterized in that: Main structure information fusion results morphological feature fusion results Blending results with edge details Pixel-by-pixel weighted overlay is performed to obtain the final SAR-optical fusion image. F Its mathematical expression is as follows: in, This indicates the final fusion result. W M The weights of the morphological feature information components are represented. This indicates the result of morphological feature fusion. W D Indicates the weight of the edge detail information components. This indicates the result of edge detail blending.

10. A SAR and optical image fusion system based on morphological feature decomposition, characterized in that: It includes a processor and a memory, the memory being used to store program instructions, and the processor being used to call the program instructions in the memory to execute the SAR and optical image fusion method based on morphological feature decomposition as described in any one of claims 1-9.

Images