Brain MRI image fusion method and system based on cross-like bilateral filtering

Abstract

The invention relates to a brain MRI image fusion method and system based on cross-like bilateral filtering. The method comprises the following steps: obtaining brain MRI image data to be fused; performing selective filtering on the image data by using a cross-like bilateral filter, and decomposing to obtain an energy layer and a structural layer; fusing the energy layer and the structure layer according to a fusion strategy; arithmetic addition is carried out on the energy layer and the structure layer after fusion to obtain a final fusion image; wherein the fusion strategy comprises the following steps: guiding fusion of the energy layer by using a visual saliency detection graph VSDM, and obtaining a fusion image of the energy layer through weighted average; a multi-scale morphological gradient MLMG is combined with an improved Laplacian operator WSEML based on eight neighborhoods to guide fusion of the structural layer, and a fusion image of the structural layer is obtained through weighted average. Compared with the prior art, the image fusion method is rapid and effective, and meets the requirement of real-time application.

Description

Technical Field

[0001] The present invention relates to the field of image fusion, and in particular to a brain MRI image fusion method and system based on quasi-cross bilateral filtering. Background Art

[0002] Doctors can only obtain limited information about patients from single-modality medical images. Therefore, when making clinical diagnoses, they often need to refer to the patient's pathological information and combine the patient's clinical images of different modalities for diagnosis. Therefore, from the perspective of clinical application, medical image fusion is of great significance.

[0003] The image fusion process includes image decomposition, feature fusion and image reconstruction. The adequacy of image decomposition determines whether the feature information of the source image can be fully extracted. Feature fusion is another important aspect of image fusion. Choosing appropriate fusion methods for different decomposition layers is an important step for successful image fusion. With the deepening of image theory research such as multi-scale analysis, sparse representation, and deep learning, image fusion technology has also made great progress.

[0004] Typical multi-scale transform-based methods include pyramid-based algorithms, wavelet-based algorithms, non-subsampled shearlet-based algorithms, and non-subsampled contourlet-based algorithms. However, the Laplace pyramid transform method is sensitive to noise, has no directionality, has poor stability during reconstruction, and has redundancy between the layers of the pyramid; the gradient pyramid algorithm has strong noise resistance, good stability, and fast operation speed, but at the same time has the problem of loss of details and edges; the wavelet transform is prone to the problem of ring artifacts; the non-subsampled contourlet transform reduces the Gibbs phenomenon in the image and enhances the expression of image detail information, but the non-subsampled operation increases the redundancy in the transformed data, and the non-subsampled operation in the inverse transform process leads to high computational complexity and low computational efficiency, making it unable to meet the requirements of real-time applications; the non-subsampled shearlet transform has a faster computational speed, similar to the non-subsampled contourlet transform, which combines the non-subsampled pyramid transform with several different shearlet filters to provide multi-scale and multi-directional decomposition. Due to its small support area for directional filters, the non-subsampled pyramid reduces the occurrence of the Gibbs phenomenon and improves computational efficiency.

[0005] With the development of deep learning algorithms, convolutional neural networks (CNNs) have been applied to many computer vision studies. However, existing deep learning-based algorithms require considerable training time, large computational workload, high hardware cost, complex model design, and demanding data volume.

[0006] Traditional multi-scale decomposition methods are not suitable for image fusion when the source images are affected by noise. In this case, traditional multi-scale decomposition requires at least three scales to obtain satisfactory fusion, but the negative impact is that the time complexity increases and it cannot be applied to real-time image fusion scenarios. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a brain MRI image fusion method and system based on quasi-cross bilateral filtering, so as to realize the fusion of real-time images and improve the clarity of the contour edges of the fused images.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] A brain MRI image fusion method based on quasi-cross bilateral filtering comprises the following steps:

[0010] Acquire brain MRI image data to be fused;

[0011] Selectively filtering the image data using a quasi-crossed bilateral filter to decompose the image data into an energy layer and a structure layer;

[0012] The energy layer and the structure layer are respectively fused according to the fusion strategy;

[0013] The final fused image is obtained by arithmetically adding the fused energy layer and structure layer;

[0014] Among them, the fusion strategy includes: using the visual saliency detection map VSDM to guide the fusion of the energy layer, and obtaining the fusion image of the energy layer by weighted averaging; using the multi-scale morphological gradient MLMG combined with the improved Laplace operator WSEML based on eight neighborhoods to guide the fusion of the structure layer, and obtaining the fusion image of the structure layer by weighted averaging.

[0015] Furthermore, the formula of the cross-bilateral filter is as follows:

[0016]

[0017] In the formula, k p is the normalization constant, is the filter kernel of the spatial position, is the filter kernel of pixel size, I(x,y) is the pixel of the original image, I and I β are the pixels of images of different modalities, Ω p is the filter window applied to image I.

[0018] Furthermore, the normalization constant k p The calculation formula is as follows:

[0019]

[0020] In the formula, k p is the normalization constant, σ d is the standard deviation of the spatial position, σ r is the standard deviation of pixel size, I and I β are the pixels of images of different modalities, Ω p is the filter window applied to image I.

[0021] Furthermore, the filter kernel g of the spatial position σd The calculation formula is as follows:

[0022]

[0023] In the formula, σ d is the standard deviation of the spatial position.

[0024] Furthermore, the pixel-sized filter kernel g σr The calculation formula is as follows:

[0025]

[0026] In the formula, σ r is the standard deviation of pixel size.

[0027] Furthermore, the calculation formula of the visual saliency detection map VSDM is as follows:

[0028]

[0029] Where U k represents the intensity value of pixel k, U j The value range of U is [0-255]. j is the intensity value of pixel j, ||U k -U j || represents the pixel value difference, S j is the pixel value U j The frequency of occurrence in image U.

[0030] Furthermore, the calculation formula of the fusion image of the energy layer is as follows:

[0031]

[0032] Where, vsdm A is the VSDM of the input source image A, vsdm B is the VSDM of the input source image B, I EA (x, y) is the energy layer pixel of image A, I EB(x, y) is the energy layer pixel of image B;

[0033] The calculation formula of the fused image of the structural layer is as follows:

[0034] F S (x,y)=m′ A (x,y)×I SA (x,y)+m′ B (x,y)×I SB (x,y)

[0035] Where m′ A (x, y) is the structural layer information fusion rule of the source image A, m′ B (x, y) is the structural layer information fusion rule of the source image B, I SA (x, y) is the structural layer pixel of the source image A, I SB (x,y) is the structural layer pixel of the source image B.

[0036] Furthermore, a corrosion operation is added to the multi-scale morphological gradient MLMG.

[0037] Furthermore, the calculation formula of the improved Laplace operator WSEML based on eight neighborhoods is as follows:

[0038]

[0039] Where W is a matrix with a side length of (2r+1), r is the weight radius, and EML is based on the improved Laplace method proposed by Nayar;

[0040] The calculation formula of EML is as follows:

[0041]

[0042] Where P∈{A, B}, A and B are two source images.

[0043] According to another aspect of the present invention, a brain MRI image fusion system based on quasi-cross bilateral filtering is provided, comprising:

[0044] An image acquisition module, used for acquiring brain MRI image data to be fused;

[0045] An image filtering module, which selectively filters the image data using a quasi-cross bilateral filter to decompose the image data into an energy layer and a structure layer;

[0046] The image fusion module fuses the energy layer and the structure layer according to the fusion strategy and arithmetically adds the fused energy layer and structure layer to obtain the final fused image;

[0047] Among them, the fusion strategy includes: using the visual saliency detection map VSDM to guide the fusion of the energy layer, and obtaining the fusion image of the energy layer by weighted averaging; using the multi-scale morphological gradient MLMG combined with the improved Laplace operator WSEML based on eight neighborhoods to guide the fusion of the structure layer, and obtaining the fusion image of the structure layer by weighted averaging.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. The present invention introduces VSDM to guide the fusion of energy layers, which weakens the edge loss and contrast reduction caused by using weighted average technology to fuse energy layers; for the structure layer, MLMG and WSEML are used to guide the fusion of the structure layer, which improves the clarity of the contour edge of the fused image.

[0050] 2. The present invention uses a filtering method similar to cross-bilateral filtering, taking into account the relationship between the corresponding pixel values ​​of the image to be fused, and decomposes the image into an energy layer with pure intensity information and a structural layer with details. This method can fully extract the local and unique details of the image and the outline of the focus in the structural layer, and can be applied to real-time image fusion scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the process of the present invention;

[0052] Figure 2 is an example of MLMG transformation results, where (2a) is the original grayscale image, (2b) is the edge detection image obtained by the traditional MLMG method, and (2c) is the edge detection image obtained by the MLMG combined with WSEML method;

[0053] Figure 3 is the parameter selection of MLMG, where (3a) is the grayscale image, (3b) is the contrast image when k=1, (3c) is the contrast image when k=2, (3d) is the contrast image when k=3, (3e) is the contrast image when k=4, and (3f) is the contrast image when k=5;

[0054] Figure 4 is the result of image fusion, (4a), (4b), (4c), (4d), (4e) and (4f) are the results of fusion of 6 different groups of images, (a) and (g) of each group of images are the source images to be fused, (l) is the fusion result of the method proposed in the present invention, (b), (c), (d), (e), (f), (h), (i), (j) and (k) are the fusion results of nine other medical image fusion methods published in recent years. DETAILED DESCRIPTION

[0055] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0056] Abbreviations involved:

[0057] Visual Saliency Detection Map: VSDM

[0058] Multi-level morphology gradient: multi-level morphology gradient, MLMG

[0059] Weighted sum of eight neighborhood based modified Laplacian, WSEML

[0060] Example 1

[0061] This embodiment provides a brain MRI image fusion method based on cross-bilateral filtering. Figure 1 As shown, the following steps are included:

[0062] S1, obtaining brain MRI image data to be fused;

[0063] In this embodiment, 8 pairs of data are selected from the whole brain atlas. The images are strictly registered images, no preprocessing is required, and the spatial resolution is 256×256.

[0064] S2, selectively filtering the image data using a quasi-crossed bilateral filter to decompose the image data into an energy layer and a structure layer;

[0065] The formula of the bilateral filter was improved to obtain a formula of a quasi-cross bilateral filter. This formula innovatively introduced the Euclidean distance of pixels between the images to be fused as a weight. This scheme can highlight the complementarity between images of different modalities.

[0066] The formula of the original bilateral filter is as follows:

[0067]

[0068] In the formula, k p is the normalization constant, is the filter kernel of the spatial position, is the filter kernel of pixel size, I(x,y) is the pixel of the original image, I(i,j) represents the pixel with the coordinates of the pixel point (i,j), and I(i,j) is the neighboring pixel of I(x,y);

[0069] The formula of the cross-bilateral filter is as follows:

[0070]

[0071] In the formula, k p is the normalization constant, is the filter kernel of the spatial position, is the filter kernel of pixel size, I(x,y) is the pixel of the original image, I β (i,j) is the neighboring pixel of I(x,y), I E is the pixel of the image energy layer, and Ω is the filter window applied to image I.

[0072] Normalization constant k p The calculation formula is as follows:

[0073]

[0074] In the formula, k p is the normalization constant, σ d is the standard deviation of the spatial position, σ r is the standard deviation of pixel size, I and I β are the pixels of images of different modalities, and Ω is the filter window applied to image I.

[0075] The filter kernel g of the spatial position σd The calculation formula is as follows:

[0076]

[0077] In the formula, σ d is the standard deviation of the spatial position.

[0078] The pixel-sized filter kernel g σr The calculation formula is as follows:

[0079]

[0080] In the formula, σ r is the standard deviation of pixel size.

[0081] The energy layer I of the image can be obtained by using a cross-bilateral filtering method. E , and then get the structure layer I of the image S , the calculation formula is as follows:

[0082] I(x,y)=I S (x,y)+I E (x,y)

[0083] Where I(x,y) is the pixel of the original image, IS (x, y) is the structural layer pixel of the image, I E (x,y) is the energy layer pixel of the image.

[0084] S3, integrating the energy layer and the structure layer respectively according to the fusion strategy;

[0085] The visual saliency detection map VSDM is used to guide the fusion of energy layers, making full use of the advantages of edge contour extraction to improve the edge contour clarity of the fused image and fully retain the image detail information;

[0086] A unique weighted average technique based on visual weighted maps is used to fuse the energy layers, which partially compensates for the loss of contrast. The calculation formula is as follows:

[0087]

[0088] Where U k represents the intensity value of pixel k, U j The value range of U is [0-255]. j is the intensity value of pixel j, ||U k -U j || represents the pixel value difference, S j is the pixel value U j The frequency of occurrence in image U.

[0089] vsdm A and vsdm B The VSDM of the input source images A and B can be obtained by weighted averaging the fused image F of the energy layer EL. E , the calculation formula is as follows:

[0090]

[0091] In the formula, I EA (x, y) is the energy layer pixel of image A, I EB (x,y) is the energy layer pixel of image B.

[0092] If the energy layers of the two images have equal VSDM at certain locations, the fused image will be obtained by averaging the energy layers. A >vsdm B , then the fused image will contain EA For more information, when vsdm A <vsdm B When the fused image contains EB for more information.

[0093] For the fusion of structural layers, a structural layer SL fusion strategy combining multi-scale morphological gradient MLMG and improved Laplacian operator WSEML based on eight-neighborhood is proposed. The corrosion operation is added to the multi-scale morphological gradient algorithm, which can better extract edge contours and has good robustness.

[0094] In order to construct the MLMG function, we first construct the multi-scale elements, and the calculation formula is as follows:

[0095]

[0096] In the formula, SE1 represents the basic structural element, is an XOR operation, and k is the number of scales. This function can be applied to multi-scale scenarios by modifying the size of the structural element.

[0097] Next, we can use the morphological gradient operator commonly used in image processing to extract the gradient features of scale k from the image. The calculation formula is as follows:

[0098]

[0099] Where, SE k is a multi-scale element with a scale number of k, SE ε is a multi-scale element with a scale number of ε, is the XOR operation, and ⊙ is the XOR operation.

[0100] The final MLMG result is shown as follows:

[0101]

[0102] In the formula, ξ represents the weight of the kth slope, and its value is: k =2k+1.

[0103] MLMG has a better effect on edge detection, such as Figure 2 The original grayscale image used in the experiment is shown in Figure 2 As shown in (2a), the edge detection image obtained by the traditional MLMG method is as follows Figure 2 As shown in (2b), the edge detection image obtained by combining MLMG with WSEML is shown in Figure 2 As shown in (2c). Figure 2 It can be seen that the method of MLMG combined with WSEML has the best effect on edge detection.

[0104] In order to determine the best performance when selecting the ratio k, a comparative experiment was conducted, and the value of k was 1-5. The comparative results are shown in Figure 3 shown. Figure 3 (3a) is a grayscale image, Figure 3(3b) is the contrast image when k=1. Figure 3 (3c) is the contrast image when k=2. Figure 3 (3d) is the contrast image when k=3, Figure 3 (3e) is the contrast image when k=4. Figure 3 (3f) is the contrast image when k=5; it can be observed that when k is 4, the details and edges are well preserved.

[0105] In order to fully preserve the structure and detail information of SL, WSEML is used to extract the structure and details of SL. Compared with the traditional weighted local energy, this function has higher computational efficiency and stronger detail extraction ability, and is defined as follows:

[0106]

[0107] Where W is a matrix with a side length of (2r+1) and r is the weight radius.

[0108] Set the value of each element in W to 2r-d, where d is the distance from the adjacent element to the center. EML is based on the modified Laplace method proposed by Nayar, which adds directional derivatives of the diagonal lines and increases the weight coefficients. The calculation formula of EML is as follows:

[0109]

[0110] Where P∈{A, B}, A and B are two source images.

[0111] The final effective value of the structure layer information can be achieved by multiplying MLMG with WSEML as follows:

[0112]

[0113] The final information fusion rules at the structural layer are as follows:

[0114]

[0115] In order to further describe the structural features of the image in detail, m(x,y) is further refined into m′ i (x,y), as shown below:

[0116]

[0117] Where Ω is a local region centered at (x, y) of size T×T, and the values ​​of i are A and B.

[0118] The fused image F of the structure layer SL S It can be obtained by weighted average of the structure layers of image A and image B as follows:

[0119] F S (x,y)=m′ A (x,y)×I SA (x,y)+m′ B (x,y)×I SB (x,y)

[0120] In the formula, I SA (x, y) is the structural layer pixel of image A, I SB (x,y) is the structural layer pixel of image B.

[0121] By arithmetically adding the fused images of the energy layer EL and the structure layer SL, the final fused image is obtained as follows:

[0122] F(x,y)=F S (x,y)+F E (x,y)

[0123] In the formula, F S (x, y) is the fused image of the structure layer, F E (x,y) is the fused image of the energy layer.

[0124] In order to verify the feasibility of the fusion scheme, the proposed method is compared with nine other medical image fusion methods published in recent years, including: multi-level edge preserving filtering and pulse CNN-based methods, boundary measurement pulse coupled neural network fusion strategy and energy attribute fusion strategy-based method (Tan), parameter adaptive pulse CNN (PAPCN) method, joint bilateral filter and local gradient energy-based (JBF) method, convolutional neural network-based method, phase consistency and local Laplace energy-based method, adaptive co-occurrence filter-based (ACOF) method, pre-trained neural network-based (PTNN) method and Laplace resynthesis (LRD)-based method. By observing the results of image fusion, Figure 4 is the image fusion result. This embodiment provides fusion results of 6 groups of different images, such as Figure 4 As shown in (4a), (4b), (4c), (4d), (4e) and (4f), (a) and (g) of each set of images are source images to be fused, (l) is the fusion result of the method proposed in this embodiment, (b), (c), (d), (e), (f), (h), (i), (j) and (k) are the fusion results of nine other medical image fusion methods published in recent years. Figure 4 It can be seen that the method proposed in this embodiment achieves good fusion effects in extracting texture details, color fidelity and image contrast.

[0125] In order to further evaluate the fused image, objective evaluation indicators are used to provide a more convincing objective evaluation. The evaluation indicators include information entropy (EN), edge information retention (Q AB / F ), structural contrast (SCD), mean structural similarity index (MSSIM), feature mutual information (FMI), noise or artifact rate added to the fused image (N AB / F ), mutual information (MI), Pearson linear correlation coefficient (CC) and visual information fidelity fusion (VIFF). Only the information entropy is obtained from the fused image, and the values ​​of other indicators are related to both the original image and the fused image, so the obtained objective evaluation indicators are more convincing. Table 1 shows Figure 4 Quantitative evaluation results of the fused images in , different evaluation indicators are used to evaluate the fused images obtained by all fusion methods, and the maximum value is bolded; in addition, in order to verify the advantage of this method in fusion speed, the running time of each experimental object is calculated, and the specific values ​​are shown in Table 1. Table 1 is shown below.

[0126] Table 1 Quantitative evaluation results of fused images

[0127]

[0128] From the data in Table 1, it can be observed that the fusion method proposed in this embodiment has achieved the best results for most indicators, which is also consistent with the subjective visual evaluation results of the fusion results. By comparison, it is obvious that the fusion method proposed in this embodiment has achieved the best performance in terms of running speed. With the increase of experimental computer configuration, the fusion method proposed in this embodiment will have the function of real-time fusion.

[0129] Example 2

[0130] This embodiment provides a brain MRI image fusion system based on quasi-cross bilateral filtering, including:

[0131] An image acquisition module, used for acquiring brain MRI image data to be fused;

[0132] An image filtering module, which selectively filters the image data using a quasi-cross bilateral filter to decompose the image data into an energy layer and a structure layer;

[0133] The image fusion module fuses the energy layer and the structure layer according to the fusion strategy and arithmetically adds the fused energy layer and structure layer to obtain the final fused image;

[0134] Among them, the fusion strategy includes: using the visual saliency detection map VSDM to guide the fusion of the energy layer, and obtaining the fusion image of the energy layer by weighted averaging; using the multi-scale morphological gradient MLMG combined with the improved Laplace operator WSEML based on eight neighborhoods to guide the fusion of the structure layer, and obtaining the fusion image of the structure layer by weighted averaging.

[0135] The rest is the same as in Example 1.

[0136] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A brain MRI image fusion method based on quasi-cross bilateral filtering, characterized in that: The following steps are involved: Acquire brain MRI image data to be fused; Selectively filtering the image data using a quasi-crossed bilateral filter to decompose the image data into an energy layer and a structure layer; The energy layer and the structure layer are respectively fused according to the fusion strategy; The fused energy layer and structure layer are arithmetically added to obtain the final fused image; Among them, the fusion strategy includes: using the visual saliency detection map VSDM to guide the fusion of the energy layer, and obtaining the fusion image of the energy layer by weighted averaging; using the multi-scale morphological gradient MLMG combined with the improved Laplace operator WSEML based on eight neighborhoods to guide the fusion of the structure layer, and obtaining the fusion image of the structure layer by weighted averaging.

2. The brain MRI image fusion method based on quasi-crossed bilateral filtering according to claim 1, characterized in that: The formula of the cross-bilateral filter is as follows: In the formula, k p is the normalization constant, is the filter kernel of the spatial position, is the filter kernel of pixel size, I(x,y) is the pixel of the original image, I β (i,j) is the neighboring pixel of I(x,y), I E is the pixel of the image energy layer, and Ω is the filter window applied to image I.

3. The brain MRI image fusion method based on quasi-crossed bilateral filtering according to claim 2, characterized in that: The normalization constant k p The calculation formula is as follows: In the formula, k p is the normalization constant, σ d is the standard deviation of the spatial position, σ r is the standard deviation of pixel size, I and I β are the pixels of images of different modalities, and Ω is the filter window applied to image I.

4. The brain MRI image fusion method based on quasi-crossed bilateral filtering according to claim 2, characterized in that: The filter kernel at the spatial position The calculation formula is as follows: In the formula, σ d is the standard deviation of the spatial position.

5. The brain MRI image fusion method based on quasi-crossed bilateral filtering according to claim 2, characterized in that: The filter kernel of the pixel size The calculation formula is as follows: In the formula, σ r is the standard deviation of pixel size.

6. The brain MRI image fusion method based on quasi-crossed bilateral filtering according to claim 1, characterized in that: The calculation formula of the visual saliency detection map VSDM is as follows: Where U k represents the intensity value of pixel k, U j The value range of U is [0-255]. j is the intensity value of pixel j, ||U k -U j || represents the pixel value difference, S j is the pixel value U j The frequency of occurrence in image U.

7. The brain MRI image fusion method based on quasi-crossed bilateral filtering according to claim 1, characterized in that: The calculation formula of the fusion image of the energy layer is as follows: Where, vsdm A is the VSDM of the input source image A, vsdm B is the VSDM of the input source image B, I EA (x, y) is the energy layer pixel of image A, I EB (x, y) is the energy layer pixel of image B; The calculation formula of the fused image of the structural layer is as follows: F S (x,y)=m′ A (x,y)×I SA (x,y)+m′ B (x,y)×I SB (x,y) In the formula, m′ A (x, y) is the structural layer information fusion rule of the source image A, m′ B (x, y) is the structural layer information fusion rule of the source image B, I SA (x, y) is the structural layer pixel of the source image A, I SB (x, y) is the structural layer pixel of the source image B.

8. The brain MRI image fusion method based on quasi-crossed bilateral filtering according to claim 1, characterized in that: A corrosion operation is added to the multi-scale morphological gradient MLMG.

9. The brain MRI image fusion method based on quasi-crossed bilateral filtering according to claim 1, characterized in that: The calculation formula of the improved Laplace operator WSEML based on eight neighborhoods is as follows: Where W is a matrix with a side length of (2r+1), r is the weight radius, and EML is based on the improved Laplace method proposed by Nayar; The calculation formula of EML is as follows: Where P∈{A, B}, A and B are two source images.

10. A brain MRI image fusion system based on quasi-cross bilateral filtering, characterized in that: include: An image acquisition module, used for acquiring brain MRI image data to be fused; An image filtering module, which selectively filters the image data using a quasi-cross bilateral filter to decompose the image data into an energy layer and a structure layer; The image fusion module fuses the energy layer and the structure layer according to the fusion strategy and arithmetically adds the fused energy layer and structure layer to obtain the final fused image; Among them, the fusion strategy includes: using the visual saliency detection map VSDM to guide the fusion of the energy layer, and obtaining the fusion image of the energy layer by weighted averaging; using the multi-scale morphological gradient MLMG combined with the improved Laplace operator WSEML based on eight neighborhoods to guide the fusion of the structure layer, and obtaining the fusion image of the structure layer by weighted averaging.

Images