Multi-source image fusion method based on edge window filtering

Abstract

The present invention discloses a multi-source image fusion method based on edge window filtering. The present invention utilizes edge window filtering, a saliency detection method combining rarity color statistics with gradient energy optimization, and a multi-level fusion strategy to fuse infrared and visible light images. The specific method includes: firstly performing multi-scale decomposition on the infrared and visible light images to be fused using edge window filtering to generate base layer and detail layer images; performing saliency detection on the infrared and visible light images respectively using a saliency detection method combining rarity color statistics with gradient energy optimization to obtain a saliency map; then, obtaining a fusion weight map through saliency comparison and filtering optimization; secondly, using fusion weights to guide the fusion of base layer images to obtain a fused base layer; at the same time, adopting a multi-level fusion strategy to fuse the detail layer; finally, reconstructing the fused base layer and the fused detail layer to obtain a fused image.

Description

Technical Field

[0001] The present invention belongs to the field of multi-source image processing technology, and in particular relates to a multi-source image fusion method based on edge window filtering. Background Art

[0002] Due to limitations in sensor imaging principles and technological advancements, the information collected by a single sensor is always limited, making it difficult to meet the demand for more comprehensive scene information in specific application contexts. Therefore, how to integrate images collected by various sensors to generate an image that is rich in information and more conducive to human perception has gradually become a research hotspot.

[0003] When using different sensors to perform joint imaging of the same scene, the imaging can be mainly divided into two forms according to the type and number of imaging detectors: (1) joint imaging of sensors based on different imaging principles, and (2) imaging integration of the same sensor with different parameters.

[0004] In multi-sensor imaging, combining infrared and visible light offers numerous advantages. Infrared detectors, which receive external thermal radiation, offer advantages such as clear imaging targets, strong anti-interference capabilities, and 24 / 7 operation. However, their imaging resolution is generally low, unable to accurately depict detailed texture information in a scene, and images can only be displayed in grayscale. Visible light sensors, which receive reflected light, offer advantages such as high spatial resolution, clear texture details, and a better fit for human visual perception. However, they are more sensitive to lighting conditions and are easily affected by environmental factors such as inclement weather. Therefore, in these situations, it is necessary to complement the advantageous information in infrared and visible light images to obtain a complete picture of the scene.

[0005] Research on multi-source image fusion technology dates back to the 1980s, when Daily et al. first applied it to remote sensing. Since then, multi-source image fusion technology has continued to develop, yielding numerous excellent algorithms. Currently, the vast majority of fusion algorithms can be categorized as follows: multi-scale decomposition-based methods, sparse representation-based methods, subspace-based methods, saliency-based methods, and neural network-based methods. However, current algorithms suffer from limitations such as noise interference and information loss, leaving significant room for improvement. Summary of the Invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a multi-source image fusion method based on edge window filtering, in order to solve the problem of information loss at different scales in the separation process existing in the prior art, better retain image information, and maintain the contrast of the target and the coherence of the salient area, thereby significantly improving the contrast and clarity of the target in the fused image.

[0007] To achieve the above object, the present invention includes the following main steps:

[0008] A multi-source image fusion method based on edge window filtering includes the following steps:

[0009] Step (1): input the multi-source image to be fused, wherein the multi-source image is an infrared image I taken at the same scene. IR and visible light image I Vis , I IR and I Vis are equal in size;

[0010] Step (2): the infrared image I IR and visible light image I Vis Perform edge window filtering to obtain the infrared base layer image B IR and the visible light base layer image B Vis ;

[0011] Step (3) uses the saliency detection method of rare color statistics combined with gradient energy optimization to detect the infrared image I IR and visible light image I Vis Perform saliency detection and generate infrared saliency map U IR and visible light saliency map U Vis ;

[0012] Step (4): for the infrared saliency map U IR and visible light saliency map U Vis Perform significance comparison to obtain the initial fusion weight of the base layer and optimize to obtain the final fusion weight and

[0013] Step (5), use and Guiding infrared base layer image B IR and the visible light base layer image B Vis Fusion, generating a fused base layer image FuB;

[0014] Step (6) uses the hybrid fusion strategy of image gradient feature (GC) and intensity variance (IV) to fuse the infrared detail layer and the visible light detail layer to obtain the fused detail layer image FuD i ;

[0015] Step (7): fusion base layer image FuB and fusion detail layer image FuD i The fused image FuI is obtained by adding them together.

[0016] The edge window filtering uses the kernel function F to perform multi-scale decomposition on the input image I, including the following steps:

[0017] Step 1: Select a pixel point arbitrarily from the input image I;

[0018] Step 2: Get eight side windows from the downward (D), right (R), upward (U), left (L), southwest (SW), southeast (SE), northeast (NE), and northwest (NW) sides of the selected pixel.

[0019] Step 3: Calculate the edge window filter (SWF) output of any pixel in the input image according to the following formula:

[0020]

[0021]

[0022] Where m represents one of the side windows, S = {L, R, U, D, NW, NE, SW, SE} is the set of side windows, and I m is the side window output obtained by applying the kernel function F in the side window m, N m is the weighted sum based on the side window set, is the side window in eight directions of the lth target pixel based on the kernel function F, q j is the intensity of the input image I at the lth target pixel, w lj is the weight of the jth pixel near the lth target pixel based on the kernel function F, I t is with q j The side window output with the smallest L2 distance is the final side window filter output;

[0023] Step 4: After all pixels of the input image I are selected, the f-th filtering result I is obtained. f , f=1,2,...,n,I 0 =I, will I f and the f-1th filtering result I f-1 Subtract to get the detail layer image D of the current level f , denoted as D f =I f-1 -I f , the final filtering result is the base layer image B, where n is the decomposition level;

[0024] Among them, the input image I is taken as the infrared image I IR and visible light image I Vis , get the infrared base layer image B IR and the visible light base layer image B Vis .

[0025] Compared with the existing technology, the present invention has the following advantages:

[0026] First, the present invention performs edge window filtering on all pixel values ​​of the infrared image and the visible light image to complete multi-scale decomposition, thereby overcoming the deficiency of the existing technology in that information of different scales is lost during the separation process, and making the present invention have better image information retention.

[0027] Second, the present invention proposes a composite saliency detection method that combines rarity color statistics with gradient energy optimization. This method is applied to perform saliency detection on infrared and visible light images to better maintain the contrast of the target and the continuity of the salient areas, enabling the present invention to significantly improve the contrast and clarity of the target in the fused image. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the invention process.

[0029] Figure 2 This is a flow chart of multi-scale decomposition based on edge window filtering in an example of the present invention.

[0030] Figure 3 This is the definition diagram of the side window in the side window filtering.

[0031] Figure 4 This is the fusion weight graph construction process of the composite saliency detection method based on rarity color statistics combined with gradient energy optimization (RCSAGE) in the example of the present invention.

[0032] Figure 5 The following are the results of infrared and visible light image saliency detection in the present invention. (a) is IR, (b) is VI, (c) is IR saliency map, and (d) is VI saliency map.

[0033] Figure 6 Figure 1 shows the weight generation process for infrared and visible light images in this example. (a) is the initial IR weight map, (b) is the initial VI weight map, (c) is the optimized IR weight map, and (d) is the optimized VI weight map.

[0034] Figure 7 The figure shows the comparison results of the fusion effects of nine different fusion algorithms on the "camp" sequence; among them, (a) is an infrared image; (b) is a visible light image; (c) is BF; (d) is GFF; (e) is CBF; (f) is NSCT; (g) is MST-SR; (h) is GTF; (i) is MF; (j) is QDBI; (k) is FMSPD; and (l) is the method of the present invention.

[0035] Figure 8The figure shows the comparison of the fusion effects of nine different fusion algorithms on the "Kaptein" sequence; (a) is an infrared image; (b) is a visible light image; (c) is BF; (d) is GFF; (e) is CBF; (f) is NSCT; (g) is MST-SR; (h) is GTF; (i) is MF; (j) is QDBI; (k) is FMSPD; and (l) is the method of the present invention.

[0036] Figure 9 The figure shows the comparison results of the fusion effects of nine different fusion algorithms on the "Marne" sequence; among them, (a) is an infrared image; (b) is a visible light image; (c) is BF; (d) is GFF; (e) is CBF; (f) is NSCT; (g) is MST-SR; (h) is GTF; (i) is MF; (j) is QDBI; (k) is FMSPD; and (l) is the method of the present invention.

[0037] Figure 10 The figure shows the comparison results of the fusion effects of nine different fusion algorithms on the "Tank" sequence; among them, (a) is an infrared image; (b) is a visible light image; (c) is BF; (d) is GFF; (e) is CBF; (f) is NSCT; (g) is MST-SR; (h) is GTF; (i) is MF; (j) is QDBI; (k) is FMSPD; and (l) is the method of the present invention.

[0038] Figure 11 The figure shows the comparison results of the fusion effects of nine different fusion algorithms on the "Road" sequence; among them, (a) is an infrared image; (b) is a visible light image; (c) is BF; (d) is GFF; (e) is CBF; (f) is NSCT; (g) is MST-SR; (h) is GTF; (i) is MF; (j) is QDBI; (k) is FMSPD; and (l) is the method of the present invention.

[0039] Figure 12 The figure shows the comparison results of the fusion effects of nine different fusion algorithms on the "Kayak" sequence; among them, (a) is an infrared image; (b) is a visible light image; (c) is BF; (d) is GFF; (e) is CBF; (f) is NSCT; (g) is MST-SR; (h) is GTF; (i) is MF; (j) is QDBI; (k) is FMSPD; and (l) is the method of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] As mentioned above, existing multi-source image fusion algorithms generally have defects such as noise interference and information loss, and still need further improvement. Based on this, the present invention proposes a multi-source image fusion method based on edge window filtering. Based on infrared and visible light images taken in the same scene, the infrared and visible light images to be fused are first subjected to multi-scale decomposition using edge window filtering to generate base layer and detail layer images. A saliency detection method combining rarity color statistics with gradient energy optimization is used to generate saliency maps by performing saliency detection (SF) on the infrared and visible light images respectively. Then, a fusion weight map is obtained through saliency comparison and filtering optimization. Secondly, the fusion weights are used to guide the fusion of the base layer images to obtain a fused base layer. At the same time, a multi-level fusion strategy is adopted to fuse the detail layer. Finally, the fused base layer and the fused detail layer are reconstructed to obtain the fused image.

[0042] To demonstrate the effectiveness of the proposed method, six sets of infrared and visible light images under different environments were selected from the public TNO dataset to conduct qualitative and quantitative analysis of the proposed fusion algorithm. The experiment used six different scene images of "camp", "Kaptein", "Marne", "Tank", "Road" and "Kayak" in the TNO image fusion dataset. Figure 1 ,The basic process of image fusion for TNO image data is as follows:

[0043] (1) Input the multi-source images to be fused.

[0044] Input an infrared image to be fused I IR and a visible light image I to be fused Vis , where I IR and I Vis Taken at the same scene, both are equal in size.

[0045] (2) For infrared image I IR Perform edge window filtering to obtain the infrared base layer image B IR .

[0046] (2a) The present invention uses a box filter as the kernel function F of the edge window filter to perform multi-scale decomposition on the input image I to enhance the edge preservation capability and reduce the algorithm complexity, such as Figure 2 shown.

[0047] (2b) Take infrared image I IR For the input image I, select a pixel from it at random.

[0048] (2c) Eight side windows are obtained from the downward (D), rightward (R), upward (U), leftward (L), southwest (SW), southeast (SE), northeast (NE) and northwest (NW) sides of the selected pixel. The structural definition of the side window in the side window filtering can be referred to Figure 3 As shown in (a), the side windows are defined in the continuous case with parameters θ and r, where θ is the angle of the window relative to the horizontal line and r is the radius of the filter window, which is manually defined and fixed for all windows. ρ∈{o,r}, (x,y) is the position of the target pixel. By keeping the (x,y) position unchanged and changing θ, the direction of the window can be adjusted so that its side remains aligned with pixel l. To simplify the operation, eight side windows are defined in the discrete case, as follows Figure 3 As shown in (b)-(d), they correspond to θ = k × π / 2, k∈[0,3].

[0049] (2d) Apply the kernel function F to the eight side windows respectively, and obtain eight different outputs in turn. Select the side window output with the smallest L2 distance to the input intensity as the final output to complete the side window filtering.

[0050] Specifically, the edge window filter (SWF) output of any pixel of the input image is calculated according to the following formula:

[0051]

[0052]

[0053] Where m represents one of the side windows, S = {L, R, U, D, NW, NE, SW, SE} is the set of side windows, and I m is the side window output obtained by applying the kernel function F in the side window m, N m is the weighted sum based on the side window set, is the side window in eight directions of the lth target pixel (i.e., the pixel selected in step (2b)) based on the kernel function F, q j is the intensity of the input image I at the lth target pixel, w lj is the weight of the jth pixel near the lth target pixel based on the kernel function F, I t is with q j The side window output with the smallest L2 distance is the final side window filtering output.

[0054] (2e) Determine the infrared image I IR Are all the pixels selected? If so, the f-th filtering result is obtained. f=1,2,...,n,in the initial state Will and the f-1th filtering result Subtract and get the infrared detail layer image of the current level Expressed as: The final filtering result is the infrared base layer image B IR , continue to step (3); otherwise, execute step (2b); where f is the current filtering number, n is the decomposition level, and in this embodiment, it is 3.

[0055] (3) For visible light image I Vis Perform edge window filtering to obtain the visible light base layer image B Vis .

[0056] (3a) Same as step (2a), box filter is also used as the kernel function F of edge window filtering, as shown in Figure 3 shown.

[0057] (3b) Take the visible light image I Vis For the input image I, select a pixel from it at random.

[0058] (3c) Same as step (2c), eight side windows are obtained from the downward (D), rightward (R), upward (U), leftward (L), southwest (SW), southeast (SE), northeast (NE), and northwest (NW) sides of the selected pixel point.

[0059] (3d) Similar to step (2d), apply the kernel function F to each of the eight side windows, obtaining eight different outputs. Select the side window output with the smallest L2 distance to the input intensity as the final output, completing the side window filtering. The side window filter (SWF) output for any pixel in the image is calculated similarly to step (2d).

[0060] (3e) Determine the visible light image I Vis Are all the pixels selected? If so, the f-th filtering result is obtained. f=1,2,...,n,in the initial state Will Compared with the last filtering result Subtract to get the visible light detail layer image of the current level The final filtering result is the visible light base layer image B Vis , continue to step (4), otherwise, execute step (3b); where f is the current filtering order, n is the decomposition level, and in this algorithm it is 3. It is easy to understand that the value of f in step (2e) and step (3e) is the same.

[0061] (4) The composite saliency detection method using rarity color statistics combined with gradient energy optimization is used to detect the infrared image I IR and visible light image I Vis Perform saliency detection and obtain infrared saliency map SIR and visible light saliency map S Vis ,refer to Figure 4 , the specific steps are as follows:

[0062] (4a) Calculate the value of the specified pixel I in the input image I according to the following formula: k The significance value of S(I k ):

[0063]

[0064] Among them, I k ∈[0,255], I is the input image, I o is any pixel in the input image, ||·|| represents the distance between color values, which can be converted to

[0065] S(I k )=||I k -I1||+||I k -I2||+…+||I k -I N ||

[0066] Where N is the total number of pixels in the input image, I1~I N Represents the first to Nth pixels. Given an input image, any pixel I o The values ​​of are known.

[0067] (4b) Calculate the specified pixel I according to the following formula k Rarity color statistics T(I k ):

[0068]

[0069] Among them, N(I k ) is the specified pixel I k The rarity of , σ represents the minimum value.

[0070] (4c) Calculate the gradient energy GE(x,y) of the input image according to the following formula:

[0071]

[0072] Where ω is the neighborhood of pixel (x, y), the window radius is set to 3 in the present invention, I h ,I v ,I b Represent the gradient features on the horizontal, vertical and diagonal lines respectively.

[0073] (4d) Calculate the composite saliency detection result of the input image by combining rarity color statistics with gradient energy optimization according to the following formula:

[0074] U=T(I k )*GE(x,y)

[0075] For infrared image I IR ,have:

[0076] U IR =T(I IR )*GE IR

[0077] For the visible light image I Vis ,have:

[0078] U Vis =T(I Vis )*GE Vis

[0079] Among them, U IR Represents the saliency detection result of the infrared image, that is, the infrared saliency map U IR , U Vis Represents the saliency detection result of the visible light image, that is, the visible light saliency map U Vis .

[0080] (5) For infrared saliency map U IR and visible light saliency map U Vis Perform significance comparison to obtain the initial fusion weight of the base layer and optimize to obtain the final fusion weight and The specific steps are as follows:

[0081] (5a) For infrared saliency map U IR and visible light saliency map U Vis The initial fusion weight is obtained by performing a saliency comparison. Specifically, the initial fusion weight O of the infrared image base layer and the visible light image base layer at (x, y) is calculated according to the following formula: IR (x,y) and O Vis (x,y):

[0082]

[0083]

[0084] Among them, U IR (x,y) and U Vis (x, y) represents the saliency value of the infrared image and the visible light image at (x, y) respectively; IR (x,y) constitutes the initial fusion weight O of the infrared image base layer IR , each O Vis (x,y) constitutes the initial fusion weight O of the base layer of the visible light image Vis .

[0085] (5b) The initial weight map usually has noise and may not be completely aligned with the source image boundary, resulting in a decrease in the quality of the fused image. Therefore, the present invention uses anisotropic guided filtering (AGF) to optimize the initial fusion weights to obtain the optimized fusion weights W for infrared and visible light images. IR and W Vis , which can be expressed as:

[0086] W IR =AGF(O IR )

[0087] W Vis =AGF(O Vis )

[0088] (5c) W IR and W Vis Normalize to get the final fusion weight and

[0089] (6) Use the final fusion weight of step (5) and Guiding base layer image B IR and B Vis Fusion, according to the following formula, calculate the fused base layer image FuB:

[0090]

[0091] (7) Generate fused detail layer image FuD i .

[0092] This paper uses a hybrid fusion strategy of image gradient characteristics (GC) and intensity variance (IV) to fuse the infrared detail layer and the visible light detail layer. The gradient characteristics (GC) reflect the contrast of tiny details in the image. The stronger the gradient characteristics, the more prominent the edge features of the image. The intensity variance (IV) reflects the local energy of the image. The greater the local energy, the clearer the corresponding area of ​​the image.

[0093] The composite fusion strategy for the detail layer can be expressed as the following formula to calculate the value of each element in the fused detail layer:

[0094]

[0095] in, Represents the information at the i-th detail layer (x, y) in the infrared detail layer, Represents the information at the i-th detail layer (x, y) in the visible light detail layer, and Represent the gradient features of the i-th detail layer in the infrared detail layer and the visible light detail layer, and Denotes the intensity variance of the i-th detail layer in the infrared detail layer and the visible light detail layer, FuD i (x,y) represents the fusion result of the i-th detail layer (x,y) of the infrared detail layer and the i-th detail layer (x,y) of the visible light detail layer.

[0096] (7c) All FuD i (x,y) constitutes the fused detail layer image FuD i .

[0097] (8) Fusion base layer image FuB and fusion detail layer image FuD i The fused image FuI is obtained by adding them together. The calculation formula is as follows:

[0098]

[0099] Subjective evaluation is popular and direct in fusion quality evaluation, but it is easily affected by human factors. At the same time, when the difference between fused images is small, it is difficult to make an accurate judgment only through subjective evaluation. Therefore, the present invention introduces objective evaluation indicators for objective evaluation on the basis of subjective evaluation. The objective evaluation indicators used in the present invention are: information entropy (EN), average gradient (AG), structural similarity (SSIM), image fidelity (VIF), edge preservation factor (Q AB / F ). Among them, EN represents the amount of information contained in the fused image, AG is the image clarity measure, SSIM and VIF represent the degree of information retention of the fused image to the source image, and Q AB / F It represents the degree of preservation of edge information in the fused image.

[0100] For all five evaluation metrics used in this paper, larger values ​​indicate better fusion results. Each row represents the calculated values ​​of the same evaluation metric based on the fusion results corresponding to different algorithms. The method with the best performance under the same evaluation metric is shown in bold.

[0101] The following table shows the average fusion results of each method for six groups of image sequences. It can be seen that in the comparative experiments with various advanced algorithms in different scenarios, the numerical results of the algorithm proposed in this invention are better under most evaluation criteria, that is, the fusion effect is better.

[0102]

[0103]

[0104] refer to Figure 5 and Figure 6, showing the results of saliency detection for infrared and visible light images and the weight generation process. Subjective evaluation methods assess the quality of fused images based on the human visual system. The visual quality can indicate the quality of the fusion result. This paper selected six sets of multi-focus source images from a public dataset for comparative experiments and conducted a qualitative analysis of the experimental results.

[0105] The ideal infrared and visible light fusion image contains both the significant target information of the infrared image and the clear texture details of the visible light image. The comparison results of the method of the present invention and the nine selected comparison algorithms in the six scenes of "Camp", "Kaptein", "Marne", "Tank", "Road" and "Kayak" are as follows: Figure 7-12 The above source images have all been strictly registered, and the defects in the comparison algorithm results have been marked with red frames at the corresponding positions.

[0106] Among them, Figures 7 to 12 In the figure, (a) is the infrared image, (b) is the visible light image, (c) is the fusion result of the BF algorithm, (d) is the fusion result of the GFF algorithm, (e) is the fusion result of the CBF algorithm, (f) is the fusion result of the NSCT algorithm, (g) is the fusion result of the MST-SR algorithm, (h) is the fusion result of the GTF algorithm, (i) is the fusion result of the MF algorithm, (j) is the fusion result of the QDBI algorithm, (k) is the fusion result of the FMSPD algorithm, and (l) is the fusion result of the algorithm proposed in this invention.

[0107] In summary, in comparative experiments with various advanced algorithms in different scenarios, the algorithm proposed in this invention has the best comprehensive performance in both subjective and objective evaluations, indicating that the method of this invention has excellent algorithm performance and wide applicability.

Claims

1. A multi-source image fusion method based on edge window filtering, characterized in that: The steps include: Step (1): input the multi-source image to be fused, wherein the multi-source image is an infrared image I taken at the same scene. IR and visible light image I Vis , I IR and I Vis are equal in size; Step (2): the infrared image I IR and visible light image I Vis Perform edge window filtering to obtain the infrared base layer image B IR and the visible light base layer image B Vis ; Step (3) uses the saliency detection method of rare color statistics combined with gradient energy optimization to detect the infrared image I IR and visible light image I Vis Perform saliency detection and generate infrared saliency map U IR and visible light saliency map U Vis ; The specific steps are as follows: Step 1: Calculate the specified pixel I in the input image I according to the following formula k The significance value of S(I k ): S(I k )=||I k -I1||+||I k -I2||+…+||I k -I N || Where I is the input image, I k ∈[0,255], ||·|| represents the distance between color values, N is the total number of pixels in the input image, I1~I N represents the first to Nth pixels; Step 2: Calculate the specified pixel I according to the following formula: k Rarity color statistics T(I k ): Among them, N(I k ) is the pixel I specified in step 1 k The rarity of , σ represents the minimum value; Step 3: Calculate the gradient energy GE(x,y) of the input image according to the following formula: Where ω is the neighborhood of pixel (x, y), the window radius is set to 3 in the present invention, I h ,I v ,I b Represents the gradient features on the horizontal, vertical and diagonal lines respectively; Step 4: Calculate the composite saliency detection result of the input image’s rarity color statistics combined with gradient energy optimization according to the following formula: U=T(I k )*GE(x,y); Take the input image I as the infrared image I IR and visible light image I Vis , get the infrared saliency map U IR and visible light saliency map U Vis ; Step (4): for the infrared saliency map U IR and visible light saliency map U Vis Perform significance comparison to obtain the initial fusion weight of the base layer and optimize to obtain the final fusion weight and Step (5), use and Guiding infrared base layer image B IR and the visible light base layer image B Vis Fusion, generating a fused base layer image FuB; Step (6) uses the hybrid fusion strategy of image gradient feature (GC) and intensity variance (IV) to fuse the infrared detail layer and the visible light detail layer to obtain the fused detail layer image FuD i ; Step (7): fusion base layer image FuB and fusion detail layer image FuD i The fused image FuI is obtained by adding them together.

2. The multi-source image fusion method based on edge window filtering according to claim 1, characterized in that: The edge window filtering comprises the following steps: Step 1: Select a pixel point arbitrarily from the input image I; Step 2: Get eight side windows from the downward (D), right (R), upward (U), left (L), southwest (SW), southeast (SE), northeast (NE), and northwest (NW) sides of the selected pixel. Step 3: Calculate the edge window filter (SWF) output of any pixel in the input image according to the following formula: Where m represents one of the side windows, S = {L, R, U, D, NW, NE, SW, SE} is the set of side windows, and I m is the side window output obtained by applying the kernel function F in the side window m, N m is the weighted sum based on the side window set, is the side window in eight directions of the lth target pixel based on the kernel function F, q j is the intensity of the input image I at the lth target pixel, w lj is the weight of the jth pixel near the lth target pixel based on the kernel function F, I t is with q j The side window output with the smallest L2 distance is the final side window filter output; Step 4: After all pixels of the input image I are selected, the f-th filtering result I is obtained. f , f=1,2,...,n,I 0 =I, will I f and the f-1th filtering result I f-1 Subtract to get the detail layer image D of the current level f , denoted as D f =I f-1 -I f , the final filtering result is the base layer image B, where n is the decomposition level; Among them, the input image I is taken as the infrared image I IR and visible light image I Vis , get the infrared base layer image B IR and the visible light base layer image B Vis .

3. The multi-source image fusion method based on edge window filtering according to claim 2, characterized in that: The kernel function F of the edge window filter is a box filter.

4. The multi-source image fusion method based on edge window filtering according to claim 1, characterized in that: The specific steps of step (4) are as follows: Step (4a), for the infrared saliency map U IR and visible light saliency map U Vis Perform significance comparison to obtain the initial fusion weight; Step (4b): Use anisotropic guided filtering (AGF) to optimize the initial fusion weights to obtain the optimized fusion weights W of infrared and visible light images. IR and W Vis ; Step (4c), W IR and W Vis Normalize to get the final fusion weight and 5. The multi-source image fusion method based on edge window filtering according to claim 4, characterized in that: In step (4a), the initial fusion weight O of the infrared image base layer and the visible light image base layer at (x, y) is calculated according to the following formula: IR (x,y) and O Vis (x,y): Among them, U IR (x,y) and U Vis (x, y) represents the saliency value of the infrared image and the visible light image at (x, y) respectively; IR (x,y) constitutes the initial fusion weight O of the infrared image base layer IR , each O Vis (x,y) constitutes the initial fusion weight O of the base layer of the visible light image Vis ; In step (4b), the initial fusion weight is optimized using anisotropic guided filtering (AGF), which is expressed as W IR =AGF(O IR ) W Vis =AGF(O Vis )。 6. The multi-source image fusion method based on edge window filtering according to claim 1 or 4, characterized in that: In step (5), the calculation formula of the fused base layer image FuB is as follows:

7. The multi-source image fusion method based on edge window filtering according to claim 1, characterized in that: In step (6), the value of each element in the fused detail layer is calculated according to the following formula: in, Represents the information at the i-th detail layer (x, y) in the infrared detail layer, Represents the information at the i-th detail layer (x, y) in the visible light detail layer, and Represent the gradient features of the i-th detail layer in the infrared detail layer and the visible light detail layer, and Denotes the intensity variance of the i-th detail layer in the infrared detail layer and the visible light detail layer, FuD i (x, y) represents the fusion result of the ith detail layer (x, y) of the infrared detail layer and the ith detail layer (x, y) of the visible light detail layer. All FuD i (x,y) constitutes the fused detail layer image FuD i .

8. The multi-source image fusion method based on edge window filtering according to claim 4, characterized in that: In step (7), the calculation formula of the fused image FuI is as follows: Here, n represents the decomposition level.

Images