Infrared and visible light image perception fusion method based on multi-scale structure decomposition
By converting infrared and visible light images into human visual response space and using multi-scale structural decomposition and bidirectional significance aggregation strategies, the problem of poor visual perception in the prior art is solved, and high-quality infrared and visible light images are achieved.
Patent Information
- Application Number
- CN202210381391.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-12
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-04-12
AI Technical Summary
Existing infrared and visible image fusion methods rarely consider the characteristics of human visual systems, resulting in the fusion results being greatly affected by external physical factors and lacking visual perception effects.
Using a multi-scale structure decomposition method, infrared and visible light images are converted into human visual response space for comparison and fusion. Through the scale-aware edge-retention filtering algorithm and bidirectional significance aggregation strategy, the fusion weight is determined and a good visual perception effect is generated. Fusion images.
Fusion in a unified visual response space eliminates the influence of external physical factors, generates fusion images with rich information and good visual perception effects, solves the potential defects of the existing technology and improves the fusion effect.
Smart Images

Figure CN114897751B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an infrared and visible light image perception fusion method based on multi-scale structural decomposition, and belongs to the technical field of multi-sensor image fusion. Background Art
[0002] Image fusion technology is of great significance in image processing and computer vision, and has been widely used in military, remote sensing, medical image processing, industrial detection and other fields. Among them, infrared and visible light image fusion has become one of the most studied branches due to its unique application. Visible light images usually have high resolution and contain important details of the scene, but their imaging quality is easily affected by external factors such as weather and light. In contrast, infrared images contain hidden information that is lost in visible light images and can reflect the thermal radiation information of the scene, but their detail information is usually poor. Therefore, the information of visible light and infrared images is complementary to a certain extent. Fusion of infrared and visible light images can obtain a relatively complete scene mapping. A basic principle of infrared and visible light image fusion is to retain the significant information in infrared and visible light images as much as possible. In addition, it is hoped that the fused image can introduce fewer artifacts and have a good visual perception effect.
[0003] Generally, infrared and visible light image fusion includes three important steps, namely feature extraction, fusion strategy formulation and image reconstruction. According to the different analysis tools used in the above process, the existing infrared and visible light image fusion algorithms can be divided into six categories: multi-scale transform-based methods, subspace-based methods, sparse representation-based methods, saliency-based methods, deep learning-based methods and hybrid methods. Among these methods, the most widely studied and applied are fusion methods based on multi-scale transforms. This type of method first uses transform technology to decompose the source image to obtain multi-scale information, and then uses a certain fusion strategy to fuse each scale information one by one. Among them, the Laplacian pyramid is a classic transform technology often used for multi-scale decomposition, and it derives multi-scale decomposition technologies such as contrast pyramid, steerable pyramid and morphological pyramid. Wavelet transform, as another important multi-scale decomposition tool, provides a method to decompose an image into a low-pass layer image and detail layer images in different directions, thereby reducing the noise in the fused image. On this basis, researchers have proposed improved discrete wavelet transform, contourlet transform, shearlet transform and other analysis tools, which have better decomposition performance. In addition, many edge-preserving filters such as bilateral filters and guided filters have been proposed and widely used in multi-scale decomposition of images. They can maintain the spatial continuity of image structure and reduce the generation of halos and artifacts. In the fusion process, the maximum value and weighted average strategy are often used to determine the fusion weight. Toet et al. decomposed the source image using the contrast pyramid transform and selected the maximum contrast value as the fusion coefficient. Adu et al. proposed to use the weighted average strategy to calculate the weight coefficient of the decomposed image, and then fuse the images of the same scale through the weight coefficient.
[0004] Existing fusion methods focus more on preserving salient information or avoiding artifacts in infrared and visible images, while less consideration is given to perceptual issues and the characteristics of the human visual system (HVS). Considering the mechanism of HVS can not only produce what we usually think of as visually pleasing fusion results, but more importantly, it can help address potential flaws in current fusion frameworks. Typically, the image fusion process involves using visual features from different source images, especially by comparing them to determine fusion weights or obtain appropriate information fusion strategies. However, visual features are easily affected by external physical conditions (such as ambient lighting, characteristics of different sensors, etc.), which means that these features are not placed on an equal and clear basis when compared and fused, which affects the fusion quality, especially for infrared and visible image fusion, where the response characteristics of the two sensors are very different and the visual information in the visible spectrum is severely affected by changes in external lighting conditions. Summary of the invention
[0005] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and propose an infrared and visible light image perception fusion method based on multi-scale structural decomposition. The method transforms the physical intensity of the source image into the visual response space of the HVS, so that the input information from different source images can be compared and fused in a unified human visual response space, in which all features are in the same perceptual state, thereby eliminating external physical factors that may affect the fusion process, and finally generating a fused image with rich information and good visual perception effect, solving the current potential defects and improving the fusion effect.
[0006] The technical solution of the present invention is:
[0007] The infrared and visible light image perception fusion method based on multi-scale structure decomposition comprises the following steps:
[0008] Step 1: Perform multi-scale structural decomposition on the infrared and visible light images of the same scene based on the scale-aware edge preserving (SAEP) filtering algorithm to obtain infrared and visible light multi-scale filtered images. and Where j = 0, 1, ..., N; N is the number of scale layers;
[0009] Step 2: The infrared and visible light multi-scale filtered images obtained in step 1 are and Converted to the HVS visual response space to obtain the multi-scale perceptual contrast of the infrared image Multi-scale perceptual contrast of visible light images
[0010] The conversion method is:
[0011] (1) Calculate the multi-scale adaptive contrast of infrared and visible light images based on the contrast sensitivity and local adaptation mechanism of HVS and
[0012]
[0013]
[0014] in, is the j-th low-pass image of the infrared image, is the j-th bandpass image of the infrared image, is the j-th low-pass image of the visible light image, is the j-th bandpass image of the visible light image, t is an adaptive parameter, is a set value, preferably, t=1, α is an adjustment parameter, preferably, α=0.8,
[0015] (2) Based on the nonlinear conversion mechanism of HVS, the initial values of multi-scale perceptual contrast of infrared and visible light images are calculated. and
[0016] when When taking positive values and 0,
[0017]
[0018] when When taking negative values,
[0019]
[0020] when When taking positive values and 0,
[0021]
[0022] when When taking negative values,
[0023]
[0024] Among them, h is the threshold, preferably, h = 0.5; c is a constant, set to c = 21.3, at different scale layers, the p value is different, when j from small to large, when N = 4, the p values are: 1.40, 1.15, 1.04, 1.15;
[0025] (3) The initial value of multi-scale perceptual contrast obtained in step (2) and Noise and intensity saturation suppression are performed to obtain the multi-scale perceptual contrast of the final infrared image. Multi-scale perceptual contrast of visible light images
[0026] The noise suppression method is as follows:
[0027]
[0028]
[0029] Among them, th is the threshold, which is used to distinguish noise from useful information. is the average gray value of the source image; since noise usually appears in small-scale layers, it is usually set
[0030] The intensity suppression method is as follows:
[0031]
[0032]
[0033] Different from noise, overexposure often occurs in large-scale layers. r is the overexposure suppression parameter, I 0 represents the source image;
[0034] Step 3: Based on the visual characteristics of the human eye, the lowest-level low-pass images of the infrared and visible light images are and Adaptively adjust and determine the fusion weight based on the saliency strategy to obtain the lowest-level low-pass fusion image of infrared and visible light.
[0035] The specific method is:
[0036] (1) The lowest-level low-pass image of infrared and visible light images and Perform adaptive adjustment to obtain the adjusted bottom-level low-pass image and
[0037]
[0038]
[0039] Where l is the threshold, which reflects the average background brightness of the lowest low-pass image after adjustment, and is set to l = 128;
[0040] (2) Determine the lowest-level low-pass image of visible light based on the saliency fusion strategy The fusion weight w is:
[0041]
[0042] in represents the Gaussian filtering operation, and are the saliency maps of the lowest-level low-pass images of visible light and infrared images, respectively, and are calculated as follows:
[0043]
[0044]
[0045] in, and They represent the grayscale values of pixel n and its adjacent pixel k in the underlying image area Ω of the infrared image, and They represent the grayscale values of pixel n and its adjacent pixel k in the underlying image area Ω of the visible light image respectively;
[0046] Thus, the infrared and visible light lowest level low-pass fusion image for
[0047]
[0048] Step 4: Determine the multi-scale perceptual contrast of infrared images using a bidirectional saliency aggregation strategy Multi-scale perceptual contrast of visible light images The fusion weights of , one direction is the top-down combination of pixel-level saliency, and the other is the aggregation of the opposite direction of structural saliency, and further obtain the fusion image of the j-th layer perceptual contrast of infrared and visible light images;
[0049] Specifically:
[0050] (1) For pixel-level saliency, perceptual contrast and Aggregate from small scale to large scale to get the j-th layer perceptual contrast Pixel-level saliency and the jth layer perceived contrast Pixel-level saliency
[0051]
[0052]
[0053] (2) For structural saliency, perceptual contrast and Aggregation is performed from large scale to small scale respectively, and the adjusted bottom-level low-pass layer image is also considered and Thus, the j-th layer perceptual contrast is obtained The structural significance and the jth layer perceived contrast The structural significance
[0054]
[0055] Where sf is the structural significance function, which is calculated as follows:
[0056]
[0057] γ is the equilibrium parameter, let γ = 0.1; s 1 、s 2 It is related to the eigenvalue of the gradient covariance matrix C and is obtained by the following formula:
[0058]
[0059] Among them I x (X) and I y (X) represents the local window W i The gradient of pixel X in the x and y directions.
[0060] (3) Calculate the perceptual contrast of the jth layer The overall significance and the jth layer perceived contrast The overall significance
[0061]
[0062]
[0063] Where β is the equilibrium parameter, β=5, is the saliency adjustment map of the jth layer of the infrared image, is the saliency adjustment map of the jth layer of the visible light image, and its value is as follows:
[0064]
[0065]
[0066] in, is the source infrared image, represents the average gray value of the infrared image in the neighborhood Ω; sg represents the sigmoid function, u is a control parameter. When u takes different values, the shape of the sigmoid function is different. Here, u=5.
[0067] (4) Overall saliency of perceived contrast in each layer of visible and infrared images and The fusion weight of the perceptual contrast of each layer of the visible light image is:
[0068]
[0069] Among them, at different scale layers, u has different values, u = 0.1*2 4-j .
[0070] Furthermore, the fusion image of the infrared and visible light images with perceptual contrast at layer j is for
[0071] Step 5: Get the final fused image through inverse transformation and reconstruction process:
[0072]
[0073] in, is the lowest-level low-pass fused image, The inverse transformation process in step 2 yields:
[0074]
[0075] Perceptual contrast images for the fused layers.
[0076] Beneficial Effects
[0077] 1. This paper proposes a new perception framework based on multi-scale structure decomposition for infrared and visible light image fusion. The proposed framework fully considers the relevant characteristics of HVS and can help solve the potential defects of current fusion research in visual information perception.
[0078] 2. The present invention constructs a multi-scale structure decomposition method based on SAEP filter to design a perceptual fusion framework. Compared with other algorithms, this method has excellent edge preservation and scale perception characteristics, and can obtain image structures of different scales, where edge information is retained in each layer and small-scale details can be regarded as structures with fine spatial scales.
[0079] 3. The present invention proposes a new bidirectional saliency aggregation algorithm to determine the fusion weight of multi-scale perceptual contrast. The algorithm fully considers the pixel-level saliency information and large-scale structural information, so as to obtain a fused image with rich information and good visual perception effect.
[0080] 4. The proposed framework combines some key features of HVS, including multi-scale processing channels, contrast sensitivity, local adaptation, and super-threshold characteristics. All relevant key features of HVS are integrated into the proposed framework to simulate human visual responses in complex scenes, thereby creating a visual response space represented by multi-scale perceptual contrast in HVS.
[0081] 5. The present invention constructs a multi-scale structure decomposition by using a scale-aware edge preserving (SAEP) filter, which has good scale separation and edge preservation properties. Image structures of different scales are obtained through decomposition, and their edges are preserved in each layer, and the details can be regarded as image structures with fine spatial scales.
[0082] 6. In the fusion process, the present invention proposes a bidirectional saliency aggregation strategy to fuse the perceptual contrast of each scale, one direction is aggregated from top to bottom in the scale space to obtain pixel-level saliency, and the other direction is reversely aggregated to calculate structural saliency. Then the two types of saliency are combined and the fusion weight is calculated according to the sigmoid function. BRIEF DESCRIPTION OF THE DRAWINGS
[0083] Figure 1 The sigmoid function for different values of u;
[0084] Figure 2 is a flow chart of the fusion framework of the present invention;
[0085] Figure 3 Comparison of fusion images of infrared and visible light images obtained by different methods. Among them, (a) is an infrared image, (b) is a visible light image, (c) is a fusion image of infrared and visible light images obtained by the WLS method, (d) is a fusion image of infrared and visible light images obtained by the U2Fusion method, (e) is a fusion image of infrared and visible light images obtained by the IFCNN method, and (f) is a fusion image of infrared and visible light images obtained by the method of the present invention. DETAILED DESCRIPTION
[0086] The present invention is described in detail below in conjunction with the accompanying drawings.
[0087] The present invention proposes a perceptual fusion framework of infrared and visible light images based on multi-scale structure decomposition, which draws on the relevant mechanisms of the human visual system (HVS) to transform the source images into the visual response space of the HVS for comparison and fusion.
[0088] Based on this, the specific implementation scheme of the present invention is:
[0089] Assume that the input infrared and visible light images are I r and I v ,like Figure 2 As shown, the fusion steps are as follows:
[0090] Step 1: According to the multi-scale processing channel characteristics of HVS, the present invention constructs a multi-scale structural decomposition based on the scale-aware edge preservation (SAEP) filtering algorithm to obtain infrared and visible light multi-scale filtered images and
[0091] I j =SAEP(I j-1 ,λ j ,r j,0 ,r j,1 ),j=1,2,…,N
[0092] Among them I 0 =I, λ is the global smoothing weight, λ 1 =0.1,λ j+1 =λ j +0.9; r is the scale parameter, the scale is [r j,0 ,r j,1] will be smoothed. In this example, r 1,0 =0,r 1,1 =4,r j+1,1 =2r j,1 ,r j+1,0 =r j,1 , filtering times N = 4.
[0093] Furthermore, the infrared and visible light bandpass images and It is obtained by the following formula:
[0094] B j =I j-1 -I j ,j=1,2,…,N
[0095] Step 2: Based on the contrast sensitivity, local adaptation and super-threshold characteristics of HVS, the bandpass and low-pass images obtained by the above multi-scale decomposition are converted into the visual response space of HVS to obtain the multi-scale perceptual contrast of the infrared image. Multi-scale perceptual contrast of visible light images
[0096] (1) Calculate the multi-scale adaptive contrast of infrared and visible light images based on the contrast sensitivity and local adaptive mechanism of HVS and
[0097]
[0098] Among them, I j and B j are the j-th layer low-pass image and band-pass image of the infrared or visible light image respectively; t is an adaptive parameter, which takes different values according to the characteristics of the infrared and visible light images; α is an adjustment parameter, preferably, α=0.8.
[0099] (2) The adaptive contrast obtained by (1) adapts to human vision to a certain extent, but it is still not the perceptual contrast in the visual response space. HVS has a nonlinear conversion function that helps to obtain perceptual contrast in this unified space. Based on the nonlinear conversion mechanism of HVS, the multi-scale perceptual contrast of infrared and visible light images is obtained. and
[0100]
[0101] Where R jTake a positive value. If it is negative, take its absolute value as input for calculation and invert its output. h is the threshold. Preferably, h=0.5. c is a constant, set to c=21.3. At different scale layers, the p value is different. When the scale increases from small to large, its values are: 1.40, 1.15, 1.04, 1.15, 1.35, 1.93 respectively.
[0102] (3) Further suppress the noise and intensity saturation of the obtained perceived contrast to obtain the multi-scale perceived contrast of the final infrared image Multi-scale perceptual contrast of visible light images Typically, smaller scale layers contain more noise, while larger scale layers have more overexposure. Therefore, different suppression methods are applied in different frequency layers.
[0103] The noise suppression method is as follows:
[0104]
[0105] Where th is the threshold value, which is used to distinguish noise from useful information. is the average grayscale value of the source image; in this example,
[0106] The method to suppress intensity saturation is as follows:
[0107]
[0108] Where r is the overexposure suppression parameter, I 0 Represents the source image.
[0109] Step 3: The lowest low-pass layer of the infrared and visible light images reflects the background information of the scene. According to the visual characteristics of the human eye, the lowest low-pass image of the infrared and visible light images is and Perform adaptive adjustment and determine its fusion weight based on the saliency strategy:
[0110] (1) The lowest-level low-pass image of infrared and visible light images and Perform adaptive adjustment to obtain the adjusted bottom-level low-pass image and
[0111]
[0112] Wherein l is a threshold value, which reflects the average background brightness of the lowest low-pass image after adjustment, and is set to l=128; α is an adjustment parameter, preferably, α=0.8.
[0113] (2) Calculate the lowest-level low-pass image of visible light based on the saliency fusion strategy The fusion weight w is:
[0114]
[0115] in represents the Gaussian filtering operation, and are the saliency maps of the lowest-level low-pass images of visible light and infrared images, respectively, and are calculated as follows:
[0116]
[0117] Among them A N (n) and A N (k) represent the grayscale values of pixel n and its adjacent pixel k in the underlying image area Ω respectively.
[0118] Thus, the infrared and visible light lowest level low-pass fusion image for
[0119]
[0120] Step 4: In the visual response space of HVS, the perceived contrast usually contains fine pixel-level salient information and image structural information. Therefore, for the multi-scale perceived contrast of infrared images Multi-scale perceptual contrast of visible light images The present invention proposes a bidirectional saliency aggregation strategy to fully aggregate these features and determine the fusion weights based on this, one direction of which is a top-down combination of pixel-level saliency and the other is an aggregation of structural saliency in the opposite direction.
[0121] (1) For pixel-level saliency, the present invention superimposes the perceptual contrast from small scale to large scale to obtain the j-th layer perceptual contrast C j The pixel-level saliency D j :
[0122]
[0123] The j-th layer of pixel-level saliency contains the fine-grained information of the current layer and smaller-scale layers, which can retain more complete details and make the final fused image more delicate.
[0124] (2) For structural saliency, the present invention aggregates the perceptual contrast from large scale to small scale. In addition, due to the adjusted bottom-level low-pass layer image A N contains the basic structural information of the source image, so we take it into account to obtain a relatively complete structural saliency. The j-th layer perceptual contrast C jThe structural significance of G j for:
[0125]
[0126] Among them, sf is the structural significance function, which can reflect the structural information such as corners in the image. It is calculated as follows:
[0127] α is a balance parameter. In the present invention, α=0.1; s 1 、s 2 It is related to the eigenvalue of the gradient covariance matrix C and is obtained by the following formula:
[0128]
[0129] Among them I x (X) and I y (X) represents the local window w i The gradient of pixel X in the x and y directions.
[0130] (3) Calculate the perceptual contrast C of the jth layer j The overall significance S j :
[0131] S j =M j *(D j +β*G j )
[0132] Where β is the balance parameter, β=5; * indicates the element-by-element multiplication operation; M j It is a saliency adjustment map, which can help the fused image capture more highlight target information and less noise from the infrared image. For infrared and visible light images, its values are as follows:
[0133]
[0134]
[0135] in, is the source infrared image, represents the average gray value of the infrared image in the neighborhood Ω; sg represents the sigmoid function, u is the control parameter. When u takes different values, the shape of the sigmoid function is different, such as Figure 1 As shown, here, u=5.
[0136] (4) Overall saliency of perceived contrast in each layer of visible and infrared images and The fusion weight of the perceptual contrast of each layer of the visible light image is:
[0137]
[0138] In different scale layers, u has different values. In this example, u = 0.1*2 4-j .
[0139] Furthermore, the fusion process of infrared and visible image perceptual contrast layers can be described as
[0140] in, is the fused image of the j-th layer perceptual contrast.
[0141] Step 5: The lowest low-pass fused image obtained in the visual response space of HVS based on steps 3 and 4 And the fused perceptual contrast images at each scale The final fused image is obtained through the inverse transformation and reconstruction process:
[0142]
[0143] in, The inverse transformation process in step 2 yields:
[0144]
[0145] Figure 3 The figure is a comparison of the fusion image of the present invention and other methods. Among them, (a) is an infrared image, (b) is a visible light image, (c), (d), (e), and (f) are the fusion results of the WLS method, the U2Fusion method, the IFCNN method, and the present invention method, respectively. It can be seen that the framework of the present invention can obtain better fusion results by performing fusion in a consistent and well-defined visual response space, because the relevant characteristics of the human visual system are fully considered.
Claims
1. Infrared and visible light image perception fusion method based on multi-scale structure decomposition, characterized by The steps of the method include: Step 1: Perform multi-scale structural decomposition on the infrared and visible light images of the same scene to obtain infrared and visible light multi-scale filtered images. and ,in ; is the number of scale layers; Step 2: The infrared and visible light multi-scale filtered images obtained in step 1 are and Converted to the HVS visual response space to obtain the multi-scale perceptual contrast of the infrared image Multi-scale perceptual contrast of visible light images ; Step 3: The lowest low-pass image of the infrared and visible light images and Adaptively adjust and determine the fusion weight based on the saliency strategy to obtain the lowest-level low-pass fusion image of infrared and visible light. ; Step 4: Determine the multi-scale perceptual contrast of the infrared image Multi-scale perceptual contrast of visible light images The fusion weights of infrared and visible light images are obtained. Layer-aware contrast fused images; Step 5: Fusion the infrared and visible light lowest-level low-pass images obtained in step 3 The fused image of the perceived contrast of each layer of the infrared and visible light images obtained in step 4 is then inversely transformed and reconstructed to obtain the final fused image; In step 2, the infrared and visible light multi-scale filtered images are and Converted to the HVS visual response space to obtain the multi-scale perceptual contrast of the infrared image Multi-scale perceptual contrast of visible light images The specific method is: (1) Calculate multi-scale adaptive contrast of infrared and visible light images and : in, The infrared image Layer low pass image, The infrared image Layer bandpass image, For visible light images Layer low pass image, For visible light images Layer bandpass image, is an adaptive parameter, is a set value, and , To adjust the parameters, take , , ; (2) Calculate the initial value of multi-scale perceptual contrast of infrared and visible light images and : when When taking positive values and 0, when When taking negative values, when When taking positive values and 0, when When taking negative values, in, is the threshold value, is a constant; At different scales, The value is different, when From small to large, when hour, The values are: 1.40, 1.15, 1.04, 1.15; (3) The initial value of multi-scale perceptual contrast obtained in step (2) and Noise and intensity saturation suppression are performed to obtain the multi-scale perceptual contrast of the final infrared image. Multi-scale perceptual contrast of visible light images : The noise suppression method is as follows: , in, is the threshold value, is the average gray value of the source image; ; The intensity suppression method is as follows: ; is the overexposure suppression parameter, Represents the grayscale value of the source image.
2. The infrared and visible light image perception fusion method based on multi-scale structural decomposition according to claim 1 is characterized by: In the step 1, multi-scale structure decomposition is performed on the infrared and visible light images of the same scene based on a scale-aware edge preserving (SAEP) filtering algorithm.
3. The infrared and visible light image perception fusion method based on multi-scale structural decomposition according to claim 1 is characterized by: In step 3, the lowest low-pass image of the infrared and visible light images is and Adaptively adjust and determine the fusion weight based on the saliency strategy to obtain the lowest-level low-pass fusion image of infrared and visible light. The method is: (1) Low-pass images of infrared and visible light images and Perform adaptive adjustment to obtain the adjusted bottom-level low-pass image and : in is the threshold value; (2) Determine the lowest-level low-pass image of visible light based on the saliency fusion strategy The fusion weight : in represents the Gaussian filtering operation, and are the saliency maps of the lowest-level low-pass images of visible light and infrared images, respectively, and are calculated as follows: in, and Respectively represent the underlying image area of the infrared image Medium pixel and its neighboring pixels The gray value of and Respectively represent the underlying image area of the visible light image Medium pixel and its neighboring pixels Gray value of Thus, the infrared and visible light lowest level low-pass fusion image for 。 4. The infrared and visible light image perception fusion method based on multi-scale structural decomposition according to claim 3 is characterized by: Step 4, using a bidirectional saliency aggregation strategy to determine the multi-scale perceptual contrast of infrared images Multi-scale perceptual contrast of visible light images The fusion weights of infrared and visible light images are obtained. The method of fusing images with layer-aware contrast is: (1) Perceived contrast and Aggregate from small scale to large scale to get Layer Perceived Contrast Pixel-level saliency , and Layer Perceived Contrast Pixel-level saliency : (2) Perceived contrast and Aggregate from large scale to small scale to get Layer Perceived Contrast The structural significance , and Layer Perceived Contrast The structural significance ; in is the structural significance function, which is calculated as follows: is the balance parameter, , It is obtained by the following formula: in and Respectively represent local windows Pixels within Gradients in the x and y directions; (3) Calculate the Layer Perceived Contrast The overall significance , and Layer Perceived Contrast The overall significance : in is the balance parameter, The infrared image The saliency adjustment map of the layer, For visible light images The saliency adjustment map of the layer has the following values: in, is the gray value of the source infrared image, represents the average gray value of the infrared image in the neighborhood Ω; represents the sigmoid function, , To control the parameters; (4) Based on the overall saliency of the perceived contrast of each layer of visible light and infrared images and , the fusion weight of the perceptual contrast of each layer of the visible light image is: Infrared and visible light images Layer-aware contrast fused images for 。 5. The infrared and visible light image perception fusion method based on multi-scale structural decomposition according to claim 4 is characterized by: In step 5, the method of obtaining the final fused image through the inverse transformation and reconstruction process is: in, is the lowest-level low-pass fused image, The inverse transformation process in step 2 yields: Perceptual contrast images for the fused layers.
6. The infrared and visible light image perception fusion method based on multi-scale structural decomposition according to claim 5 is characterized by: 。 7. The infrared and visible light image perception fusion method based on multi-scale structural decomposition according to claim 5 is characterized by: 。 8. The infrared and visible light image perception fusion method based on multi-scale structural decomposition according to claim 5 is characterized by: , , 。
