A circuit board infrared and visible light image fusion method based on DTCWT

Abstract

The application discloses a circuit board infrared and visible light image fusion method based on DTCWT and belongs to the field of image processing. In the method, an infrared image with pseudo-color coding is selected as input, and visible light color images and infrared pseudo-color images are fused in three channels in an RGB space. In the method, low-frequency parts mainly retain information from the infrared image, so that the low-frequency background parts basically present pseudo-color effects reflecting the distribution of hot areas; high-frequency parts adopt appropriate fusion criteria to retain edge detail information in the infrared and visible light images. Although the method discards most color information in the visible light image, the obtained fused image is more in line with human visual habits and actual engineering needs.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, in particular to a circuit board infrared and visible light image fusion method based on DTCWT BACKGROUND

[0002] In the current common infrared and visible light image fusion methods, a single channel image is mostly taken as the input, and the information represented is relatively single. In addition, the existing infrared and visible light image fusion methods usually involve color transfer, resulting in unnatural colors of the fused image. Therefore, the intelligibility and visualization effect of the fused image obtained by the existing methods are poor, which is not conducive to the accurate understanding and effective use of image information in engineering applications.

[0003] After analyzing the overhead and performance of the existing various infrared and visible light image fusion methods, it is found that the existing methods have certain deficiencies. Some methods require a large amount of computing resources and time overhead in the fusion process, resulting in slow processing speed, which is not suitable for real-time applications or large-scale data processing scenarios. In addition, some methods perform poorly in terms of fusion image quality and information preservation, and cannot well balance the preservation of image details and overall information. These problems limit the application range and effect of the existing methods in the engineering field, and a more efficient and effective fusion method needs to be sought to meet the actual needs. SUMMARY

[0004] Considering the limitations of the existing methods in solving the problem of infrared and visible light image fusion under the application of circuit board detection, the present application proposes a circuit board infrared and visible light image fusion method based on DTCWT.

[0005] The method of the present application uses DTCWT to decompose the source images to be fused, obtaining low frequency subbands and high frequency subbands of the infrared and visible light images; a fusion rule based on two-scale decomposition and saliency detection is used to fuse the low frequency subbands, and a fusion rule based on local energy detection is used to fuse the high frequency subbands; and DTCWT inverse transformation is performed on the fused low frequency and high frequency subbands to obtain the fusion result of the infrared and visible light images.

[0006] Thus, the technical solution of the present application is a circuit board infrared and visible light image fusion method based on DTCWT, which comprises the following steps:

[0007] S1: performing dual-tree complex wavelet transform (DTCWT) on the input infrared and visible light source images to obtain low frequency subbands D IR and D VI and corresponding high frequency subbands and Where k represents the scale of the high-frequency subband, l represents the direction of the high-frequency subband, and the value of l is an integer between [1,6];

[0008] S2: Low frequency subband D for infrared and visible light images IR and D VI Perform two-scale decomposition; Gaussian low-pass filtering is performed on the low-frequency sub-bands of the infrared and visible light images to obtain the corresponding base layer image D IR,B and D VI,B ; Perform differential operation on the low-frequency subband and the base layer to obtain the corresponding detail layer image D IR,D and D VI,D ;

[0009] S3: Perform saliency detection on the low-frequency sub-bands of infrared and visible light images to obtain saliency maps S of the low-frequency sub-bands of infrared and visible light images respectively. IR and S VI ; Based on the results of saliency detection, calculate the detail layer D IR,D and D VI,D The fusion weight graph W IR,D and W VI,D ;

[0010] S4: Use the linear weighted method with preset weight coefficients to fuse the base layer D IR,B and D VI,B , get the base layer fusion image D F,B ; Using the calculated fusion weight map W IR,D and W VI,D Fusion detail layer D IR,D and D VI,D , get the detail layer fusion image D F,D ; Fuse the base layer image D F，B and detail layer fusion image D F，D Superposition to obtain the fusion image D of infrared and visible light low-frequency sub-bands F ;

[0011] S5: Calculate the local energy of the infrared and visible light high-frequency sub-bands and obtain the local energy map of the high-frequency sub-bands and

[0012] S6: Calculate the matching degree between the corresponding high-frequency sub-band images of the infrared and visible light images to obtain the matching degree map M k，l ;

[0013] S7: Matching degree M between high-frequency sub-band images based on infrared and visible light images k，l , as well as the density distribution of the local energy of the high-frequency sub-band in each direction, the corresponding high-frequency sub-bands are subjected to pixel-level weighted fusion to obtain the fusion image of the infrared and visible light high-frequency sub-bands

[0014] S8: Fusion sub-band D of low frequency and high frequency using DTCWT F and Inverse transform reconstruction, get the final output fusion image.

[0015] Further, in S3, the calculation method of the saliency map of the infrared and visible light image low frequency sub-band is the absolute value of the difference between the base layer image obtained by Gaussian low-pass filtering and the filtered image obtained by median filtering, and the fusion weight coefficient of the low frequency sub-band detail layer is calculated as follows:

[0016]

[0017] W IR，D (i,j) represents the fusion weight coefficient of the infrared low frequency sub-band detail layer at position (i,j), W VI，D (i,j) represents the fusion weight coefficient of the visible light low frequency sub-band detail layer at position (i,j), S IR (i,j) represents the saliency value of the infrared image at position (i,j), S VI (i,j) represents the saliency value of the visible light image at position (i,j).

[0018] Further, in S4, the base layer fusion image D F , B is calculated by linear weighting of the base layer D IR，B and D VI,B of the low frequency sub-band of the infrared and visible light images:

[0019] D F,B = 0.9*D IR,B + 0.1*D VI,B

[0020] The detail layer fusion image D F，D is calculated by the detail layer D IR,D and D VI,D of the low frequency sub-band of the infrared and visible light images and their fusion weight maps W IR,D and W VI，D :

[0021] D F,D = W IR,D *D IR,D +W VI,D *D VI,D .

[0022] Further, in S5, the local energy of the infrared and visible light high frequency sub-band and is calculated as follows:

[0023]

[0024] where w denotes a 5x5 neighborhood centered at (i,j) in the image, denotes the infrared high-frequency sub-band energy at position (i,j) in the l direction at k scale, denotes the visible light high-frequency sub-band energy at position (i,j) in the l direction at k scale, denotes the pixel value of the infrared high-frequency sub-band image at position (i ′ ′ j) in the l direction at k scale, denotes the pixel value of the visible light high-frequency sub-band image at position (i ′ ′ j) in the l direction at k scale;

[0025] Further, in S6, the matching degree between the infrared and visible light high-frequency sub-bands to be fused is calculated as follows:

[0026]

[0027] M k,l (i,j) denotes the matching degree of the infrared and visible light high-frequency sub-bands at position (i,j) in the l direction at k scale;

[0028] Further, in S7, the high-frequency sub-bands of the infrared and visible light are fused in the following manner:

[0029] When the matching degree M k,l (i,j) between the high-frequency sub-bands to be fused is less than a, where a is a preset threshold, if If denotes the pixel value of the fused image at position (i,j);

[0030] When the matching degree M k,l (i,j) between the high-frequency sub-bands to be fused is greater than or equal to a, if and

[0031]

[0032] If or

[0033]

[0034] ​​​​​​The application provides an infrared and visible light image fusion method based on DTCWT. The method decomposes infrared and visible light source images into low frequency and high frequency subbands by DTCWT, fuses corresponding subbands by appropriate fusion rules, and obtains a fused image by inverse DTCWT. Unlike general infrared and visible light image fusion methods, in order to improve the visualization effect and readability of infrared images, the method selects pseudo-color coded infrared images as input, and fuses visible light color images and infrared pseudo-color images in three channels in RGB space. The low frequency part mainly retains information from infrared images, so that the low frequency background part basically presents pseudo-color effect reflecting thermal zone distribution; the high frequency part adopts appropriate fusion rules to retain edge detail information in infrared and visible light images. The fusion method discards most color information in visible light images, but the obtained fused image is more in line with human visual habits and actual engineering needs. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 It is a schematic diagram of the image fusion framework based on DTCWT.

[0036] Figure 2 It is a low frequency subband fusion flowchart.

[0037] Figure 3 It is a high frequency subband fusion flowchart.

[0038] Figure 4 It is an infrared source image.

[0039] Figure 5 It is a visible light source image.

[0040] Figure 6 It is a fused image. DETAILED DESCRIPTION

[0041] The application will be described in detail below with reference to the accompanying drawings.

[0042] The embodiment provides an infrared and visible light image fusion method based on dual tree complex wavelet transform (DTCWT), and a basic framework thereof is shown in Figure 1 , wherein the fusion methods of low frequency and high frequency subbands are shown in Figure 2 and Figure 3 . The specific steps of the method are as follows:

[0043] S1, source image DTCWT decomposition. 2-level decomposition of input infrared and visible light source images is carried out by DTCWT, Figure 4 and Figure 5 are infrared and visible light source images used in the example. The low frequency subbands of the obtained infrared and visible light images are respectively denoted as D IRand D VI , respectively. and (where k represents the scale of the high-frequency subband, and l represents the direction of the high-frequency subband, and l takes an integer between 1 and 6).

[0044] S2, two-scale decomposition of the low-frequency subband.

[0045] S2-1, base layer extraction. Perform Gaussian low-pass filtering on the low-frequency subbands D IR and D VI , respectively, to obtain the base layer images D IR,B and D VI,B .

[0046] S2-2, detail layer extraction. Perform difference operation on the low-frequency subbands D IR and D VI and their corresponding base layers D IR,B and D VI，B , respectively, to obtain the detail layer images D IR,D and D VI,D .

[0047] S3, saliency detection of the low-frequency subband.

[0048] S3-1, saliency map calculation. Perform saliency detection on the low-frequency subbands of the infrared and visible light images, respectively, to obtain the saliency maps S IR and S VI of the low-frequency subbands of the infrared and visible light images, respectively. The saliency calculation method is as follows:

[0049]

[0050] where D IR,M and D VI,M are the results of the median filtering of the low-frequency subbands of the infrared and visible light images, respectively.

[0051] S3-2, fusion weight map calculation. Based on the results of the saliency detection, calculate the fusion weight maps W IR,D and W VI,D of the detail layers D IR,D and D VI,D :

[0052]

[0053] S4, fusion of the low-frequency subband.

[0054] S4-1, base layer fusion. Fuse the base layers D IR,B and D VI,B using a linear weighting method with a preset weight coefficient to obtain the base layer fusion image D F,B :

[0055] D F,B =0.9*D IR,B +0.1*D VI,B

[0056] S4-2, detail layer fusion. Using the calculated fusion weight map W IR,D and W VI,D Fusion detail layer D IR,D and D VI ,D , get the detail layer fusion image D F,D :

[0057] D F,D =W IR,D *D IR,D +W VI,D *D VI,D

[0058] S4-3, the base layer and the detail layer are superimposed. The base layer is fused with the image D F,B and detail layer fusion image D F,D Superposition to obtain the fusion image D of infrared and visible light low-frequency sub-bands F :

[0059] D F =D F,B +D F,D

[0060] S5, high frequency sub-band local energy calculation. Calculate the local energy map of each high frequency sub-band of infrared and visible light images respectively. and

[0061]

[0062] Where w represents the 5×5 neighborhood centered at (i, j) in the image.

[0063] S6, high frequency sub-band matching calculation. Calculate the matching degree M between the high frequency sub-bands corresponding to the infrared and visible light images. k,l :

[0064]

[0065] S7, high-frequency sub-band fusion. The high-frequency sub-band matching threshold α is preset to 0.75. Different fusion rules are used for the two cases where the matching degree between the high-frequency sub-bands to be fused is lower than or not lower than the threshold α. The specific introduction is as follows:

[0066] C1, the matching degree between the high-frequency sub-bands to be fused is lower than α, that is, M k,lWhen (i, j) < α, the pixel value of the fused image is the larger of the pixel values ​​of the two high-frequency sub-bands to be fused at that point:

[0067]

[0068] C2, the matching degree between the high-frequency sub-bands to be fused is not less than α, that is, M k,l When (i, j) ≥ α, the pixel value of the fused image depends on the local energy of the two high-frequency sub-bands to be fused at that point and their density distribution in all directions.

[0069] C2-1, if and The pixel value of the fused image is calculated as follows:

[0070]

[0071] C2-2, if or The pixel value of the fused image is calculated as follows:

[0072]

[0073] S8, DTCWT inverse transform reconstruction. Use DTCWT to reconstruct the fused low-frequency subband D F and high frequency sub-band Perform inverse transformation and reconstruction to obtain the final output fusion image, such as Figure 6 shown.

[0074] This invention fuses infrared and visible light images to more effectively detect faults and defects on circuit boards. Infrared images can reveal temperature anomalies, while visible light images can reveal detailed circuit board structures. Combining these two methods can more accurately identify the location and type of faults, improving the efficiency and accuracy of fault detection.

[0075] The present invention performs image fusion based on the dual-tree complex wavelet transform (DTCWT). The resulting fused image can present a pseudo-color effect reflecting the temperature zone distribution of the circuit board, while retaining and enhancing edge and detail information, making the fused image more in line with human visual habits and facilitating engineers and technicians to inspect and analyze the circuit board.

[0076] This method avoids unnatural color transfer during the fusion of infrared and visible light images. Furthermore, compared to other multi-scale transforms, DTCWT balances detail capture with computational efficiency. Therefore, this method can meet the efficiency requirements of practical engineering applications.

[0077] The embodiment illustrates how to realize high-quality image fusion by decomposing, fusing and reconstructing infrared and visible light images through a dual-tree complex wavelet transform, and is suitable for application scenarios such as circuit board detection which need multi-source information fusion. The above only describes the preferred embodiment of the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for fusing infrared and visible light images of a circuit board based on DTCWT, the method comprising the following steps: S1: Perform dual-tree complex wavelet transform on the input infrared and visible light images to obtain the low-frequency subband D of the infrared and visible light images IR and D VI , and the corresponding high-frequency sub-band and Where k represents the scale of the high-frequency subband, l represents the direction of the high-frequency subband, and the value of l is an integer between [1,6]; S2: Low frequency subband D for infrared and visible light images IR and D VI Perform two-scale decomposition, specifically including: Gaussian low-pass filtering on the low-frequency sub-bands of infrared and visible light images to obtain the corresponding base layer image D IR,B and D VI,B ; Perform differential operation on the low-frequency subband and the base layer to obtain the corresponding detail layer image D IR,D and D VI,D ; S3: Perform saliency detection on the low-frequency sub-bands of infrared and visible light images to obtain saliency maps S of the low-frequency sub-bands of infrared and visible light images respectively. IR and S VI ; Based on the results of saliency detection, calculate the detail layer D IR,D and D VI,D The fusion weight graph W IR ,D and W VI,D ; S4: Use the linear weighted method with preset weight coefficients to fuse the base layer D IR,B and D VI,B , get the base layer fusion image D F ,B ; Using the calculated fusion weight map W IR,D and W VI,D Fusion detail layer D IR,D and D VI,D , get the detail layer fusion image D F ,D ; Fuse the base layer image D F,B and detail layer fusion image D F,D Superposition to obtain the fusion image D of infrared and visible light low-frequency sub-bands F ; S5: Calculate the local energy of the infrared and visible light high-frequency sub-bands and obtain the local energy map of the high-frequency sub-bands and S6: Calculate the matching degree between the corresponding high-frequency sub-band images of the infrared and visible light images to obtain the matching degree map M k,l ; S7: Matching degree map M between high frequency sub-band images based on infrared and visible light images k,l , as well as the density distribution of the local energy of the high-frequency sub-band in each direction, the corresponding high-frequency sub-bands are subjected to pixel-level weighted fusion to obtain the fusion image of the infrared and visible light high-frequency sub-bands S8: Using DTCWT to fusion image of infrared and visible light low frequency sub-bands F and fusion images of infrared and visible high-frequency sub-bands Perform inverse transform and reconstruction to obtain the final output fused image.

2. The DTCWT-based circuit board infrared and visible light image fusion method according to claim 1, characterized in that: In S3, the saliency map of the low-frequency sub-bands of infrared and visible light images is calculated as the absolute value of the difference between the base layer image obtained by Gaussian low-pass filtering and the filtered image obtained by median filtering. The fusion weight coefficient of the detail layer of the low-frequency sub-band is calculated as follows: W IR,D (i,j) represents the fusion weight map W IR,D The fusion weight coefficient of the infrared low-frequency sub-band detail layer at position (i, j), W VI,D (i,j) represents the fusion weight map W VI,D The fusion weight coefficient of the visible light low-frequency sub-band detail layer at position (i, j), S IR (i,j) represents the saliency map S IR The saliency value of the infrared image at position (i, j), S VI (i,j) represents the saliency map S VI The saliency value of the visible light image at position (i, j) in .

3. The DTCWT-based circuit board infrared and visible light image fusion method according to claim 1, characterized in that: In S4, the base layer fused image D F,B The base layer D is composed of the low-frequency sub-bands of infrared and visible light images. IR,B and D VI,B The linear weighted calculation is: D F,B =0.9*D IR,B +0.1*D VI,B Detail layer fused image D F,D The detail layer D of the low-frequency sub-bands of infrared and visible light images IR,D and D VI,D and its fusion weight graph W IR,D and W VI,D Calculation yields: D F,D =W IR,D *D IR,D +W VI,D *D VI,D 。 4. The method for fusing infrared and visible light images of a circuit board based on DTCWT according to claim 1, characterized in that: In S5, the local energy of infrared and visible high-frequency sub-bands and The calculation is as follows: Where w represents the 5×5 neighborhood range centered at (i, j) in the image. represents the infrared high-frequency subband energy at position (i, j) in the k-scale and l-direction. represents the energy of the high-frequency sub-band of visible light at position (i, j) in the k-scale and l-direction. Indicates the k-scale, l-direction, position (i ′ ,j ′ ) is the pixel value of the infrared high-frequency sub-band image, Indicates the position (i ′ ,j ′ ) is the pixel value of the visible light high-frequency sub-band image at .

5. The DTCWT-based circuit board infrared and visible light image fusion method according to claim 4, characterized in that: In S6, the matching degree between the infrared and visible high-frequency sub-bands to be fused is calculated as follows: M k,l (i, j) represents the matching degree of the infrared and visible light high-frequency sub-bands at the position (i, j) in the k scale and l direction.

6. The DTCWT-based circuit board infrared and visible light image fusion method according to claim 5, characterized in that: In S7, the high-frequency sub-bands of infrared and visible light are fused in the following way: The matching degree M between the high-frequency sub-bands to be fused k,l (i,j)<α, where α is the preset threshold, if but like but Represents the pixel value of the fused image at position (i, j); The matching degree M between the high-frequency sub-bands to be fused k,l When (i,j)≥α, if and but like or but

Images