Circuit board infrared and visible light image fusion method based on DTCWT

Through DTCWT, decomposition of infrared and visible images and adopting appropriate fusion rules, the problem of insufficient computing resources and image quality in the prior art is solved, efficient image fusion is achieved, and the visualization effect of circuit board detection and fault detection accuracy is improved. .

CN118982464AActive Publication Date: 2024-11-19UNIV OF ELECTRONICS SCI & TECH OF CHINA

Patent Information

Application Number
CN202410857811.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-28
Publication Date
2024-11-19
Estimated Expiration
2044-06-28

AI Technical Summary

Technical Problem

The existing infrared and visible image fusion method is large in computing resources and time overhead, resulting in slow processing speed, poor image quality and information retention, and the inability to effectively balance image details and overall information, limiting the scope of application in engineering applications. and effects.

Method used

Using the DTCWT-based method, infrared and visible light images are decomposed into low-frequency and high-frequency subbands, and the low-frequency subbands are fused by the fusion rules of two-scale decomposition and significance detection, and the high-frequency subbands are fused by the local energy detection rules. The bands are fused and the fused image is obtained by inverse DTCWT transformation.

Benefits of technology

It improves the visual effect and readability of images, enhances the retention of edge and detailed information, conforms to human visual habits, is suitable for multi-source information fusion in engineering applications, and improves the efficiency and accuracy of fault detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982464A_ABST
    Figure CN118982464A_ABST
Patent Text Reader

Abstract

The invention discloses a circuit board infrared and visible light image fusion method based on DTCWT, and belongs to the field of image processing. According to the method, a pseudo-color coded infrared image is selected as input, and a visible light color image and an infrared pseudo-color image are fused on three channels in an RGB space. Wherein the low-frequency part mainly retains information from an infrared image, so that the low-frequency background part basically presents a pseudo-color effect reflecting hot area distribution; a proper fusion criterion is adopted for the high-frequency part, and edge detail information in the infrared and visible light images is reserved. According to the fusion method, although most color information in the visible light image is abandoned, the obtained fusion image better conforms to human visual habits and actual engineering requirements.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular to a method for fusing infrared and visible light images of a circuit board based on DTCWT. Background Art

[0002] Most of the common infrared and visible light image fusion methods currently use single-channel images as input, and the information represented is relatively simple. In addition, existing infrared and visible light image fusion methods usually involve color transfer, which causes the fused image to present unnatural colors. Therefore, the fusion image obtained by the existing method has poor comprehensibility and visualization effect, which is not conducive to the accurate understanding and effective use of image information in engineering applications.

[0003] The cost and performance of various existing infrared and visible light image fusion methods were analyzed, and it was found that the existing methods have certain shortcomings. Some methods require a lot of computing resources and time overhead during the fusion process, resulting in slow processing speed and are not suitable for real-time applications or large-scale data processing scenarios. In addition, some methods perform poorly in terms of fused image quality and information retention, and cannot balance the retention of image details and overall information well. These problems limit the application scope and effect of existing methods in the engineering field, and it is necessary to seek more efficient and effective fusion methods to meet actual needs. Summary of the invention

[0004] Taking into account the limitations of existing methods for solving the problem of infrared and visible light image fusion in circuit board detection applications, the present invention proposes a circuit board infrared and visible light image fusion method based on DTCWT.

[0005] The method of the present invention uses DTCWT to decompose the source image to be fused to obtain low-frequency sub-bands and high-frequency sub-bands of infrared and visible light images; a fusion rule based on two-scale decomposition and saliency detection is used to fuse the low-frequency sub-bands, and a fusion rule based on local energy detection is used to fuse the high-frequency sub-bands; and a DTCWT inverse transform is performed on the fused low-frequency and high-frequency sub-bands to obtain the fusion result of the infrared and visible light images.

[0006] Therefore, the technical solution of the present invention is a method for fusing infrared and visible light images of a circuit board based on DTCWT, which comprises the following steps:

[0007] S1: Perform dual-tree complex wavelet transform (DTCWT) on the input infrared and visible light images to obtain the low-frequency subband D of the infrared and visible light images. IR and D VI , and the corresponding high frequency sub-band and Where k represents the scale of the high-frequency subband, l represents the direction of the high-frequency subband, and the value of l is an integer between [1,6];

[0008] S2: Low frequency subband D for infrared and visible light images IR and D VI Perform two-scale decomposition; Gaussian low-pass filtering is performed on the low-frequency sub-bands of the infrared and visible light images to obtain the corresponding base layer image D IR,B and D VI,B ; Perform differential operation on the low-frequency subband and the base layer to obtain the corresponding detail layer image D IR,D and D VI,D ;

[0009] S3: Perform saliency detection on the low-frequency sub-bands of the infrared and visible light images to obtain the saliency maps S of the low-frequency sub-bands of the infrared and visible light images respectively. IR and S VI ; Based on the results of saliency detection, calculate the detail layer D IR,D and D VI,D The fusion weight graph W IR,D and W VI,D ;

[0010] S4: Use the linear weighting method with preset weight coefficients to fuse the base layer D IR,B and D VI,B , get the base layer fusion image D F,B ; Using the calculated fusion weight map W IR,D and W VI,D Fusion detail layer D IR,D and D VI,D , get the detail layer fusion image D F,D ; Fuse the base layer with image D F,B And the detail layer fusion image D F,D Superposition, to obtain the fusion image D of infrared and visible light low-frequency sub-bands F ;

[0011] S5: Calculate the local energy of the infrared and visible light high-frequency sub-bands and obtain the local energy map of the high-frequency sub-bands and

[0012] S6: Calculate the matching degree between the corresponding high-frequency sub-band images of the infrared and visible light images to obtain a matching degree graph M k,l ;

[0013] S7: Matching degree M between high frequency sub-band images based on infrared and visible light images k,l , and the density distribution of the local energy of the high-frequency sub-band in each direction, the corresponding high-frequency sub-bands are weighted fused at the pixel level to obtain the fused image of the infrared and visible light high-frequency sub-bands

[0014] S8: Using DTCWT to fusion sub-band D of low and high frequencies F and Perform inverse transform and reconstruction to obtain the final output fused image.

[0015] Furthermore, in S3, the calculation method of the saliency map of the low-frequency sub-band of the infrared and visible light images is the absolute value of the difference between the base layer image obtained by Gaussian low-pass filtering and the filtered image obtained by median filtering, and the fusion weight coefficient of the detail layer of the low-frequency sub-band is calculated as follows:

[0016]

[0017] W IR,D (i, j) represents the fusion weight coefficient of the infrared low-frequency sub-band detail layer at position (i, j), W VI,D (i, j) represents the fusion weight coefficient of the visible light low-frequency sub-band detail layer at position (i, j), S IR (i, j) represents the saliency value of the infrared image at position (i, j), S VI (i, j) represents the saliency value of the visible light image at position (i, j).

[0018] Further, in S4, the base layer fused image D F , B is composed of the base layer D of the low-frequency sub-bands of the infrared and visible light images IR,B and D VI,B The linear weighted calculation yields:

[0019] D F,B =0.9*D IR,B +0.1*D VI,B

[0020] Detail layer fused image D F,D The detail layer D of the low-frequency sub-bands of infrared and visible light images IR,D and D VI,D and its fusion weight graph W IR,D and W VI,D The calculation results are:

[0021] D F,D =W IR,D *D IR,D +W VI,D *D VI,D .

[0022] Furthermore, in S5, the local energy of the infrared and visible high-frequency sub-bands and The calculation is as follows:

[0023]

[0024] Where w represents the 5×5 neighborhood centered at (i, j) in the image. represents the infrared high-frequency subband energy at position (i, j) in the k-scale and l-direction. represents the high-frequency subband energy of visible light at position (i, j) in the k-scale and l-direction. Indicates the position (i) at scale k, direction l ′ ,j ′ ) is the pixel value of the infrared high-frequency sub-band image at, Indicates the position (i) at scale k, direction l ′ ,j ′ ) is the pixel value of the high-frequency sub-band image of visible light;

[0025] Furthermore, in S6, the matching degree between the infrared and visible light high frequency sub-bands to be fused is calculated as follows:

[0026]

[0027] M k,l (i, j) represents the matching degree of the infrared and visible light high-frequency sub-bands at the position (i, j) in the k scale and the l direction;

[0028] Furthermore, in S7, the high-frequency sub-bands of infrared and visible light are fused in the following way:

[0029] The matching degree M between the high-frequency sub-bands to be fused k,l (i,j)<α, where α is the preset threshold. but like but Represents the pixel value of the fused image at position (i, j);

[0030] The matching degree M between the high-frequency sub-bands to be fused k,l When (i,j)≥α, if and but

[0031]

[0032] like or but

[0033]

[0034] The present invention proposes a circuit board infrared and visible light image fusion method based on DTCWT. The method uses DTCWT to decompose infrared and visible light source images into low-frequency and high-frequency sub-bands, and adopts appropriate fusion rules to fuse the corresponding sub-bands, and then obtains the fused image through DTCWT inverse transformation. Different from the general infrared and visible light image fusion method, in order to improve the visualization effect and readability of infrared images, this method uses pseudo-color coded infrared images and as input, and fuses the visible light color image and infrared pseudo-color image on three channels in RGB space. Among them, the low-frequency part mainly retains the information from the infrared image, so that the low-frequency background part basically presents a pseudo-color effect reflecting the distribution of hot areas; the high-frequency part adopts appropriate fusion criteria to retain the edge detail information in the infrared and visible light images. Although this fusion method abandons most of the color information in the visible light image, the obtained fused image is more in line with human visual habits and actual engineering needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of the image fusion framework based on DTCWT.

[0036] Figure 2 This is the low-frequency sub-band fusion flow chart.

[0037] Figure 3 This is the high frequency sub-band fusion flow chart.

[0038] Figure 4 Infrared source image.

[0039] Figure 5 is the visible light source image.

[0040] Figure 6 To fuse the images. DETAILED DESCRIPTION

[0041] The present invention is described in detail below with reference to the accompanying drawings.

[0042] This embodiment provides a circuit board infrared and visible light image fusion method based on dual-tree complex wavelet transform (DTCWT). The basic framework is as follows: Figure 1 As shown in the figure, the fusion methods of low-frequency and high-frequency sub-bands are as follows: Figure 2 and Figure 3 The specific steps of this method are as follows:

[0043] S1, source image DTCWT decomposition. Use DTCWT to perform a two-level decomposition of the input infrared and visible light source images. Figure 4 and Figure 5 are the infrared and visible light source images used in this example. The low-frequency sub-bands of the infrared and visible light images obtained after decomposition are denoted as D IRand D VI , and the corresponding high-frequency sub-bands are recorded as and (where k represents the scale of the high-frequency sub-band, l represents the direction of the high-frequency sub-band, and the value of l is an integer between [1, 6]).

[0044] S2, two-scale decomposition of low-frequency subband.

[0045] S2-1, base layer extraction. Low frequency subband D of infrared and visible light images IR and D VI Perform Gaussian low-pass filtering to obtain the base layer image D IR,B and D VI,B .

[0046] S2-2, detail layer extraction. The low frequency subband D IR and D VI The corresponding base layer D IR,B and D VI,B Perform differential operation to obtain the detail layer image D IR,D and D VI,D .

[0047] S3, low frequency sub-band saliency detection.

[0048] S3-1, saliency map calculation. Perform saliency detection on the low-frequency sub-bands of the infrared and visible light images to obtain saliency maps S of the low-frequency sub-bands of the infrared and visible light images respectively. IR and S VI The significance is calculated as follows:

[0049]

[0050] Where D IR,M and D VI,M These are the results of low-frequency sub-band median filtering for infrared and visible light, respectively.

[0051] S3-2, fusion weight map calculation. Based on the results of saliency detection, the detail layer D is calculated. IR,D and D VI,D The fusion weight graph W IR,D and W VI,D :

[0052]

[0053] S4, low frequency sub-band fusion.

[0054] S4-1, base layer fusion. The base layer D is fused using a linear weighted method with a preset weight coefficient. IR,B and D VI,B , get the base layer fusion image D F,B :

[0055] D F,B =0.9*D IR,B +0.1*D VI,B

[0056] S4-2, detail layer fusion. Using the calculated fusion weight map W IR,D and W VI,D Fusion detail layer D IR,D and D VI ,D , get the detail layer fusion image D F,D :

[0057] D F,D =W IR,D *D IR,D +W VI,D *D VI,D

[0058] S4-3, the base layer and the detail layer are superimposed. The base layer is fused with image D F,B And the detail layer fusion image D F,D Superposition, to obtain the fusion image D of infrared and visible light low-frequency sub-bands F :

[0059] D F =D F,B +D F,D

[0060] S5, high frequency sub-band local energy calculation. Calculate the local energy map of each high frequency sub-band of infrared and visible light images respectively. and

[0061]

[0062] Where w represents the 5×5 neighborhood centered at (i, j) in the image.

[0063] S6, high frequency sub-band matching calculation. Calculate the matching degree M between the high frequency sub-bands corresponding to the infrared and visible light images. k,l :

[0064]

[0065] S7, high-frequency sub-band fusion. The high-frequency sub-band matching threshold α is preset to 0.75. Different fusion rules are used for the two situations where the matching degree between the high-frequency sub-bands to be fused is lower than or not lower than the threshold α. The specific introduction is as follows:

[0066] C1, the matching degree between the high-frequency sub-bands to be fused is lower than α, that is, M k,lWhen (i,j)<α, the pixel value of the fused image is the larger pixel value of the two high-frequency sub-bands to be fused at that point:

[0067]

[0068] C2, the matching degree between the high-frequency sub-bands to be fused is not less than α, that is, M k,l When (i, j) ≥ α, the pixel value of the fused image depends on the local energy of the two high-frequency sub-bands to be fused at that point and their density distribution in all directions.

[0069] C2-1, if and The pixel value of the fused image is calculated as follows:

[0070]

[0071] C2-2, if or The pixel value of the fused image is calculated as follows:

[0072]

[0073] S8, DTCWT inverse transform reconstruction. Use DTCWT to transform the fused low-frequency subband D F and high frequency subband Perform inverse transform reconstruction to obtain the final output fused image, such as Figure 6 shown.

[0074] The present invention can more effectively detect faults and defects on circuit boards by fusing infrared images with visible light images. Infrared images can show temperature anomalies, while visible light images can show the detailed structure of circuit boards. The combination of the two can more accurately identify the location and type of faults, thereby improving the efficiency and accuracy of fault detection.

[0075] The present invention performs image fusion based on dual-tree complex wavelet transform (DTCWT). The obtained fused image can present a pseudo-color effect reflecting the temperature zone distribution of the circuit board. At the same time, the edge and detail information can be retained and enhanced, so that the fused image is more in line with human visual habits, which is convenient for engineers and technicians to inspect and analyze the circuit board.

[0076] The present invention avoids unnatural color transfer in the fusion process of infrared and visible light images. In addition, compared with other multi-scale transforms, DTCWT takes into account both detail capture capability and computational efficiency. Therefore, this method can meet the high efficiency requirements in practical engineering applications.

[0077] The embodiment illustrates how to decompose, fuse and reconstruct infrared and visible light images through dual-tree complex wavelet transform, thereby achieving high-quality image fusion, which is suitable for application scenarios such as circuit board detection that require multi-source information fusion. The above is only a preferred embodiment of the present invention, and any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for fusion of infrared and visible light images of circuit boards based on DTCWT, the method comprising the following steps: S1: Perform dual-tree complex wavelet transform on the input infrared and visible light source images to obtain the low-frequency subband D of the infrared and visible light images. IR and D VI , and the corresponding high frequency sub-band and Where k represents the scale of the high-frequency subband, l represents the direction of the high-frequency subband, and the value of l is an integer between [1,6]; S2: Low frequency subband D for infrared and visible light images IR and D VI Perform two-scale decomposition; Gaussian low-pass filtering is performed on the low-frequency sub-bands of the infrared and visible light images to obtain the corresponding base layer image D IR,B and D VI,B ; Perform differential operation on the low-frequency subband and the base layer to obtain the corresponding detail layer image D IR,D and D VI,D ; S3: Perform saliency detection on the low-frequency sub-bands of the infrared and visible light images to obtain the saliency maps S of the low-frequency sub-bands of the infrared and visible light images respectively. IR and S VI ; Based on the results of saliency detection, calculate the detail layer D IR,D and D VI,D The fusion weight graph W IR ,D and W VI,D ; S4: Use the linear weighting method with preset weight coefficients to fuse the base layer D IR,B and D VI,B , get the base layer fusion image D F ,B ; Using the calculated fusion weight map W IR,D and W VI,D Fusion detail layer D IR,D and D VI,D , get the detail layer fusion image D F ,D ; Fuse the base layer with image D F,B And the detail layer fusion image D F,D Superposition, to obtain the fusion image D of infrared and visible light low-frequency sub-bands F ; S5: Calculate the local energy of the infrared and visible light high-frequency sub-bands and obtain the local energy map of the high-frequency sub-bands and S6: Calculate the matching degree between the corresponding high-frequency sub-band images of the infrared and visible light images to obtain a matching degree graph M k,l ; S7: Matching degree M between high frequency sub-band images based on infrared and visible light images k,l , and the density distribution of the local energy of the high-frequency sub-band in each direction, the corresponding high-frequency sub-bands are weighted fused at the pixel level to obtain the fused image of the infrared and visible light high-frequency sub-bands S8: Using DTCWT to fusion sub-band D of low and high frequencies F and Perform inverse transform and reconstruction to obtain the final output fused image.

2. The DTCWT-based circuit board infrared and visible light image fusion method as claimed in claim 1, characterized in that: In S3, the calculation method of the saliency map of the low-frequency sub-band of the infrared and visible light images is the absolute value of the difference between the base layer image obtained by Gaussian low-pass filtering and the filtered image obtained by median filtering. The fusion weight coefficient of the detail layer of the low-frequency sub-band is calculated as follows: W IR,D (i, j) represents the fusion weight coefficient of the infrared low-frequency sub-band detail layer at position (i, j), W VI,D (i, j) represents the fusion weight coefficient of the visible light low-frequency sub-band detail layer at position (i, j), S IR (i, j) represents the saliency value of the infrared image at position (i, j), S VI (i, j) represents the saliency value of the visible light image at position (i, j).

3. The DTCWT-based circuit board infrared and visible light image fusion method as claimed in claim 1, characterized in that: In S4, the base layer fused image D F,B The base layer D is composed of low-frequency sub-bands of infrared and visible light images. IR,B and D VI,B The linear weighted calculation yields: D F,B =0.9*D IR,B +0.1*D VI,B Detail layer fused image D F,D The detail layer D of the low-frequency sub-bands of infrared and visible light images IR,D and D VI,D and its fusion weight graph W IR,D and W VI,D The calculation results are: D F,D =W IR,D *D IR,D +W VI,D *D VI,D 。 4. The DTCWT-based circuit board infrared and visible light image fusion method as claimed in claim 1, characterized in that: In S5, the local energy of the infrared and visible high-frequency sub-bands and The calculation is as follows: Where w represents the 5×5 neighborhood centered at (i, j) in the image. represents the infrared high-frequency subband energy at position (i, j) in the k-scale and l-direction. represents the high-frequency subband energy of visible light at position (i, j) in the k-scale and l-direction. Indicates the position (i) at scale k, direction l ′ ,j ′ ) is the pixel value of the infrared high-frequency sub-band image at, Indicates the position (i) at scale k, direction l ′ ,j ′ ) is the pixel value of the high-frequency sub-band image of visible light at .

5. The DTCWT-based circuit board infrared and visible light image fusion method as claimed in claim 1, characterized in that: In S6, the matching degree between the infrared and visible high-frequency sub-bands to be fused is calculated as follows: M k,l (i, j) represents the matching degree of the infrared and visible light high-frequency sub-bands at the position (i, j) in the k scale and the l direction.

6. The DTCWT-based circuit board infrared and visible light image fusion method according to claim 1, characterized in that: In S7, the high-frequency sub-bands of infrared and visible light are fused in the following way: The matching degree M between the high-frequency sub-bands to be fused k,l (i,j)<α, where α is the preset threshold. but like but Represents the pixel value of the fused image at position (i, j); The matching degree M between the high-frequency sub-bands to be fused k,l When (i,j)≥α, if and but like or but

Citation Information

Patent Citations

  • Visible light and infrared images fusion method based on NSST and system thereof

    CN104504673A

  • Visible light and infrared image fusion algorithm based on NSCT domain bottom layer visual features

    CN105719263A

  • Infrared and visible image fusion method and system based on visual saliency

    CN109447909A

  • Image fusion method and system based on frequency band coefficient decomposition

    CN114092401A

  • Method and system of increasing spatial resolution of multi-dimensional optical imagery using sensor's intrinsic keystone

    WO2012106797A1

Cited By

  • Multi-platform-oriented large PCB vector diagram cloud intelligent processing method

    CN120953082A