Infrared thermal imaging detection device and detection method for electronic component defects

Abstract

The present invention relates to an infrared thermal imaging detection device and method for electronic component defects, comprising a laser heater, an infrared image sensor, and an optical module sequentially arranged above a conveyor belt, wherein the laser heater heats the electronic components on the conveyor belt, and the infrared image sensor and the optical module photograph the electronic components that are heated more. The present invention can not only obtain defect features such as scratches, chipped corners, and cracks through infrared thermal imaging, but also capture dark crack features, and perform target detection of image defects through neural networks, thereby greatly improving detection efficiency. Compared with visible light images, the infrared thermal imaging images used are less interfered by noise, dust, and stains, and can suppress the surface silk-screen texture to a certain extent, avoiding the interference of noise and silk-screen on defect detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic component detection, and specifically relates to an infrared thermal imaging detection device and a detection method for electronic component defects, mainly aimed at the electronic component production industry, and is a quality detection method for surface defects and internal defects of electronic component products. Background Art

[0002] In the field of electronic component production and processing, automated assembly lines have basically been realized. Taking precision wound inductors as an example, an electronic component probably goes through the following processing steps: (1) The magnetic core and other materials are sent into the assembly line from the equipment entrance, and the coils are wound on the precision winding machine in turn via the conveyor belt; (2) Then it is sent to the laser de-painting equipment via the conveyor belt to remove the surface coating of the enameled wire; (3) Then it is sent to the automatic pin welding table via the conveyor belt to complete the welding of the de-painted wire head and the magnetic core solder joint; (4) Then it is sent to the laser screen printer via the conveyor belt to complete the etching of the screen printing; (5) Finally, it is sent to the quality inspection station via the conveyor belt, and the inspection process is completed by CCD equipment or manual to confirm that it is intact.

[0003] A precision wound inductor has to go through more than 5 mechanical conveying processes on conveyor belts or fixtures during the processing. Mechanical conveying has high efficiency, but its disadvantage is also obvious, which is that it will cause physical damage to components and affect the product's indicators, accuracy and performance. The formula for calculating inductance is:

[0004]

[0005] Ac is the cross-sectional area of ​​the core structure, lc is the length of the closed core, N is the number of turns of the coil, μ0 is the vacuum permeability, and μr is the relative permeability of the core inside the coil. It can be seen that when there are defects in the core, Ac is the cross-sectional area of ​​the core structure, lc is the length of the closed core, and μr is the relative permeability of the core inside the coil. This will cause the actual inductance value L to be inaccurate. The quality factor Q of the inductor is:

[0006]

[0007] Ac is the cross-sectional area of ​​the conductor, w is the angular frequency, 2(w+l) is the length of each turn of the coil, and p is the conductor material.

[0008] During the production process, if the mechanical movement squeezes and pulls the enameled coil, it will cause the coil to deform and stretch, and the solder joints to loosen. If the length of each turn of the coil is too uneven, the quality factor will not meet the product requirements. It can be seen that calculating tiny device defects may lead to: (1) The basic parameters of the product cannot meet the requirements. (2) The uniformity of the product indicators is not good. (3) There are also potential problems with the product's life and power consumption. Therefore, general component processing equipment will have a dedicated detection unit.

[0009] The current electronic component surface defect detection industry uses detection equipment that identifies simple defects and manual assisted detection methods. This type of method has a low degree of automation, low efficiency, and very high production costs. In addition, the current equipment has limited detection capabilities and is unable to identify complex defects such as dark lines, cracks in background noise, and internal injuries. Summary of the invention

[0010] For precision electronic components such as inductors to be inspected, the present invention provides an infrared thermal imaging detection device and method for electronic component defects. Compared with conventional methods for detecting deformation, concentricity, parallelism, poor appearance, size and other conventional tests, this method mainly detects scratches, broken corners, cracks, dark cracks and other defects that are difficult to detect. It also has the advantages of noise suppression and is not affected by dust, mosquitoes, oil and silk screen printing. Dark cracks, complex noise background, silk screen interference, etc. are the key technical bottlenecks for defect detection of electronic components at present, which have a great impact on the accuracy of defect detection.

[0011] This invention uses infrared heating technology and neural networks to improve the detection function of scratches, chipping, cracks, dark cracks and other defects that are difficult to detect, and improve the defect detection rate of electronic components. Precision electronic components detected by this method do not need manual inspection and meet product yield requirements, which can improve production efficiency and reduce production costs.

[0012] The specific technical solutions are:

[0013] An infrared thermal imaging detection device for electronic component defects includes a laser heater, an infrared image sensor and an optical module which are sequentially arranged above a conveyor belt. The laser heater heats the electronic components on the conveyor belt, and the infrared image sensor and the optical module photograph the electronic components that are heated the most.

[0014] The electronic components are heated by the laser heater on the conveyor belt and arrive under the infrared image sensor and the optical module at a speed v. The infrared image sensor takes pictures at a time interval T. The shooting interval is s, the component size is u, and T is less than (s-2u) / v.

[0015] In the images periodically collected by the infrared image sensor, the edge of the electronic component has a high temperature due to the large height change during laser heating. According to the infrared temperature response characteristics, the edge part is clearly distinguished from other areas. The contour of the electronic component is extracted using the Canny operator, and the contour is segmented by a rectangular envelope to obtain n primary sub-images;

[0016] Rapidly analyze defects of electronic components in a frame of image through parallel processing.

[0017] The sub-graph processing method comprises the following steps:

[0018] S1. The sizes of n primary sub-images are different, so some scaling is needed to convert the sub-image size to w*h;

[0019] S2, remove salt and pepper noise by median filtering;

[0020] S3, highlighting edges and defect patterns through a high-pass filter;

[0021] S4. Reduce the number of channels of U-NET by 4 times, reduce the model parameters to less than 2MB, and obtain a lightweight U-NET network;

[0022] S5, send the preprocessed image to the lightweight U-NET network;

[0023] S6, perform rectangular envelope segmentation on the sub-graph segmented by U-NET to obtain a secondary sub-graph;

[0024] S7, scaling the secondary subgraph to obtain a secondary subgraph of size k*j;

[0025] S8, sending the secondary sub-image into the CNN classification network to classify the region;

[0026] S9. Arrange all secondary classification results, and determine whether there is a secondary sub-graph classified as defective in the primary sub-graph; if not, the electronic component corresponding to the primary sub-graph is not defective, otherwise, list the defect list of all secondary sub-graphs.

[0027] The method of constructing the U-NET network is as follows:

[0028] (1) Collect 1,000 infrared thermal imaging images of various defects and normal workpieces;

[0029] (2) Manually mark the defect location, size and defect category;

[0030] (3) Increase the number of samples of each type to more than 2,000 through data enhancement methods;

[0031] (4) Verify the number of samples in each category and use the enhancement method in 3 to ensure that the number of images in each category is close;

[0032] (5) Scale all images and convert the sub-image size to w*h;

[0033] (6) Remove salt and pepper noise by median filtering;

[0034] (7) Highlight edges and defect patterns through a high-pass filter;

[0035] (8) Use the preprocessed images as training sets and validation sets to train a lightweight U-NET network;

[0036] (9) After the training is completed, save the training model;

[0037] (10) Loading the training model for device defect detection;

[0038] The defect categories include scratches, chipped corners, cracks, dark cracks, silk screen, normal, and other seven categories.

[0039] The data enhancement methods include rotation, scaling, grayscale adjustment, noise addition, and random cropping.

[0040] The technical solution provided by the present invention has the following technical effects:

[0041] (1) This invention uses infrared thermal imaging to not only obtain defect features such as scratches, chipped corners, and cracks, but also capture dark crack features, and perform target detection of image defects through neural networks, greatly improving detection efficiency.

[0042] (2) Compared with visible light images, the infrared thermal imaging images used in this invention are less disturbed by noise, dust, and stains, and can suppress the surface silk-screen texture to a certain extent, thereby avoiding the interference of noise and silk-screen on defect detection.

[0043] (3) The present invention can not only distinguish product defects, but also identify defect types directly based on defect detection through a multi-classification neural network, which is convenient for subsequent defect root cause analysis.

[0044] (4) The invention can segment multiple component sub-images using only an infrared thermal imaging camera, segment the sub-images in parallel, and then send the sub-images to a lightweight semantic segmentation network for segmentation, and finally send them to a CNN network for classification;

[0045] (5) The invention is beneficial to infrared thermal imaging, can efficiently extract the edges of electronic components, and is used for electronic component image segmentation with low computational complexity;

[0046] (6) The invention processes multiple component sub-graphs in parallel, ensuring real-time processing and improving production efficiency;

[0047] (7) The invention uses a lightweight improved UNET network and a simple CNN classification network, which can utilize the segmentation accuracy of the neural network while also ensuring the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the structure of the device of the present invention;

[0049] Figure 2 It is a schematic diagram of the detection process of the present invention;

[0050] Figure 3 It is a schematic diagram of the U-NET network connection of the present invention. DETAILED DESCRIPTION

[0051] The specific technical solutions for implementing the present invention are described in detail.

[0052] like Figure 1 As shown, an infrared thermal imaging detection device for electronic component defects includes a laser heater 2, an infrared image sensor and an optical module 1 which are sequentially arranged above a conveyor belt 4. The laser heater 2 heats the electronic components 3 on the conveyor belt 4, and the infrared image sensor and the optical module 1 take pictures of the electronic components 3 that are heated the most.

[0053] The present invention includes two main links, the sub-image segmentation of electronic components 3 and the segmentation and classification of component surface defects. The infrared image containing n electronic components 3 on the surface heated by laser is collected by infrared sensor. The electronic components 3 are vibrated by the vibration plate before entering the heating zone and sparsely distributed on the conveyor belt 4. After the infrared image is collected, the image is semantically segmented and the infrared image is divided into n infrared sub-images. Defects are located and classified in parallel for the n infrared sub-images, that is, defect target identification is completed. If there is a defective area, the defect type and defect location are recorded.

[0054] like Figure 2 As shown, the specific process is:

[0055] The electronic component 3 is heated by the laser heater 2 on the conveyor belt 4, and after heating, it reaches the infrared image sensor and the optical module 1 at a speed v. The infrared image sensor takes pictures at a time interval T. The shooting interval is s, the component size is u, and T is less than (s-2u) / v.

[0056] In the images periodically collected by the infrared image sensor, the edge of the electronic component has a high temperature due to the large height change during laser heating. According to the infrared temperature response characteristics, the edge part is clearly distinguished from other areas. The contour of the electronic component is extracted using the Canny operator, and the contour is segmented by a rectangular envelope to obtain n primary sub-images;

[0057] Rapidly analyze defects of electronic components in a frame of image through parallel processing.

[0058] The sub-graph processing method comprises the following steps:

[0059] S1. The sizes of n primary sub-images are different, so some scaling is needed to convert the sub-image size to w*h;

[0060] S2, remove salt and pepper noise by median filtering;

[0061] S3, highlighting edges and defect patterns through a high-pass filter;

[0062] S4. Reduce the number of channels of U-NET by 4 times, reduce the model parameters to less than 2MB, and obtain a lightweight U-NET network;

[0063] S5, send the preprocessed image to the lightweight U-NET network;

[0064] S6, perform rectangular envelope segmentation on the sub-graph segmented by U-NET to obtain a secondary sub-graph;

[0065] S7, scaling the secondary subgraph to obtain a secondary subgraph of size k*j;

[0066] S8, sending the secondary sub-image into the CNN classification network to classify the region;

[0067] S9. Arrange all secondary classification results, and determine whether there is a secondary sub-graph classified as defective in the primary sub-graph; if not, the electronic component corresponding to the primary sub-graph is not defective, otherwise, list the defect list of all secondary sub-graphs.

[0068] Since the infrared component surface images are very similar to many medical images, U-NET is selected as the semantic segmentation network in this method. First, the infrared images of the electronic component surface have relatively simple image semantics and relatively fixed structures, mainly edges, cracks, scratches, cracks, dark cracks and some silk-screen information; secondly, for a certain type of electronic components, in order to reduce development costs, the amount of infrared image data of the component surface is also small, and the U-NET model is relatively simple and has a small number of parameters, which can prevent problems such as overfitting; thirdly, when the components in the infrared images are heated, there are temperature differences between different components. For electronic components in the same image or in different images, the image color distribution and depth are different, and they are multimodal like medical images; in the application of defect detection in electronic components, defects need to be classified and analyzed later. U-NET has good interpretability and can help analyze the causes of defects.

[0069] However, the parameters of U-NET are still too large for industrial applications. In order to reduce the parameters, the present invention makes appropriate improvements to U-NET and obtains a network for target defect segmentation of electronic components. The following improvements are made based on the characteristics of infrared thermal imaging images of electronic components.

[0070] 1. Usually, the resolution of infrared thermal imaging is low. The resolution of infrared sub-images after segmentation according to electronic components is generally

[0071] The image elements are relatively simple, so the original input image size of 572*572 in U-NET is compressed to the original 316*316. In order to ensure that the input image size is 316*316, the mirror symmetry supplement method is used to obtain.

[0072] 2. If Figure 3 As shown in the figure, reducing the number of channels of the convolution kernel to one-fourth of the original can compress the number of parameters by three-quarters.

[0073] 3. Use LeakyReLU as the activation function in the convolution operation, where a is the slope, to avoid the gradient vanishing problem when x<0.

[0074] LeakyReLU(x)=max(0,x)+a*min(0,x)

[0075] The operation between the dotted boxes 31 indicates the maximum pooling of size 2*2;

[0076] The operation between the dotted boxes 32 indicates that the size of the upsampling convolution is 2*2;

[0077] 33Dark cells and light boxes indicate convolution kernels;

[0078] The operation between the boxes marked inside the dotted box 34 is the convolution operation with a convolution kernel size of 3*3 and an activation function of LeakyRelu;

[0079] 35 indicates the copying and cropping operation of the dark square on the left. For example, in the top network, the output of the left convolutional layer with a size of 312*312 needs to be copied and cropped to 136*136 and merged with the input of the right convolutional layer. The same operation is performed on other layers.

[0080] The improved U-NET for defect recognition is lightweight and can achieve good results with a small number of training samples. It is very useful in scenarios where multiple electronic components need sample training. Due to sub-image segmentation, the size of the image is greatly reduced, which further reduces the network parameters. The overall parameter amount is about 1M, which is very lightweight.

[0081] The method of building the U-NET model is:

[0082] (1) Collect 1,000 infrared thermal imaging images of various defects and normal workpieces;

[0083] (2) Manually mark the defect location, size and defect category;

[0084] (3) Increase the number of samples of each type to more than 2,000 through data enhancement methods;

[0085] (4) Verify the number of samples in each category and use the enhancement method in 3 to ensure that the number of images in each category is close;

[0086] (5) Scale all images and convert the sub-image size to w*h;

[0087] (6) Remove salt and pepper noise by median filtering;

[0088] (7) Highlight edges and defect patterns through a high-pass filter;

[0089] (8) Use the preprocessed images as training sets and validation sets to train a lightweight U-NET network;

[0090] (9) After the training is completed, save the training model;

[0091] (10) Load the training model for device defect detection.

Claims

1. A detection method of an infrared thermal imaging detection device for electronic component defects, wherein the infrared thermal imaging detection device for electronic component defects is characterized in that: It comprises a laser heater (2) and an infrared image sensor and an optical module (1) which are sequentially arranged above a conveyor belt (4), wherein the laser heater (2) heats the electronic components (3) on the conveyor belt (4), and the infrared image sensor and the optical module (1) photograph the electronic components (3) that are heated the most; Characterized in that the detection method comprises the following steps: The electronic component (3) is heated by the laser heater (2) on the conveyor belt (4), and after heating, it reaches the bottom of the infrared image sensor and the optical module (1) at a speed v. The infrared image sensor takes pictures at a time interval T. The shooting interval is s, the component size is u, and T is less than (s-2u) / v; In the images periodically collected by the infrared image sensor, the edge of the electronic component has a high temperature due to the large height change during laser heating. According to the infrared temperature response characteristics, the edge part is clearly distinguished from other areas. The contour of the electronic component is extracted using the Canny operator, and the contour is segmented by a rectangular envelope to obtain n primary sub-images; Rapidly analyze defects of electronic components in a frame of image through parallel processing.

2. The detection method of the infrared thermal imaging detection device for electronic component defects according to claim 1, characterized in that: The parallel processing method of the primary subgraph comprises the following steps: S1. The sizes of n primary sub-images are different, so some scaling is needed to convert the sub-image size to w*h; S2, remove salt and pepper noise by median filtering; S3, highlighting edges and defect patterns through a high-pass filter; S4. Reduce the number of channels of U-NET by 4 times, reduce the model parameters to less than 2MB, and obtain a lightweight U-NET network; S5, send the preprocessed image to the lightweight U-NET network; S6, perform rectangular envelope segmentation on the sub-graph segmented by U-NET to obtain a secondary sub-graph; S7, scaling the secondary subgraph to obtain a secondary subgraph of size k*j; S8, sending the secondary sub-image into the CNN classification network to classify the region; S9. Arrange all secondary classification results, and determine whether there is a secondary sub-graph classified as defective in the primary sub-graph; if not, the electronic component corresponding to the primary sub-graph is not defective, otherwise, list the defect list of all secondary sub-graphs.

3. The detection method of the infrared thermal imaging detection device for electronic component defects according to claim 2, characterized in that: The construction method of the U-NET network is: (1) Collect 1,000 infrared thermal imaging images of various defects and normal workpieces; (2) Manually mark the defect location, size and defect category; (3) Increase the number of samples of each type to more than 2,000 through data enhancement methods; (4) Verify the number of samples in each category and use the enhancement method in 3 to ensure that the number of images in each category is close; (5) Scale all images and convert the sub-image size to w*h; (6) Remove salt and pepper noise through median filtering; (7) Highlight edges and defect patterns through a high-pass filter; (8) Use the preprocessed images as training sets and validation sets to train a lightweight U-NET network; (9) Save the training model after training is completed; (10) Load the trained model for device defect detection.

4. The detection method of the infrared thermal imaging detection device for electronic component defects according to claim 3 is characterized in that: The categories of defects include scratches, chipped corners, cracks, dark cracks, silk screen, normal, and other seven categories.

5. The detection method of the infrared thermal imaging detection device for electronic component defects according to claim 3, characterized in that: The data enhancement methods include rotation, scaling, grayscale adjustment, noise addition, and random cropping.

Images