PCB board defect detection method and device

Abstract

The application discloses a PCB defect detection method and device, wherein the method comprises the following steps: obtaining a to-be-tested PCB image and a standard PCB image; comparing the to-be-tested PCB image with the standard PCB image to determine a defect position local image; inputting the defect position local image into a trained improved GoogleNet network model to identify a defect type of the PCB, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving an Inception structure of a GoogleNet network. The application can detect defects of the PCB, improve detection efficiency and accuracy, and reduce maintenance cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, in particular to a PCB defect detection method and device. BACKGROUND

[0002] With the increasing precision and density of PCBs, some defects in the lines of the PCBs are increasingly difficult to find. In the prior art, the PCB defect detection is usually performed by manual detection, traditional image processing or automatic optical detection. However, in the manual detection, the accuracy is easily affected by the subjectivity of workers, the speed is slow, and the subjective error is easily caused by long-time work. In the traditional image processing, the accuracy is easily affected by multiple errors, and the image is easily affected by noise pollution, which cannot guarantee the accuracy. In the automatic optical detection, although the accuracy is high, the corresponding algorithm process needs to be designed for different types of PCBs, i.e. different defects, which is complex and the equipment is too expensive, so the maintenance cost is high.

[0003] Therefore, there is an urgent need for a PCB defect detection scheme which can overcome the above problems. SUMMARY

[0004] The embodiment of the present application provides a PCB defect detection method, which is used for PCB defect detection, improves detection efficiency and accuracy, and reduces maintenance cost. The method comprises the following steps:

[0005] obtaining a to-be-tested PCB image and a standard PCB image;

[0006] comparing the to-be-tested PCB image and the standard PCB image to determine a defect position local image;

[0007] inputting the defect position local image into a trained improved GoogleNet network model to identify the defect type of the PCB, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving the Inception structure of the GoogleNet network.

[0008] The embodiment of the present application provides a PCB defect detection device, which is used for PCB defect detection, improves detection efficiency and accuracy, and reduces maintenance cost. The device comprises the following steps:

[0009] an image obtaining module, configured to obtain a to-be-tested PCB image and a standard PCB image;

[0010] a position determining module, configured to compare the to-be-tested PCB image and the standard PCB image to determine a defect position local image;

[0011] The defect identification module is configured to input the defect position local image into a trained improved GoogleNet network model to identify a defect type of the PCB, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving an Inception structure of the GoogleNet network.

[0012] The embodiment of the present application also provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the PCB defect detection method when executing the computer program.

[0013] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program for implementing the PCB defect detection method.

[0014] Compared with the technical scheme for detecting the defects of the PCB by using manual detection, traditional image processing and automatic optical detection in the prior art, the embodiment of the present application obtains a to-be-tested PCB image and a standard PCB image, compares the to-be-tested PCB image with the standard PCB image to determine a defect position local image, inputs the defect position local image into a trained improved GoogleNet network model to identify a defect type of the PCB, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving an Inception structure of the GoogleNet network. The embodiment of the present application does not need to perform manual detection and design different process algorithms for different types of PCBs, but only needs to improve the Inception structure of the GoogleNet network to obtain an improved GoogleNet network model, train the improved GoogleNet network model according to historical defect position local images, input the defect position local image into the trained improved GoogleNet network model after comparing the to-be-tested PCB image with the standard PCB image to determine the defect position local image, and thus the detection of the defects of the PCB can be realized, the problem of difficult defect detection caused by miniaturization, high precision and high density of the PCB is overcome, the detection efficiency and accuracy are improved, and the maintenance cost is effectively reduced. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only need to be some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort. In the drawings:

[0016] Figure 1 The schematic diagram of the PCB defect detection method in the embodiment of the present application is shown in the figure.

[0017] Figure 2 The schematic diagram of another PCB defect detection method in the embodiment of the present application is shown in the figure.

[0018] Figure 3 The schematic diagram of the comparison method between the image of the PCB to be tested and the image of the standard PCB in the embodiment of the present application is shown in the figure.

[0019] Figure 4 The defect binary image in the specific embodiment of the present application is shown in the figure.

[0020] Figures 5-6 The defect position local image in the specific embodiment of the present application is shown in the figure.

[0021] Figure 7 The schematic diagram of the improved GoogleNet network model training method in the embodiment of the present application is shown in the figure.

[0022] Figure 8 The schematic diagram of the improved method of the Inception structure of the GoogleNet network in the embodiment of the present application is shown in the figure.

[0023] Figure 9 The structure diagram of the improved GoogleNet network model in the embodiment of the present application is shown in the figure.

[0024] Figure 10 The structure diagram of the PCB defect detection device in the embodiment of the present application is shown in the figure.

[0025] Figure 11 The structure diagram of the computer device in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0026] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only need to be some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort. In the drawings:

[0027] In order to perform PCB defect detection, improve detection efficiency and accuracy, and reduce maintenance cost, the embodiment of the present application provides a PCB defect detection method, as shown in Figure 1As shown, the method can comprise:

[0028] Step 101, obtaining a to-be-tested PCB board image and a standard PCB board image;

[0029] Step 102, comparing the to-be-tested PCB board image and the standard PCB board image to determine a defect position local image;

[0030] Step 103, inputting the defect position local image into a trained improved GoogleNet network model to identify a defect type of the PCB board, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving an Inception structure of a GoogleNet network.

[0031] As shown, Figure 1 As shown, compared with the technical scheme of the prior art that detects a PCB board defect by using manual detection, traditional image processing, and automatic optical detection, the embodiment of the present application obtains a to-be-tested PCB board image and a standard PCB board image, compares the to-be-tested PCB board image and the standard PCB board image to determine a defect position local image, and inputs the defect position local image into a trained improved GoogleNet network model to identify a defect type of the PCB board, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving an Inception structure of a GoogleNet network. The embodiment of the present application does not need to perform manual detection or design different process algorithms for different types of PCB boards, but only needs to improve the Inception structure of the GoogleNet network to obtain an improved GoogleNet network model, train the improved GoogleNet network model according to historical defect position local images, and then compare a to-be-tested PCB board image and a standard PCB board image to determine a defect position local image and input the defect position local image into the trained improved GoogleNet network model, so that the detection of the PCB board defect is realized, the problem of difficulty in defect detection caused by miniaturization, high precision, and high density of the PCB board is overcome, the detection efficiency and accuracy are improved, and the maintenance cost is effectively reduced.

[0032] In the embodiment, a to-be-tested PCB board image and a standard PCB board image are obtained, and the to-be-tested PCB board image and the standard PCB board image are compared to determine a defect position local image.

[0033] As shown, Figure 2 The PCB board defect detection method further comprises:

[0034] Step 201, median filtering processing is performed on the to-be-tested PCB image.

[0035] Step 202, Laplacian operator is used to perform sharpening processing on the to-be-tested PCB image after the median filtering processing.

[0036] The to-be-tested PCB image and the standard PCB image are compared, including comparing the to-be-tested PCB image after the sharpening processing and the standard PCB image.

[0037] In the embodiment, as shown in FIG. 1, the to-be-tested PCB image and the standard PCB image are compared to determine the defect position local image, including: Figure 3

[0038] Step 301, visual difference processing is performed on the to-be-tested PCB image and the standard PCB image to obtain a defect difference image.

[0039] Step 302, binarization processing is performed on the defect difference image according to a preset gray threshold to obtain a defect binarization image.

[0040] Step 303, edge detection is performed on the defect binarization image to obtain defect contour information.

[0041] Step 304, according to the defect contour information, a contour circumscribed rectangle algorithm is used to determine the center point coordinates of the circumscribed rectangle frame at the defect position.

[0042] Step 305, according to the center point coordinates, a defect position local image is intercepted on the to-be-tested PCB image.

[0043] In specific implementation, an industrial CCD camera with high sensitivity, high imaging quality and low noise is used to shoot a high-resolution to-be-tested PCB image in a production site. After the to-be-tested PCB image is grayed, median filtering processing is performed to reduce noise pollution. Then, Laplacian operator is used to perform sharpening processing on the to-be-tested PCB image after the median filtering processing, so that the defect edge can be highlighted. The to-be-tested PCB image after the sharpening processing and the standard PCB image are compared. First, visual difference processing is performed on the to-be-tested PCB image and the standard PCB image, that is, difference positioning is performed by pixel-by-pixel subtraction to obtain a defect difference image. Then, binarization processing is performed on the defect difference image according to a preset gray threshold to obtain a defect binarization image, as shown in FIG. 2. Figure 4 ​The edge detection method can adopt a canny edge detection algorithm. Then, according to the defect contour information, the center point coordinates of the circumscribed rectangle frame at the defect position are determined by using a contour circumscribed rectangle algorithm, and the local image of the defect position is intercepted on the image of the to-be-tested PCB according to the center point coordinates. Specifically, the defect position is located by the contour circumscribed rectangle algorithm, and the center point coordinates can be found by the information of the circumscribed rectangle frame of the defect. A rectangular region of a certain pixel size is intercepted by the center coordinates, so that the local image of the defect in the original image, that is, the local image of the defect position, is obtained, as shown in Figures 5-6 Figure 5 is a local image of a short-circuit defect position, Figure 6 is a local image of an open-circuit defect position.

[0044] In the embodiment, the local image of the defect position is input into the improved GoogleNet network model trained, to identify the defect type of the PCB, wherein the improved GoogleNet network model is trained according to the historical local image of the defect position, and the improved GoogleNet network model is obtained by improving the Inception structure of the GoogleNet network.

[0045] In the embodiment, as shown in Figure 7 the improved GoogleNet network model is trained according to the historical local image of the defect position in the following manner:

[0046] Step 701, data enhancement processing is performed on the historical local image of the defect position;

[0047] Step 702, one or any combination of rotation processing, translation processing, brightness transformation processing, and noise increase processing is performed on the historical local image of the defect position after the data enhancement processing, to generate a training data set;

[0048] Step 703, the improved GoogleNet network model is trained according to the training data set.

[0049] In specific implementation, the historical local image of the defect position is subjected to data enhancement processing, so as to increase the data amount, and then the historical local image of the defect position after the data enhancement processing is subjected to rotation processing, translation processing, brightness transformation processing, noise increase processing, etc., so as to enrich the data set, so that the trained network meets the requirements of translation invariance, rotation invariance, and robustness. The improved GoogleNet network model is trained according to the training data set.

[0050] In the embodiment, the improved GoogleNet network model is trained according to the training data set, including: ​

[0051] According to the training data set, the parameters of the improved GoogleNet network model are trained by using cross-entropy loss as a loss function for gradient descent of the improved GoogleNet network model.

[0052] In this embodiment, as shown in Figure 8 The Inception structure of the GoogleNet network is improved as follows:

[0053] Step 801, replace the 5x5 convolution kernel in the Inception structure with two 3x3 convolution kernels;

[0054] Step 802, replace the pooling layer in the Inception structure with a 3x3 convolution kernel and a 3x3 and 2-hole-rate convolution kernel;

[0055] In this embodiment, the improvement of the GoogleNet network includes:

[0056] Replace the 7x7 average pooling layer at the end of the GoogleNet network structure with a 7x7 convolution layer.

[0057] Add a 1x1x6 full connection layer after the full connection layer of the Inception structure of the original GoogleNet network.

[0058] In specific implementation, the improved GoogleNet network model is as shown in Figure 9As shown, 2 3x3 convolution kernels are used instead of 5x5 convolution kernels in the Inception structure of the GoogleNet network, so that the number of model training parameters can be effectively reduced, and the receptive field of this part is kept unchanged. The original pooling layer is deleted, and a 3x3 convolution kernel and a 3x3 and a hollow rate = 2 hollow convolution kernel are used instead, so that the Inception structure increases the equivalent 7x7 scale convolution kernel to calculate the output of this layer, and more different scale feature information can be fused. The 7x7 average pooling layer in the structure at the end of the GoogleNet network is changed to a 7x7 convolution layer, which can make the network fuse more feature information at the end. A layer of full connection layer is added after the full connection layer of the original GoogleNet network Inception structure, so that the dimension of the output is the number of categories to be classified. In the process of training the network, the loss proportion of the two auxiliary classifiers of the network is set to 0.3. When using the improved GoogleNet network model to train the parameters, first, the prepared historical defect position local image is preprocessed into the size of the input layer of the GoogleNet network, such as 224x224, then the cross entropy loss is used for the loss function of the gradient descent of the improved GoogleNet network model, finally 24 pictures are used as a Mini-Batch, and the network is iterated for 30 times to train the model parameters. The whole network training process is shown in Table 1, and the input image of 224x224x3 size finally outputs a 1x1x6 tensor, where "6" is the number of defect categories to be classified.

[0059] Table 1

[0060] Type Size Number Stride Output size Convolutional layer 7x7 64 2 112x112x64 Max pooling layer 3x3 2 56x56x64 Convolutional layer 3x3 192 1 56x56x192 Max pooling layer 3x3 2 28x28x192 Inception layer 28x28x256 Inception layer 28x28x480 Max pooling layer 3x3 2 14x14x480 Inception layer 14x14x512 Inception layer 14x14x512 Inception layer 14x14x512 Inception layer 14x14x528 Inception layer 14x14x832 Max pooling layer 3x3 2 7x7x832 Inception layer 7x7x832 Inception layer 7x7x1024 Convolutional layer 7x7 1024 1 1x1x1024 Dropout layer 1x1x1024 Fully connected layer 1x1x1000 Fully connected layer 1x1x6 Output layer 1x1x6

[0061] The embodiment of the present application does not need special workers to conduct manual detection, and uses an online visual detection method to conduct computer recognition. In view of the problem that the defects of high-precision and high-density PCB board lines are not obvious, the visual detection method of the present application can accurately locate the defect position and identify the defect type. The problem that the traditional detection algorithm specially designed for each defect leads to complex algorithm, redundancy and poor robustness is solved. The embodiment of the present application identifies and locates the defects of the PCB bare board through the visual detection method, combines the respective advantages of the traditional visual processing and the convolutional neural network, makes the PCB bare board wire defect detection more accurate, and overcomes the problem that the miniaturization, high precision and high density of the PCB board cause difficulty in defect detection. It is not necessary to design a corresponding defect detection algorithm, the visual difference positioning method is used to locate the defect position coordinates, the defect area is intercepted according to the coordinates, the accurate ROI area is obtained, and the defect classification network is input, so that the defect type is identified. Compared with directly training the target detection network algorithm structure, the traditional image processing is used to locate the defect position, and the target classification network is used to identify the defect, the algorithm structure is simpler, the algorithm positioning is more accurate, and the running efficiency is high. The convolutional neural network structure is improved, so that the model training is more rapid and accurate, and the defect identification and classification are more accurate.

[0062] Based on the same inventive concept, the embodiment of the present application also provides a PCB board defect detection device, as described in the following embodiment. Since the principles of solving the problems are similar to the PCB board defect detection method, the implementation of the PCB board defect detection device can be referred to the implementation of the method, and the repeated parts will not be described again.

[0063] Figure 10 The structural diagram of the PCB board defect detection device in the embodiment of the present application is shown in FIG. 1, and the PCB board defect detection device comprises: Figure 10

[0064] An image obtaining module 1001 is configured to obtain a to-be-detected PCB board image and a standard PCB board image.

[0065] A position determining module 1002 is configured to compare the to-be-detected PCB board image and the standard PCB board image to determine a defect position local image.

[0066] A defect identifying module 1003 is configured to input the defect position local image into a trained improved GoogleNet network model to identify a defect type of the PCB board, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving an Inception structure of a GoogleNet network.

[0067] ​In one embodiment, the Inception structure of the GoogleNet network is improved as follows:

[0068] The 5x5 convolution kernel in the Inception structure is replaced by two 3x3 convolution kernels;

[0069] The pooling layer in the Inception structure is replaced by a 3x3 convolution kernel and a 3x3 and 2-hole-rate convolution kernel.

[0070] In summary, compared with the technical solutions of the prior art for detecting defects of a PCB by manual detection, traditional image processing and automatic optical detection, the embodiment of the present application compares a to-be-detected PCB image and a standard PCB image to determine a defect position local image, inputs the defect position local image into a trained improved GoogleNet network model to identify a defect type of the PCB, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving the Inception structure of the GoogleNet network. The embodiment of the present application does not need to perform manual detection or design different process algorithms for different types of PCBs. Instead, the embodiment of the present application only needs to improve the Inception structure of the GoogleNet network to obtain an improved GoogleNet network model, train the improved GoogleNet network model according to historical defect position local images, and then compare the to-be-detected PCB image and the standard PCB image to determine a defect position local image, which is then input into the trained improved GoogleNet network model. As a result, the embodiment of the present application can detect defects of a PCB, overcome the problem of difficulty in defect detection caused by miniaturization, high precision and high density of a PCB, improve detection efficiency and accuracy, and effectively reduce maintenance costs.

[0071] Based on the foregoing inventive concept, as shown in Figure 11 The present application further provides a computer device 1100, which comprises a memory 1110, a processor 1120, and a computer program 1130 stored in the memory 1110 and executable on the processor 1120, wherein the processor 1120 implements the foregoing PCB defect detection method when executing the computer program 1130.

[0072] Based on the foregoing inventive concept, the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the foregoing PCB defect detection method.

[0073] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0074] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0075] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0077] The specific embodiments described above are illustrative for purposes of the present application. The particular implementations are not intended to limit the scope of the present application, which is defined by the appended claims. Numerous variations, changes, and substitutions can be made without departing from the application. It is the intention of the present application to embrace all such variations, changes, and substitutions that fall within the scope of the present application.

Claims

1. A method of PCB board defect detection, characterized in that, The method comprises the following steps: obtaining a to-be-tested PCB image and a standard PCB image; comparing the to-be-tested PCB image and the standard PCB image to determine a defect position local image; inputting the defect position local image into a trained improved GoogleNet network model to identify the defect type of the PCB, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving the Inception structure of the GoogleNet network. The Inception structure of the GoogleNet network is improved in the following manner: two 3×3 convolution kernels are used to replace the 5×5 convolution kernel in the Inception structure; a 3×3 convolution kernel and a hollow convolution kernel are used to replace the pooling layer in the Inception structure, wherein the hollow convolution kernel is a 3×3 convolution kernel with a hollow rate of 2.

2. The PCB board defect detection method of claim 1, wherein, The method further comprises the following steps: performing median filtering processing on the to-be-tested PCB image; performing sharpening processing on the to-be-tested PCB image after the median filtering processing by using a Laplacian operator; comparing the to-be-tested PCB image and the standard PCB image, which comprises comparing the to-be-tested PCB image after the sharpening processing and the standard PCB image.

3. The PCB board defect detection method of claim 1, wherein, The method of comparing the to-be-tested PCB image and the standard PCB image to determine the defect position local image comprises the following steps: performing visual difference processing on the to-be-tested PCB image and the standard PCB image to obtain a defect difference image; performing binarization processing on the defect difference image according to a preset gray threshold to obtain a defect binarization image; performing edge detection on the defect binarization image to obtain defect contour information; determining the center point coordinates of the circumscribed rectangle frame at the defect position by using a contour circumscribed rectangle algorithm according to the defect contour information; cutting the defect position local image from the to-be-tested PCB image according to the center point coordinates.

4. The method of claim 1, wherein the step of detecting defects on the PCB board is performed by a machine vision system. The improved GoogleNet network model is trained according to historical defect position local images in the following manner: performing data enhancement processing on the historical defect position local images; generating a training data set by performing one or any combination of rotation processing, translation processing, brightness transformation processing and noise increase processing on the historical defect position local images after the data enhancement processing; training the improved GoogleNet network model according to the training data set.

5. The PCB board defect detection method of claim 4, wherein, The method of training the improved GoogleNet network model according to the training data set comprises the following steps: training the parameters of the improved GoogleNet network model by using cross-entropy loss as the loss function of gradient descent of the improved GoogleNet network model according to the training data set.

6. A PCB board defect detection apparatus, characterized by, The method comprises the following steps: an image obtaining module for obtaining a to-be-tested PCB image and a standard PCB image; a position determining module for comparing the to-be-tested PCB image and the standard PCB image to determine a defect position local image; The defect recognition module is configured to input the defect position local image into a trained improved GoogleNet network model to recognize a defect type of the PCB, wherein the improved GoogleNet network model is trained according to historical defect position local images, and the improved GoogleNet network model is obtained by improving an Inception structure of a GoogleNet network. The Inception structure of the GoogleNet network is improved in the following manner: Two 3*3 convolution kernels are used to replace 5*5 convolution kernels in the Inception structure; A 3*3 convolution kernel and a hollow convolution kernel are used to replace a pooling layer in the Inception structure, wherein the hollow convolution kernel is a 3*3 convolution kernel with a hollow rate of 2.

7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the method of any one of claims 1-5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program for executing the method of any one of claims 1-5.

Images