Method, device and system for detecting microdefects of PCB (Printed Circuit Board) and medium

Abstract

The invention discloses a PCB microdefect detection method, device and system and a medium, and the method comprises the following steps: dividing a to-be-detected PCB into a plurality of to-be-detected areas according to PCB graphic characteristics, and obtaining the center coordinates of the to-be-detected areas on a to-be-detected PCB detection graph; obtaining a corresponding detection image block according to the area size of each to-be-detected region; dividing the detection image block into a plurality of to-be-detected sub-images, converting the plurality of to-be-detected sub-images into a high-dimensional matrix, inputting the high-dimensional matrix into a pre-trained defect detection model, and outputting a micro-defect classification result; according to the method, the image is acquired from the local area, so that the image resolution is higher, and the microdefect detection capability is stronger; and the detection model is used for automatically carrying out micro-defect detection on the image, so that the problem of possible missing of personnel is solved compared with manual microscope visual inspection.

Description

Technical Field

[0001] The invention relates to a method, device, system and medium for detecting micro-defects of a PCB board, and belongs to the technical field of printed circuit board detection. Background Art

[0002] The appearance inspection of PCB boards before shipment is to compare the difference between the board to be inspected and the perfect board through the automated visual inspection machine (AVI), mark the difference points and then hand it over to manual confirmation. As the graphic pads are getting smaller and smaller and the customer's appearance requirements are getting higher and higher, AVI equipment has been iterating to improve the detection capability, mainly by improving the camera resolution (pixel: 25umà18umà10um), so as to meet the requirements of detecting micro defects of PCB boards; however, with AVI using high-resolution cameras, the number of false points generated by the machine will increase, which means that the man-hours for manual confirmation will increase; at the same time, the size of many micro defects is only 20-30um, because the setting of AVI parameters and the AVI machine generally require 3 pixels to image, this type of micro defects still cannot be detected by AVI; in order to overcome the problem of insufficient detection capability of AVI machines for micro defects, many PCB board manufacturers have added a manual microscope inspection after AVI inspection. This inspection is mainly for local areas with micro defect detection requirements (such as BGA areas, IC areas), and micro defects are manually visually inspected by area under a high magnification microscope. Although this method improves the ability to detect micro-defects, the additional manual microscope inspection is slow and time-consuming, which increases the cost of PCB board inspection. At the same time, manual inspection is not as stable as machine inspection and there is a risk of omission. Summary of the invention

[0003] The purpose of the present invention is to provide a method, device, system and medium for detecting micro-defects of PCB boards. The present invention collects images of local areas through a high-resolution area array camera, and the image resolution is higher, and the detection capability of micro-defects is stronger; and a defect detection model is used to automatically detect micro-defects in the image, thereby solving the problem of possible omission of personnel.

[0004] In order to achieve the above objectives / solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0005] In a first aspect, a method for detecting micro-defects of a PCB board comprises: Divide the PCB to be tested into several areas to be tested according to the PCB graphic features; Taking any corner of the PCB to be tested as the origin, establishing a plane coordinate system of the PCB to be tested according to the size of the PCB, and obtaining the coordinates of the center of the area to be tested on the test map of the PCB to be tested; Based on the coordinates of the center of the area to be detected, the corresponding detection block is obtained according to the area size of each area to be detected; Dividing the inspection image block into a number of sub-images to be inspected, converting the sub-images to be inspected into a high-dimensional matrix, inputting the high-dimensional matrix into a pre-trained defect detection model, and outputting a micro-defect classification result; The NMS processing is performed on the sub-image to be tested with micro-defects, and the coordinates of the micro-defects are converted into coordinates on the detection image of the PCB board to be tested. The NMS processing is performed on the detection image of the PCB board to be tested to obtain the final detection result of the PCB board to be tested.

[0006] Furthermore, the PCB to be tested is divided into a number of areas to be tested according to the PCB graphic features, specifically including: According to the distribution position and quantity of the PCB graphic features of the micro defects to be detected on the PCB to be detected, the PCB to be detected is divided into several areas to be detected; The PCB graphic features include: a ball grid array structure area and a component pin patch area.

[0007] Furthermore, based on the coordinates of the center of the area to be detected, obtaining the corresponding detection block according to the area size of each area to be detected specifically includes: The preset area array camera is moved from the origin to the coordinates of the center of the area to be detected, and the area of ​​the detection block is defined by the preset field of view size of the area array camera; When the area of ​​the to-be-detected region is smaller than the area of ​​the detection block, a shot is taken with the center of the to-be-detected region as the base point to obtain the detection block; When the area of ​​the to-be-detected region is larger than the area of ​​the detection block, the to-be-detected region is translated with the corners of the to-be-detected region as the base point according to the length or width of the to-be-detected region, and several shots are taken to obtain several detection blocks until the detection block contains the complete to-be-detected region.

[0008] Furthermore, the step of dividing the detection image block into a plurality of sub-images to be detected specifically includes: A window is used to slide on the detection block, and after the sliding process, several sub-images to be detected of the detection block are generated; The window size is M*M pixels, and the horizontal and vertical distances of the sliding process are set to n pixels.

[0009] Furthermore, the training method of the defect detection model specifically includes: Step 1: Obtain several historical PCB board images and the classification results of micro defects in each PCB board image; Step 2: Divide the acquired historical PCB board images and classification results into training set, verification set and test set at a ratio of 60%:20%:20%; Step 3: Input the training set into the defect detection model and use the optimizer SGD to adjust the model parameters to minimize the function loss value; Step 4: When the mean average precision index of the defect detection model on the validation set does not change for 100 consecutive iterations, stop training and save the current model; Step 5: Input the test set into the saved current model to confirm whether the model can detect and correctly classify 100% of the PCB images with micro-defects in the test set, and at the same time, the proportion of PCB images without micro-defects that are misjudged as PCB images with micro-defects must be lower than the training standard of 2%. If this training standard cannot be met, go to step 6. If this training standard is met, save the defect detection model and stop model training. Step 6: Add historical PCB board images and the classification results of micro defects for each PCB board image, and repeat steps 2 to 5.

[0010] Further, the micro-defect coordinates on the sub-image to be tested are converted into coordinates on the inspection image of the PCB board to be tested, specifically including: The four vertex coordinates (x i ,y i ), converted to the coordinates of the PCB to be tested (x i +a1*n*pixel size,y i +a2*n*pixel size); Wherein: i is 1, 2, 3, 4; n is the window sliding horizontal and vertical distance settings; a1 is the number of times the window slides in the sub-image to be tested in the X direction, a2 is the number of times the window slides in the sub-image to be tested in the Y direction, and a1 and a2 are integers 0, 1, 2, 3...

[0011] In a second aspect, the present invention provides a device for detecting micro-defects of a PCB board, comprising: The detection area module is used to divide the PCB board to be tested into several detection areas according to the PCB graphic characteristics; A coordinate module is used to establish a plane coordinate system of the PCB board to be tested based on the size of the PCB board, taking any corner of the PCB board to be tested as the origin, and obtaining the coordinates of the center of the area to be tested on the test map of the PCB board to be tested; A detection block module is used to obtain corresponding detection blocks according to the size of each area to be detected based on the coordinates of the center of the area to be detected; A classification module is used to divide the inspection image block into a number of sub-images to be tested, and convert the sub-images to be tested into a high-dimensional matrix, input the high-dimensional matrix into a pre-trained defect detection model, and output a micro-defect classification result; The detection module is used to perform NMS processing on the sub-image to be tested with micro-defects, convert the micro-defect coordinates into coordinates on the detection image of the PCB board to be tested, and perform NMS processing on the detection image of the PCB board to be tested to obtain the final detection result of the PCB board to be tested.

[0012] In a third aspect, the present invention provides a PCB micro-defect detection system, comprising: Memory, for storing computer programs / instructions; A processor is used to execute the computer program / instructions to implement the steps of the above-mentioned method for detecting micro-defects of PCB boards.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for detecting micro-defects of PCB boards.

[0014] Compared with the prior art, the present invention has the following beneficial effects: the present invention uses a defect detection model to automatically detect micro-defects in images, which solves the problem of time-consuming and possible omissions in human inspection compared with manual microscope visual inspection; The present invention collects images of local areas through an area array camera, and the image resolution is higher. Compared with the mainstream AVI machines in the current market, the device has a stronger ability to detect micro defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0016] It should be noted that: The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0017] The term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " generally indicates that the related objects are in an "or" relationship. Example

[0018] like Figure 1 As shown, this embodiment provides a method for detecting micro-defects of a PCB board, comprising: Step S1: Divide the PCB into several local inspection areas according to the field of view of the high-resolution camera and the location distribution of micro-defects on the PCB, and store the center coordinates of each local inspection area, specifically including: Step S1.1: Determine how many areas on the PCB surface need to be inspected based on the type and distribution of micro-defects, and calculate the area size S (S1, S2, S3, ..., S p ), S p is the area size of the pth area to be detected; Step S1.2: Determine the area Q that can be photographed each time based on the field of view of the high-resolution camera, and calculate the number of times each area to be inspected needs to be photographed. This case uses a 5-megapixel black-and-white area array camera (pixel size is 3.45um) with a field of view of 16.8mm*14.1mm. If Q>S, then the area is photographed Figure 1 If Q < S, the area needs to be photographed multiple times by moving the camera, and the total number of photographs required is calculated as O; Step S1.3: Take one of the four corners of the PCB board as the origin coordinate and calculate the position coordinates (X o , Y o ), o is the total number of photos taken, which can be an integer of 1, 2, 3, etc.; Step S2: Place the PCB board to be inspected on a fixed position on the mobile platform and start inspection; Step S3: Move from the origin to the center coordinates of the first local detection area, triggering the camera to take a photo to obtain an image of the area; Step S4: Use a window of M*M pixels and set the horizontal and vertical distances to n pixels to slide on the image to obtain several sub-images of the detection area image, including: Use a window of M*M pixels to slide on the image. The horizontal and vertical distances of the slide are set to n pixels. Each slide generates a sub-image of the image. After the slide is completed, several sub-images of the image are generated. The setting value of M is related to the detection model in step 5. The setting of n can refer to the size of the micro-defect. In this case, M is set to 640 and n is set to 400. Step S5: Input each sub-image of the detection area into the defect detection model, detect whether there is a target defect in each sub-image and obtain the boxes of the defect, specifically including: Step S5.1: Convert the several sub-images generated by sliding the detection area window into a high-dimensional matrix, i.e., a tensor of BCHW dimensions; where B is the batch size, i.e., the number of samples to be inferred; C is the channel, i.e., the number of channels of the image; H is the height, i.e., the height of the sub-image; and W is the width, i.e., the width of the sub-image. Step S5.2: Input the tensor of the sub-graph into the defect detection model for inference, determine whether there are micro-defects to be detected in the sub-graph, and output the micro-defect type and defect boxes. The defect detection model is a target detection model that learns different annotated micro-defect types. In addition to outputting the micro-defect type and confidence, the model also outputs the box bounding box; Step S6: Perform NMS processing on the detection result of each sub-image, convert its coordinates into the coordinates of the detection block, and further perform NMS processing on the detection block, specifically including: Step S6.1: Perform NMS non-maximum suppression algorithm processing on the detection results of each sub-image. The purpose of NMS processing is to remove redundant prediction boxes and retain the best prediction boxes. Specific method: First, select the prediction bounding box B1 with the highest confidence as the benchmark from all candidate boxes, and then remove all other bounding boxes whose IOU with B1 exceeds the set threshold. Then select the bounding box B2 with the second highest confidence as the benchmark from all candidate boxes, and remove all other bounding boxes whose IOU with B2 exceeds the predetermined threshold. Repeat the above steps until all prediction boxes are used as benchmarks. The threshold is generally set to 0.5 or 0.7, and is set to 0.5 in this case; Step S6.2: Convert the coordinates of the prediction boxes in each sub-image after NMS processing into the coordinates of the detection block and map them in the detection block, specifically including: The four vertex coordinates (x i ,y i ), converted to the coordinates of the PCB to be tested (x i +a1*n*pixel size,y i +a2*n*pixel size); Where: i is 1, 2, 3, 4; n is the window sliding horizontal and vertical distance settings; a1 is the number of sliding times of the window in the sub-image to be tested in the X direction, a2 is the number of sliding times of the window in the sub-image to be tested in the Y direction, a1 and a2 are integers 0, 1, 2, 3...; Step S6.3: Perform NMS processing on the predicted box on the detection block, and finally obtain the predicted defect type and defect boxes of the detection block; Step S7: Determine whether each local inspection area of ​​the PCB board has completed the inspection. If not, the platform moves to the next local inspection area. If yes, the platform returns to the origin. Step S8: Save the detection image and detection result of each local detection area, and end the PCB board detection.

[0019] Wherein: the training method of the defect detection model specifically includes: Step 1: Obtain several historical PCB board images and the classification results of micro defects in each PCB board image; Step 2: Divide the acquired historical PCB board images and classification results into training set, verification set and test set at a ratio of 60%:20%:20%; Step 3: Input the training set into the defect detection model and use the optimizer SGD to adjust the model parameters to minimize the function loss value; Step 4: When the mean average precision index of the defect detection model on the validation set does not change for 100 consecutive iterations, stop training and save the current model; Step 5: Input the test set into the saved current model to confirm whether the model can detect and correctly classify 100% of the PCB images with micro-defects in the test set, and at the same time, the proportion of PCB images without micro-defects that are misjudged as PCB images with micro-defects must be lower than the training standard of 2%. If this training standard cannot be met, go to step 6. If this training standard is met, save the defect detection model and stop model training. Step 6: Add historical PCB board images and the classification results of micro defects for each PCB board image, and repeat steps 2 to 5. Example

[0020] This embodiment provides a PCB micro-defect detection device, including: The module for the area to be inspected is used to divide a number of areas to be inspected on the PCB board to be inspected according to the requirements of micro-defect detection of the PCB board; A coordinate module is used to establish a plane coordinate system of the PCB board to be tested based on the size of the PCB board, taking any corner of the PCB board to be tested as the origin, and obtaining the coordinates of the center of the area to be tested on the test map of the PCB board to be tested; A detection block module is used to obtain corresponding detection blocks according to the size of each area to be detected based on the coordinates of the center of the area to be detected; A classification module is used to divide the detection block into several sub-images to be tested and convert them into tensors of BCHW dimension, input the tensors of the sub-images to be tested into a pre-trained defect detection model, perform inference on the tensors of the sub-images to be tested, determine whether there are micro-defects to be detected in the sub-images to be tested, and output the classification results; The detection module is used to perform NMS processing on the classification results of the sub-image to be tested, and convert the position coordinates of the determination results of the sub-image to be tested into coordinates on the detection map of the PCB board to be tested, and perform NMS processing on the detection map of the PCB board to be tested to obtain the final detection result of the PCB board detection map. Example

[0021] This embodiment provides a PCB micro-defect detection system, including: Memory, for storing computer programs / instructions; A processor is used to execute the computer program / instructions to implement the steps of the above-mentioned method for detecting micro-defects of PCB boards. Example

[0022] This embodiment provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the above-mentioned PCB board micro-defect detection method are implemented.

[0023] The detection cost of the present invention is low. In the existing detection process, the present invention uses a solution of ordinary resolution AVI machine detection + high-resolution camera by area detection. Only one small mobile platform and one high-resolution area array camera need to be added to the overall device, and the solution cost is low. After using the ordinary resolution AVI machine for detection, the solution of the present invention is used to automatically detect micro defects one by one by area, thereby solving the problem that the micro-defect detection capability of the AVI equipment is poor, resulting in the need for manual microscope visual inspection.

[0024] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0025] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0026] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0027] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0028] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for detecting micro-defects of a PCB board, characterized in that: include: Divide the PCB to be tested into several areas to be tested according to the PCB graphic features; Taking any corner of the PCB to be tested as the origin, establishing a plane coordinate system of the PCB to be tested according to the size of the PCB, and obtaining the coordinates of the center of the area to be tested on the test map of the PCB to be tested; Based on the coordinates of the center of the area to be detected, the corresponding detection block is obtained according to the area size of each area to be detected; Dividing the inspection image block into a number of sub-images to be inspected, converting the sub-images to be inspected into a high-dimensional matrix, inputting the high-dimensional matrix into a pre-trained defect detection model, and outputting a micro-defect classification result; The NMS processing is performed on the sub-image to be tested with micro-defects, and the coordinates of the micro-defects are converted into coordinates on the detection image of the PCB board to be tested. The NMS processing is performed on the detection image of the PCB board to be tested to obtain the final detection result of the PCB board to be tested.

2. The method for detecting micro-defects of a PCB board according to claim 1, characterized in that: The method of dividing the PCB to be tested into a plurality of areas to be tested according to the PCB pattern features specifically includes: According to the distribution position and quantity of the PCB graphic features of the micro defects to be detected on the PCB to be detected, the PCB to be detected is divided into several areas to be detected; The PCB graphic features include: a ball grid array structure area and a component pin patch area.

3. The method for detecting micro-defects of a PCB board according to claim 2, characterized in that: The method of obtaining corresponding detection blocks according to the size of each area to be detected based on the coordinates of the center of the area to be detected specifically includes: The preset area array camera is moved from the origin to the coordinates of the center of the area to be detected, and the area of ​​the detection block is defined by the preset field of view size of the area array camera; When the area of ​​the to-be-detected region is smaller than the area of ​​the detection block, a shot is taken with the center of the to-be-detected region as the base point to obtain the detection block; When the area of ​​the to-be-detected region is larger than the area of ​​the detection block, the to-be-detected region is translated with the corners of the to-be-detected region as the base point according to the length or width of the to-be-detected region, and several shots are taken to obtain several detection blocks until the detection block contains the complete to-be-detected region.

4. The method for detecting micro-defects of a PCB board according to claim 3, characterized in that: The step of dividing the detection image block into a plurality of sub-images to be detected specifically includes: A window with a size of M*M pixels is used to slide on the detection block, and after the sliding process, several sub-images to be tested of the detection block are generated; Wherein: the horizontal and vertical distances of the sliding process are set to n pixels.

5. The method for detecting micro-defects of a PCB board according to claim 1, characterized in that: The training method of the defect detection model specifically includes: Step 1: Obtain several historical PCB board images and the classification results of micro defects in each PCB board image; Step 2: Divide the acquired historical PCB board images and classification results into training set, verification set and test set at a ratio of 60%:20%:20%; Step 3: Input the training set into the defect detection model and use the optimizer SGD to adjust the model parameters to minimize the function loss value; Step 4: When the mean average precision index of the defect detection model on the validation set does not change for 100 consecutive iterations, stop training and save the current model; Step 5: Input the test set into the saved current model to confirm whether the model can detect and correctly classify 100% of the PCB images with micro-defects in the test set, and at the same time, the proportion of PCB images without micro-defects that are misjudged as PCB images with micro-defects must be lower than the training standard of 2%. If this training standard cannot be met, go to step 6. If this training standard is met, save the defect detection model and stop model training. Step 6: Add historical PCB board images and the classification results of micro defects for each PCB board image, and repeat steps 2 to 5.

6. The method for detecting micro-defects of a PCB board according to claim 4, characterized in that: The micro-defect coordinates on the sub-image to be tested are converted into coordinates on the inspection image of the PCB board to be tested, specifically including: The four vertex coordinates (x i ,y i ), converted to the coordinates of the PCB to be tested (x i +a1*n*pixel size,y i +a2*n*pixel size); Wherein: i is 1, 2, 3, 4; n is the window sliding horizontal and vertical distance settings; a1 is the number of times the window slides in the sub-image to be tested in the X direction, a2 is the number of times the window slides in the sub-image to be tested in the Y direction, and a1 and a2 are integers 0, 1, 2, 3...

7. A PCB micro-defect detection device, characterized in that: include: The detection area module is used to divide the PCB board to be tested into several detection areas according to the PCB graphic characteristics; A coordinate module is used to establish a plane coordinate system of the PCB board to be tested based on the size of the PCB board, taking any corner of the PCB board to be tested as the origin, and obtaining the coordinates of the center of the area to be tested on the test map of the PCB board to be tested; A detection block module is used to obtain corresponding detection blocks according to the size of each area to be detected based on the coordinates of the center of the area to be detected; A classification module is used to divide the inspection image block into a number of sub-images to be tested, and convert the sub-images to be tested into a high-dimensional matrix, input the high-dimensional matrix into a pre-trained defect detection model, and output a micro-defect classification result; The detection module is used to perform NMS processing on the sub-image to be tested with micro-defects, convert the micro-defect coordinates into coordinates on the detection image of the PCB board to be tested, and perform NMS processing on the detection image of the PCB board to be tested to obtain the final detection result of the PCB board to be tested.

8. A PCB board micro-defect detection system, characterized in that: include: Memory, for storing computer programs / instructions; A processor, used to execute the computer program / instructions to implement the steps of the method for detecting micro-defects of a PCB board according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the method for detecting micro-defects of a PCB board described in any one of claims 1-6 are implemented.

Images