Defect detection method and device with determined reference, medium and program product

Abstract

The invention provides a defect detection method with a determined reference, which comprises the following steps of: acquiring an image of a defect-free workpiece as a template image, extracting local peak values of three primary color layer images from the template image, fusing the local peak values, summarizing all fusion results to form a threshold value set, and calculating a reference value based on each threshold value; performing binarization processing on the grey-scale images of the template images to obtain detection primitive template images, and constructing a detection primitive library; then, collecting an image of a to-be-detected workpiece as a sample detection image, and performing binarization processing on a grey-scale image of the sample detection image based on each threshold value to obtain a detection element sample detection image; and finally, performing subtraction on each detection element sample drawing and a corresponding detection element template drawing in the detection element library to obtain a defect detection result. According to the method, the deep learning technology is not used, the workload of collecting a large number of training samples and manually labeling the samples is avoided, the image processing technology is used, the requirement for the performance of processing equipment is not high, and the method is very suitable for being used in the industrial scene structured environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and particularly relates to a defect detection method, device, medium, and program product with a definite reference. Background Art

[0002] There are various types of defects on the surface of the PCB board, the defects are subtle, the unknownness is large, and the difference between classes is small, making defect detection quite difficult.

[0003] The currently popular intelligent detection method is the defect detection method based on deep learning. However, on the one hand, the deep learning algorithm requires a large number of training samples, and the training sample acquisition environment is required to be as rich and diverse as possible to improve the representativeness of the training samples; on the other hand, for the deep learning detection method based on supervised learning, professional personnel are also required to accurately label the training samples, which greatly increases the training cost. In addition, in order to ensure the detection accuracy, the complexity of the corresponding neural network is also relatively high, further increasing the training time and the difficulty of optimizing the model parameters. Specifically in the field of PCB board detection, it is reflected as:

[0004] ① It is difficult to collect a large number of and full-coverage defect samples;

[0005] ② The labeling of training samples is difficult and the workload is large for on-site workers;

[0006] ③ The training of the super-large deep neural network model also further restricts the practical application due to the requirements for equipment.

[0007] Therefore, the current defect detection method based on deep learning is difficult to be applied in the field of PCB board detection. Summary of the Invention

[0008] Aiming at the deficiencies in the prior art, the present invention provides a defect detection method, device, medium, and program product with a definite reference to solve the problem of defect detection with a definite reference detection primitive.

[0009] The present invention achieves the above technical objectives through the following technical means.

[0010] A defect detection method with a definite reference:

[0011] Step 1, construction of a detection primitive library:

[0012] Step 1.1, collect the image of a defect-free workpiece as a template image;

[0013] Step 1.2, extract the three primary color layer images of the template image;

[0014] Step 1.3, respectively extract the local peaks of the three primary color layer images;

[0015] Step 1.4: Extract one local peak from each of the three primary color images for fusion, and aggregate all the fusion results to form a threshold set;

[0016] Step 1.5: Based on each threshold in the threshold set, perform binarization on the grayscale image of the template image to obtain a detected primitive template image, and aggregate all the detected primitive template images to obtain a detected primitive library;

[0017] Step 2: Extraction of detected primitives from the sample image:

[0018] Step 2.1: Collect an image of the workpiece to be detected as the sample image;

[0019] Step 2.2: Based on each threshold in the threshold set, perform binarization on the grayscale image of the sample image to obtain a detected primitive sample image;

[0020] Step 3: Subtract each detected primitive sample image from the corresponding detected primitive template image in the detected primitive library to obtain a defect detection result.

[0021] Further, before performing binarization on the sample image, first map the sample image to the coordinate system of the template image and align it with the template image.

[0022] Further, the mapping relationship between the sample image and the template image is:

[0023]

[0024] where M is the mapping transformation matrix, a, b, c, d, e, f are the elements in the mapping transformation matrix, (x, y) represents the pixel coordinates in the template image, and (x ′ , y ′ ) represents the pixel coordinates in the sample image corresponding to (x, y);

[0025] By selecting several groups of Mark points between the template image and the sample image, based on the Mark point coordinates, use the least squares method to solve the mapping transformation matrix M, and then use the mapping transformation matrix M to map all the pixel points on the sample image to the coordinate system of the template image;

[0026] After mapping, through a shift operation, align the positions of the sample image and the template image.

[0027] Further, for detecting defects on the PCB board surface, the workpiece is a PCB board.

[0028] Further, for detecting defects occurring at the solder pads, copper wires, vias, blind vias, and silk printing positions on the PCB board.

[0029] Further, the number of Mark points is 3.

[0030] Furthermore, the method for extracting the local peak is as follows: first, construct a histogram for the base color layer image, and then extract the local peak from the histogram.

[0031] A computer device includes a memory and a processor;

[0032] The memory is used to store a computer program;

[0033] The processor is used to execute the computer program and implement the above-mentioned defect detection method with a definite reference when executing the computer program.

[0034] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the above-mentioned defect detection method with a definite reference.

[0035] A computer program product includes a computer program, and when the computer program is executed by a processor, the above-mentioned defect detection method with a definite reference is implemented.

[0036] The beneficial effects of the present invention are as follows:

[0037] (1) The present invention provides a defect detection method with a definite reference. For the defect detection problem with a definite detection primitive as a reference template, the fusion result between the local peaks of the three primary color layer images is used as a threshold to perform binary processing on the grayscale image, so as to realize the reliable separation and extraction of each detection primitive in this way, and avoid the adverse effects caused by factors such as plate color difference fluctuation, micro deformation, and process control process drift, thereby improving the accuracy of defect detection.

[0038] (2) The present invention uses Mark points to map and align the sample image, so that the sample image and the template image are accurately coordinated, ensuring the detection accuracy and reliability.

[0039] (3) In view of the limitations of the industrial scenario, the present invention does not use deep learning technology, avoiding the workload of collecting a large number of training samples and manually annotating samples. Moreover, this method uses image processing technology and has low requirements for the performance of processing equipment, and is very suitable for use in the structured environment of industrial scenarios.

[0040] (4) The method proposed by the present invention has fast detection efficiency, high detection accuracy, and low cost, and is suitable for industrial scenarios. Description of the Drawings

[0041] Figure 1 It is an example of the defect detection process of the present invention;

[0042] Figure 2 It is a flowchart of the defect detection of the present invention. Detailed Embodiments

[0043] Embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0044] Surface defects on bare PCBs can be divided into "defects with definite references" and "defects without definite references" according to the location of the defects.

[0045] Defects with a definite reference: refers to defects that occur on "detection primitives" such as pads, copper wires, through holes / blind holes, silk screens, etc. These defects can be tested against standard qualified detection primitives as reference templates.

[0046] Defects without a definite reference: such as scratches, green oil, dirt and other defects that occur in uncertain locations.

[0047] This embodiment proposes the following detection scheme for the above-mentioned defects with a certain reference:

[0048] 1. Defect detection method with a certain reference

[0049] Reference Figure 1 and Figure 2 As shown, this embodiment includes the following processing procedures:

[0050] 1. Construction of detection primitive library

[0051] 1) Template image acquisition: Collect a color image of a defect-free PCB bare board as a template image.

[0052] 2) Mark point marking: three mark points (reference points) are marked on the template image by manual marking method, which serve as reset reference points for subsequent inspection sample images.

[0053] 3) Extracting three primary color layer images: For the above template image, extract the primary color layer images of the three channels of R (red), G (green), and B (blue).

[0054] 4) Histogram peak extraction: For the three primary color layer images, construct their own histograms respectively, and then extract their own local peaks from each histogram.

[0055] 5) Grayscale: Convert the color image of the template image into a grayscale image.

[0056] 6) Binarization: fuse the local peak values ​​of the three primary color layer images in groups of three, and use the fusion results as the thresholds for binarization; summarize the fusion results of all local peak combinations to obtain the threshold set;

[0057] For example, the R primary color layer image has 3 local peaks R1, R2, and R3, the G primary color layer image has 2 local peaks G1 and G2, and the B primary color layer image has 2 local peaks B1 and B2. Then, perform peak fusion on the following 12 combinations respectively: R1G1B1, R1G1B2, R1G2B1, R1G2B2, R2G1B1, R2G1B2, R2G2B1, R2G2B2, R3G1B1, R3G1B2, R3G2B1, R3G2B2, to obtain 12 thresholds.

[0058] For each threshold in the threshold set, perform binarization on the grayscale image of the template image to obtain several detection primitive template images, and finally summarize the obtained several detection primitive template images into the detection primitive library.

[0059] 2. Detection primitive extraction of the sample image

[0060] 1) Sample image acquisition: For the PCB board that needs to be defect-detected, collect its bare board color image, denoted as the sample image.

[0061] 2) Mark point extraction: Based on the Mark points marked on the template image, extract 3 Mark points at the same positions on the PCB board in the sample image.

[0062] 3) Sample image mapping and alignment: Based on the selected Mark points, map the sample image to the coordinate system of the template image, and the mapping relationship is as follows:

[0063]

[0064] In the formula, M is the mapping transformation matrix, a, b, c, d, e, f are the elements in the mapping transformation matrix, (x, y) represents the pixel point coordinates in the template image, and (x ′ , y ′ ) represents the pixel point coordinates in the sample image corresponding to (x, y). By substituting the coordinates of three groups of corresponding Mark points on the template image and the sample image, use the least squares method to solve the mapping transformation matrix M. Then, use the mapping transformation matrix M to map all pixel points on the sample image to the coordinate system of the template image.

[0065] After mapping, the sample image may still have a positional deviation from the template image. Through a shift operation, the sample image and the template image can be aligned in position.

[0066] 4) Grayscale processing: Perform grayscale processing on the sample image to obtain the grayscale image of the sample image.

[0067] 5) Binarization: For each threshold in the threshold set, perform binarization on the grayscale image of the sample image to obtain several detection primitive sample images.

[0068] 3. Defect Detection

[0069] By subtracting each inspection sample image of the inspection elements from the corresponding inspection element template image (which is binarized by the same threshold) in the inspection element library, the defect detection result image can be obtained.

[0070] II. Device, Storage Medium, Program Product

[0071] 1. Based on the same inventive concept as the above-mentioned defect detection method with a definite reference, the present application also provides an electronic device, which includes a processor and a memory. The memory stores computer-readable code. When the computer-readable code is executed by the processor, the defect detection method with a definite reference of the present invention is implemented.

[0072] Among them, the memory includes a non-volatile storage medium and an internal memory; the non-volatile storage medium can store an operating system and computer-readable code. The computer-readable code includes program instructions, and when the program instructions are executed, the processor can be made to execute the defect detection method with a definite reference. The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory provides an environment for the operation of the computer-readable code in the non-volatile storage medium, and when the computer-readable code is executed by the processor, the processor can be made to execute the defect detection method with a definite reference.

[0073] It should be understood that the processor can be a central processing unit, other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0074] 2. The present application also provides a readable storage medium, which can be an internal storage unit of the electronic device described in the foregoing embodiment, such as the hard disk or memory of the computer device. The readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card, a secure digital card, etc. equipped on the electronic device.

[0075] 3. The present application also provides a computer program product, including a computer program or instructions, and when the computer program or instructions are executed by the processor, the defect detection method with a definite reference of the present invention is implemented.

[0076] The present invention is not limited to the above embodiments. Without departing from the essence of the present invention, any obvious improvements, substitutions or deformations that those skilled in the art can make belong to the protection scope of the present invention.

Claims

1. A defect detection method with a certain reference, characterized in that: Step 1: Build the detection primitive library: Step 1.1, collecting an image of a defect-free workpiece as a template image; Step 1.2, extracting the three primary color layer images of the template image; Step 1.3, extracting the local peak values ​​of the three base color layer images respectively; Step 1.4, select one local peak from each of the three base color images for fusion, and summarize all fusion results to form a threshold set; Step 1.5, based on each threshold in the threshold set, the grayscale image of the template image is binarized to obtain a detection primitive template image, and all detection primitive template images are aggregated to obtain a detection primitive library; Step 2: Extraction of detection primitives from sample images: Step 2.1, collecting an image of the workpiece to be inspected as a test sample image; Step 2.2, based on each threshold value in the threshold set, binarize the grayscale image of the sample image to obtain a detection primitive sample image; Step 3: Subtract each detection primitive inspection image from the corresponding detection primitive template image in the detection primitive library to obtain the defect detection result.

2. The defect detection method with a certain reference according to claim 1, characterized in that: Before binarization of the sample image, the sample image is first mapped to the template image coordinate system and aligned with the template image.

3. The defect detection method with a certain reference according to claim 2, characterized in that: The mapping relationship between the test sample image and the template image is: Where M is the mapping transformation matrix, a, b, c, d, e, f are the elements in the mapping transformation matrix, (x, y) represents the pixel coordinates in the template image, (x ′ ,y ′ ) represents the pixel coordinates corresponding to (x, y) in the sample image; By selecting several groups of Mark points between the template image and the sample image, the least square method is used to solve the mapping transformation matrix M based on the Mark point coordinates, and then the mapping transformation matrix M is used to map all the pixel points on the sample image to the template image coordinate system; After mapping, the position of the sample image is aligned with the template image through a shift operation.

4. The defect detection method with a certain reference according to claim 1, characterized in that: Used to detect PCB board surface defects, the workpiece is a PCB board.

5. The defect detection method with a certain reference according to claim 4, characterized in that: Used to detect defects on PCB boards, such as pads, copper wires, through holes, blind holes, and silk screen locations.

6. The defect detection method with a certain reference according to claim 3, characterized in that: The number of the Mark points is 3.

7. The defect detection method with a certain reference according to claim 1, characterized in that: The method for extracting the local peak value is: firstly constructing a histogram for the base color layer image, and then extracting the local peak value from the histogram.

8. A computer device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the defect detection method with a definite reference as claimed in any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor is caused to execute the defect detection method with a certain reference according to any one of claims 1 to 7.

10. A computer program product, characterized in that: The method comprises a computer program, which, when executed by a processor, implements the defect detection method with a definite reference as claimed in any one of claims 1 to 7.

Images