Defect detection method and device without certain reference, medium and program product

Abstract

The invention provides a defect detection method and device without certain reference, a medium and a program product, and the detection method comprises the steps: collecting an image of a defect-free workpiece as a template image, and converting the template image into a spectrogram; collecting an image of a to-be-detected workpiece as a sample detection image, and converting the sample detection image into a spectrogram; making a difference between the spectrogram of the template image and the spectrogram of the sample detection image to obtain a residual spectrogram; and after the residual spectrogram is subjected to filtering processing, a detection result graph is obtained through inverse Fourier transform. According to the method, template image frequency spectrum elimination and residual frequency spectrum component reconstruction are combined, so that the defect detection problem of manufacturing defects such as non-uniform green oil coating, scratches and smudginess on a PCB (Printed Circuit Board) without a deterministic reference detection primitive template is successfully solved, and high-precision detection of the defect of the deterministic reference primitive is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and particularly relates to a defect detection method, device, medium, and program product without a definite reference. Background Art

[0002] There are various and highly unknown defects on the PCB board surface, which pose great challenges to automated intelligent detection. On the one hand, since new types of defects may continuously appear, the corresponding detection algorithms are required to have a certain degree of scalability and self-adaptability. On the other hand, the small inter-class difference is also a problem that cannot be ignored. In some cases, different types of defects may have very similar characteristics, making it difficult for detection algorithms to accurately distinguish them. In such cases, the algorithm requires more refined feature extraction and classification capabilities to improve the detection accuracy.

[0003] Currently, the more popular detection algorithms are based on deep learning technology. However, due to the large amount of training data and computing resources required by deep learning technology, as well as the complex network structure and parameter tuning, it is difficult to achieve generality and efficiency in actual applications. Therefore, the detection methods based on deep learning generally have poor generality and high algorithm complexity.

[0004] Especially when dealing with the defect detection problem without a definite detection primitive as a reference template, such as manufacturing defects like uneven solder mask coating, scratches, and dirt, these defects have great randomness and shape anisotropy. Coupled with the very complex various graphic elements on the PCB board, it is very difficult to isolate such defects, further exacerbating the difficulty of defect detection. Summary of the Invention

[0005] Aiming at the deficiencies in the prior art, the present invention provides a defect detection method, device, medium, and program product without a definite reference to solve the problem of high-precision detection of defects without a definite reference primitive.

[0006] The present invention achieves the above technical objectives through the following technical means.

[0007] A defect detection method without a definite reference:

[0008] Collect an image of a defect-free workpiece as a template image, and convert the template image into a spectrogram;

[0009] Collect an image of the workpiece to be detected as a sample image, and convert the sample image into a spectrogram;

[0010] Subtract the spectrogram of the template image from the spectrogram of the sample image to obtain a residual spectrogram;

[0011] After filtering the residual spectrogram, perform an inverse Fourier transform to obtain a detection result image.

[0012] Further, the conversion method of the spectrogram is as follows: perform grayscale processing on the template image or the test sample image to obtain a grayscale image, and then perform two-dimensional Fourier transform and frequency centering processing on the grayscale image in sequence to obtain a spectrogram.

[0013] Further,

[0014] Adopt the weighted grayscale method, and the grayscale formula is:

[0015] H(x,y) = λ1·R(x,y) + λ2·G(x,y) + λ3·B(x,y)

[0016] In the formula, H(x,y) represents the grayscale image, R(x,y), G(x,y), and B(x,y) respectively represent the layer images of the R, G, and B channels of the original color image, (x,y) represents the pixel coordinates in the image, and λ1, λ2, and λ3 are the weight coefficients of the R, G, and B channels respectively, and λ1 + λ2 + λ3 = 1;

[0017] The formula for the two-dimensional Fourier transform is:

[0018]

[0019] In the formula, u and v respectively represent the frequency components in the x direction and the y direction, j is the imaginary unit, and e is the natural constant.

[0020] Further, before converting the test sample image into a spectrogram, first map the test sample image to the coordinate system of the template image and align it with the template image.

[0021] Further, the mapping relationship between the test sample image and the template image is:

[0022]

[0023] In the formula, M is the mapping transformation matrix, a, n, c, d, e, and f are the elements in the mapping transformation matrix, (x,y) represents the pixel coordinates in the template image, and (x ′ ,y ′ ) represents the pixel coordinates in the test sample image corresponding to (x,y);

[0024] By selecting several groups of Mark points between the template image and the test sample image, based on the Mark point coordinates, use the least squares method to solve the mapping transformation matrix M, and then use the mapping transformation matrix M to map all the pixel points on the test sample image to the coordinate system of the template image;

[0025] After mapping, perform a shift operation to align the positions of the test sample image and the template image.

[0026] Further, for detecting defects on the PCB board surface, the workpiece is a PCB board.

[0027] Further,

[0028] Perform low-pass filtering on the residual spectrogram, and then perform inverse Fourier transform to obtain the detection result diagram of dirt and solder mask defects;

[0029] Perform high-pass filtering on the residual spectrogram, and then perform inverse Fourier transform to obtain the detection result diagram of scratch defects.

[0030] A computer device includes a memory and a processor;

[0031] The memory is used to store computer programs;

[0032] The processor is used to execute the computer program and implement the above-mentioned defect detection method without a definite reference when executing the computer program.

[0033] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the above-mentioned defect detection method without a definite reference.

[0034] A computer program product includes a computer program, and when the computer program is executed by a processor, the above-mentioned defect detection method without a definite reference is implemented.

[0035] The beneficial effects of the present invention are as follows:

[0036] (1) The present invention provides a defect detection method without a definite reference. By combining template image spectrum elimination and residual spectrum component reconstruction, it successfully solves the defect detection problem of manufacturing defects such as uneven solder mask coating, scratches, and dirt on PCB boards, which lack a definite reference detection primitive template, and realizes high-precision detection of defects without a definite reference primitive.

[0037] (2) The present invention realizes precise alignment of the test sample image and the template image by combining precise reset in the image domain, laying a foundation for defect analysis and defect localization in the frequency domain. The filter design in the frequency domain is based on in-depth analysis and statistics of defects, improving the recognition accuracy of defects in the image domain.

[0038] (3) The present invention only performs frequency domain analysis on images, has high defect detection efficiency, and has low requirements for the performance of detection equipment, making it suitable for industrial scenario applications. Description of the Drawings

[0039] Figure 1 It is an example of the defect detection process of the present invention;

[0040] Figure 2 It is a flowchart of the defect detection of the present invention. Detailed implementation manners

[0041] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present invention, and should not be construed as a limitation to the present invention.

[0042] The surface defects on the bare PCB board can be divided into "defects with a definite reference" and "defects without a definite reference" according to the defect occurrence positions. Among them:

[0043] Defects with a definite reference: refer to the defects occurring on these "detection elements" such as pads, copper wires, vias / blind vias, silk screens, etc. Such defects can be detected by comparing with a reference template based on the standard qualified detection elements.

[0044] Defects without a definite reference: such as scratches, solder mask, dirt, etc., which are defects with indefinite occurrence positions.

[0045] In this embodiment, the following detection scheme is proposed for the above-mentioned defects without a definite reference:

[0046] I. Detection method for defects without a definite reference

[0047] Refer to Figure 1 and Figure 2 As shown, this embodiment includes the following processing procedures:

[0048] 1. Acquisition of the spectrum of the template image

[0049] 1) Acquisition of the template image: Acquire an image of a defect-free bare PCB board as the template image.

[0050] 2) Mark point marking: Use the method of manual marking to mark 3 Mark points (reference points) on the template image as the reset reference points for the subsequent sample images.

[0051] 3) Grayscale conversion of the template image: Use the weighted grayscale conversion method to fuse and weighted average the three channels of the color image to obtain the grayscale image of the template image. The grayscale conversion formula is:

[0052] H(x,y) = λ1·R(x,y) + λ2·G(x,y) + λ3·B(x,y)

[0053] In the formula, H(x,y) represents the grayscale image, R(x,y), G(x,y), and B(x,y) respectively represent the layer images of the three channels R (red), G (green), and B (blue) of the original color image, (x,y) represents the pixel coordinates in the image, and λ1, λ2, and λ3 are the weight coefficients of the three channels R, G, and B, and λ1 + λ2 + λ3 = 1.

[0054] 4) Obtain the spectrogram of the template image:

[0055] First, perform two-dimensional Fourier transform on the grayscale image of the template image. The transformation formula is as follows:

[0056]

[0057] In the formula, u and v represent the frequency components in the x and y directions respectively, j is the imaginary unit, and e is the natural constant.

[0058] After that, perform frequency centering on the transformed spectrogram F(u, v) to obtain the final required spectrogram of the template image.

[0059] 2. Obtaining the spectrum of the sample image

[0060] 1) Acquisition of the sample image: For the PCB that needs to be defect-detected, collect its bare board image, denoted as the sample image.

[0061] 2) Extraction of Mark points: Based on the Mark points marked on the template image, extract 3 Mark points at the same positions on the PCB board in the sample image.

[0062] 3) Mapping and alignment of the sample image: Based on the selected Mark points, map the sample image to the coordinate system of the template image. The mapping relationship is as follows:

[0063]

[0064] In the formula, M is the mapping transformation matrix, a, b, c, d, e, f are the elements in the mapping transformation matrix, (x, y) represents the pixel coordinates in the template image, and (x ′ , y ′ ) represents the pixel coordinates corresponding to (x, y) in the sample image. By substituting the coordinates of three groups of corresponding Mark points on the template image and the sample image, the mapping transformation matrix M is solved using the least squares method. Then, using the mapping transformation matrix M, all pixel points on the sample image are mapped to the coordinate system of the template image.

[0065] After mapping, the sample image may still have a positional deviation from the template image. The sample image and the template image can be aligned in position through a shift operation.

[0066] 4) Grayscale processing of the sample image: Similarly, use the weighted grayscale method to perform grayscale processing on the sample image to obtain the grayscale image of the sample image.

[0067] 5) Obtain the spectrogram of the sample image: Perform two-dimensional Fourier transform and spectrum centering on the grayscale image of the sample image in sequence to obtain the spectrogram of the sample image.

[0068] 3. Defect Detection

[0069] 1) Obtain the residual spectrum: Subtract the spectrum diagram of the template image from the spectrum diagram of the sample image to obtain the residual spectrum diagram. For defects such as uneven green oil coating, scratches, and dirt that have no definite reference detection primitive template, the defect information generated will be saved in the residual spectrum. Among them:

[0070] a. Detection of dirt and green oil defects: Dirt and green oil defects exhibit low-frequency characteristics in the spectrum image. Therefore, in this embodiment, a Gaussian low-pass filter is used to perform low-pass filtering on the residual spectrum diagram to filter out the high-frequency components in the spectrum and obtain a low-frequency residual spectrum diagram containing dirt and green oil defect information. The transfer function of the Gaussian low-pass filter is:

[0071]

[0072] In the formula, D(u, v) is the distance from the frequency domain point (u, v) to the frequency domain center, and σ is the standard deviation of the Gaussian distribution.

[0073] b. Detection of scratch defects: Scratch defects exhibit high-frequency characteristics in the spectrum image. Therefore, in this embodiment, a Gaussian high-pass filter is used to perform high-pass filtering on the residual spectrum diagram to filter out the low-frequency components in the spectrum and obtain a high-frequency residual spectrum diagram containing scratch defect information. The transfer function of the Gaussian high-pass filter is:

[0074]

[0075] 2) Inverse Fourier transform of the filtered image: Perform inverse Fourier transform on the residual frequency diagrams after the above low-pass filtering and high-pass filtering processes respectively to obtain the defect detection result diagrams. Among them, perform inverse Fourier transform on the low-frequency residual spectrum diagram to obtain the detection result diagram of dirt and green oil defects; perform inverse Fourier transform on the high-frequency residual spectrum diagram to obtain the detection result diagram of scratch defects. The formula for the inverse Fourier transform is as follows:

[0076]

[0077] In the formula, C(u, v) is the residual frequency diagram after low-pass filtering or high-pass filtering, and D(x, y) is the detection result diagram obtained by inverse Fourier transform.

[0078] II. Device, Storage Medium, Program Product

[0079] 1. Based on the same inventive concept as the above defect detection method without a definite reference, the present application also provides an electronic device, which includes a processor and a memory. The memory stores computer-readable code. Among them, when the computer-readable code is executed by the processor, the defect detection method without a definite reference of the present invention is implemented.

[0080] Among them, the memory includes a non-volatile storage medium and an internal memory; the non-volatile storage medium can store an operating system and computer-readable code. The computer-readable code includes program instructions, which when executed can cause the processor to execute a defect detection method without a definite reference. The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory provides an environment for the operation of the computer-readable code in the non-volatile storage medium, and when the computer-readable code is executed by the processor, it can cause the processor to execute a defect detection method without a definite reference.

[0081] It should be understood that the processor can be a central processing unit, other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0082] 2. The present application also provides a readable storage medium, which can be an internal storage unit of the electronic device described in the foregoing embodiment, such as the hard disk or memory of the computer device. The readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card, a secure digital card, etc. equipped on the electronic device.

[0083] 3. The present application also provides a computer program product, including a computer program or instructions, which when executed by the processor implement the defect detection method without a definite reference of the present invention.

[0084] The present invention is not limited to the above embodiments. Without departing from the essential content of the present invention, any obvious improvements, substitutions or deformations that those skilled in the art can make all belong to the protection scope of the present invention.

Claims

1. A defect detection method without a definite reference, characterized in that: An image of a defect-free workpiece is collected as a template image, and the template image is converted into a frequency spectrum; Collecting an image of the workpiece to be inspected as a test sample image, and converting the test sample image into a frequency spectrum; Subtract the spectrum of the template image from the spectrum of the sample image to obtain a residual spectrum; After filtering the residual spectrum, the detection result graph is obtained through inverse Fourier transform.

2. The defect detection method without a definite reference according to claim 1, characterized in that: The conversion method of the spectrum map is: graying the template image or the sample image to obtain a grayscale map, and then performing two-dimensional Fourier transform and frequency centering processing on the grayscale map in sequence to obtain a spectrum map.

3. The defect detection method without a definite reference according to claim 2, characterized in that: The weighted grayscale method is used, and the grayscale formula is: H(x,y)=λ1·R(x,y)+λ2·G(x,y)+λ3·B(x,y) Where H(x,y) represents the grayscale image, R(x,y), G(x,y), and B(x,y) represent the layered images of the three channels of the original color image R, G, and B, respectively, (x,y) represents the coordinates of the pixel point in the image, λ1, λ2, and λ3 are the weight coefficients of the three channels of R, G, and B, respectively, and λ1+λ2+λ3=1; The formula for the two-dimensional Fourier transform is: Where u and v represent the frequency components in the x-direction and y-direction respectively, j is an imaginary unit, and e is a natural constant.

4. The defect detection method without a definite reference according to claim 1, characterized in that: Before converting the sample image into a spectrum diagram, the sample image is first mapped to the template image coordinate system and aligned with the template image.

5. The defect detection method without a definite reference according to claim 4, characterized in that: The mapping relationship between the test sample image and the template image is: Where M is the mapping transformation matrix, a, b, c, d, e, f are the elements in the mapping transformation matrix, (x, y) represents the pixel coordinates in the template image, (x ′ ,y ′ ) represents the pixel coordinates corresponding to (x, y) in the sample image; By selecting several groups of Mark points between the template image and the sample image, the least square method is used to solve the mapping transformation matrix M based on the Mark point coordinates, and then the mapping transformation matrix M is used to map all the pixel points on the sample image to the template image coordinate system; After mapping, the position of the sample image is aligned with the template image through a shift operation.

6. The defect detection method without a definite reference according to claim 1, characterized in that: Used to detect PCB board surface defects, the workpiece is a PCB board.

7. The defect detection method without a definite reference according to claim 6, characterized in that: The residual spectrum is low-pass filtered and then inversely transformed by Fourier transform to obtain the detection result of dirt and green oil defects; The residual spectrum is processed by high-pass filtering and then inverse Fourier transform to obtain the detection result of scratch defects.

8. A computer device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the defect detection method without a definite reference as claimed in any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor is caused to execute the defect detection method without a definite reference according to any one of claims 1 to 7.

10. A computer program product, characterized in that: The method comprises a computer program, which, when executed by a processor, implements the defect detection method without a definite reference as claimed in any one of claims 1 to 7.

Images