Line image detection method and automatic optical detection system comprising same

By binarizing and thinning backbone images, a point cloud is established to judge feature points, which solves the problem of dependence on the parent image in the traditional automatic optical detection system, and achieves efficient self-detection and reduces computing consumption.

CN119007092APending Publication Date: 2024-11-22UTECHZONE CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202311325209.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-22
Filing Date
2023-10-13
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Traditional automatic optical detection systems require reliance on computer-assisted manufacturing archives or master images for image correction, resulting in low detection efficiency and high computational consumption, and the inability to efficiently detect self-lines of the images to be tested.

Method used

By obtaining the image to be tested, performing binary processing and backbone thinning processing, establishing point clouds and judging the line segment angle and type of feature points to judge defects, realizing self-detection based on the image to be tested itself, eliminating the correctness Reliance of the parent diagram.

Benefits of technology

It improves detection efficiency, reduces operation consumption, realizes defect detection without high dependence on the parent map, and improves detection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007092A_ABST
    Figure CN119007092A_ABST
Patent Text Reader

Abstract

The invention discloses a line image detection method and an automatic optical detection system comprising the same. The line image detection method comprises the following steps: obtaining a to-be-detected image of a to-be-detected sample; binarizing the to-be-detected image and finding out a detection area; performing backbone thinning processing on the detection area to obtain a backbone line image; establishing a plurality of point clouds of the to-be-detected image according to the backbone line image, and judging whether the feature points of the corresponding point clouds are flaws or not according to line segment angle data and point type data to which each point cloud belongs. Basic flaw detection can be carried out based on the to-be-detected image, self-inspection can be carried out according to the original image without highly depending on the parent image, and the correction and alignment time is saved, so that the detection efficiency is improved, and the operation consumption is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a circuit image detection method and an automatic optical detection system including the same, and more particularly to a circuit image detection method based on an image to be detected that is sufficient for self-circuit detection and an automatic optical detection system including the same. Background Art

[0002] Automated Optical Inspection (AOI) is a technology that uses automated equipment to perform optical inspection on workpieces. It is often used in the manufacturing process of printed circuit boards (PCBs), surface mount technology (SMT), panels, chips or other similar products.

[0003] Generally speaking, the automatic optical inspection system is mainly composed of image detection equipment, image processing software, mechanical components and control systems. During the inspection process, the automatic optical inspection system will place the inspection object in the inspection area, use the image capture device and auxiliary light source to capture the image of the workpiece surface, and then use the image processing software to perform image analysis, defect detection, shape comparison and other operations to determine whether the product meets the set standards. If there are defects, the system will generate defect detection results and perform subsequent operations according to the settings, such as classifying, marking, and removing unqualified products, or re-inspecting based on defects.

[0004] In the past, when performing automatic optical inspection, it was necessary to rely on computer-aided manufacturing files (CAM) or master images as comparison samples to perform inspection. This means that when performing defect inspection, the image must be corrected based on the position of the CAM file or master image, so that the coordinates of the image to be tested and the CAM file or master image can be synchronized before inspection can be performed, thereby consuming a lot of computing performance and time to complete image alignment and correction. Summary of the invention

[0005] The main purpose of the present invention is to provide a line image detection method, including: obtaining an image to be tested of a sample to be tested; binarizing the image to be tested and finding the detection area; performing backbone thinning processing on the detection area to obtain a backbone line image; establishing multiple point clouds of the image to be tested based on the backbone line image and judging whether the feature point of the corresponding point cloud is a defect based on the line segment angle data and the point type data belonging to each point cloud.

[0006] Another object of the present invention is to provide an automatic optical inspection device, comprising a detection platform, at least one image capture device and a detection device. The detection platform is used to set a sample to be tested. The image capture device is set on one side of the detection platform and is used to photograph the sample to be tested on the detection platform. The detection device is connected to the image capture device, and after the image capture device obtains an image of the sample to be tested, the method described above is executed to complete defect detection.

[0007] Therefore, the present invention can perform basic defect detection based on the image to be tested, without being highly dependent on the master image, and can perform self-inspection based on the original image, thus saving the time for calibration and alignment, thereby improving the efficiency of detection and reducing computing consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 Schematic diagram of the automatic optical inspection system of the present invention.

[0009] Figure 2 It is a schematic diagram of the flow of the line image detection method in the present invention.

[0010] Figure 3 Schematic diagram of image processing of the image to be tested in the present invention (I).

[0011] Figure 4 Schematic diagram (II) of image processing of the image to be tested in the present invention.

[0012] Figure 5 Schematic diagram (III) of image processing of the image to be tested in the present invention.

[0013] Figure 6 Schematic diagram (IV) of image processing of the image to be tested in the present invention.

[0014] Figure 7 Schematic diagram (V) of image processing of the image to be tested in the present invention.

[0015] Figure 8 Schematic diagram (VI) of image processing of the image to be tested in the present invention.

[0016] Fig. 9 Schematic diagram (VII) of image processing of the image to be tested in the present invention.

[0017] Fig.10 Schematic diagram (VIII) of image processing of the image to be tested in the present invention.

[0018] Fig.11 Schematic diagram (IX) of image processing of the image to be tested in the present invention.

[0019] The following are marked in the figure:

[0020] 100 Automatic Optical Inspection System

[0021] 10. Detection Platform

[0022] 20 Image capture device

[0023] 30 Auxiliary light source device

[0024] 40 Detection device

[0025] 41 Processor

[0026] 42 Storage Units

[0027] WP Samples to be tested

[0028] GF Test image

[0029] Image after TH TOPHAT processing

[0030] BH Binarized Image

[0031] IM1 image after erosion

[0032] Image after IM2 filtering

[0033] Q1 Object

[0034] D1 Foreign matter

[0035] A1 Feature Point

[0036] A2 Feature Points

[0037] D2 Defect DETAILED DESCRIPTION

[0038] The detailed description and technical content of the present invention are described as follows with reference to the accompanying drawings. The "one side" mentioned in the present invention can be the upper side, lower side, left side, right side, front side, rear side of the corresponding object, or set at any adjacent position of the object, or directly or indirectly connected to the object, etc., which is not limited in the present invention.

[0039] The following is a description of one embodiment of the present invention. For the hardware device of the present invention, please refer to Figure 1 , is a schematic diagram of the automatic optical inspection system in the present invention, as shown in the figure: This embodiment discloses an automatic optical inspection system 100, including a detection platform 10, an image capture device 20 arranged corresponding to the detection platform 10, an auxiliary light source device 30 arranged corresponding to the detection platform 10, used to provide auxiliary lighting for a sample WP to be tested, and a detection device 40 electrically connected or coupled to the image capture device 20.

[0040] The detection platform 10 is used to set the sample WP to be tested. In one embodiment, the detection platform 10 can be simply a setting platform for placing the sample WP to be tested for the image capture device 20 to shoot. In another embodiment, the detection platform 10 can be a vacuum adsorption platform for adsorbing the sample WP to be tested and leveling the sample WP to be tested, so that the image capture device 20 can shoot the leveled sample WP to be tested. In another embodiment, the detection platform 10 can be, for example, but not limited to, a linear stage, a conveyor belt, or a transfer device, and the sample WP to be tested is moved into the shooting range of the image capture device 20 to shoot the sample WP to be tested. The changes in these embodiments are not within the scope of the present invention.

[0041] The image capture device 20 is disposed on one side of the detection platform 10 and is used to photograph the sample WP on the detection platform 10 to obtain a detection image GF (eg, Figure 3 In one embodiment, the image capture device 20 may be, for example, but not limited to, an area scan camera or a line scan camera, which is not limited in the present invention. The sample WP may be, for example, but not limited to, a printed circuit board (PCB), a flexible printed circuit board (FPC), or other samples with circuits disposed thereon, which is not limited in the present invention.

[0042] The auxiliary light source device 30 is disposed on any side of the detection platform 10 to provide auxiliary lighting for the sample WP to be tested. In one embodiment, the auxiliary light source device 30 may be, for example, but not limited to providing a backlight source, an ambient light source, a ring light source, a side light source, a coaxial light source, or a parallel light source, etc., which are not limited in the present invention. Based on the spectrum difference of light, the auxiliary light source device 30 may be, for example, but not limited to, a white light source, a red light source, a green light source, a blue light source, an excitation light source, or other lighting light sources, which are not limited in the present invention.

[0043] The detection device 40 is connected or coupled to the image capture device 20, and after obtaining the image GF of the sample WP, the detection device 40 performs defect detection on the image GF. In one embodiment, the detection device 40 is directly connected to the image capture device 20, and after the image capture device 20 photographs the sample WP and performs preprocessing, the image GF is directly transmitted to the detection device 40 via a physical line or wirelessly. In another embodiment, the image capture device 20 can be connected to a centralized image storage device (such as a server for storing images), and the detection device 40 is connected to the centralized image storage device and performs defect analysis on the stored image GF. The variations of these embodiments are not limited in the present invention.

[0044] In one embodiment, the detection device 40 includes a processor 41 and a storage unit 42 connected to the processor 41. The processor 41 can be used to load the storage unit 42 to execute the program stored in the storage unit 42. In another embodiment, the processor 41 and the storage unit 42 can together constitute a computer or processor, such as a personal computer, a workstation, a host computer or other types of computers or processors, and their types are not limited here. In one embodiment, the processor 41 can be coupled to the storage unit 42. The processor 41 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessors (Microprocessors), digital signal processors (Digital Signal Processors, DSPs), programmable controllers, application-specific integrated circuits (Application Specific Integrated Circuits, ASICs), programmable logic devices (Programmable Logic Devices, PLDs) or other similar devices or combinations of these devices.

[0045] For detailed procedures of self-line detection, please refer to the following Figure 2 , which is a flow chart of the line image detection method in the present invention. Regarding the steps of image processing, for the sake of clarity, it will be supplemented by Figures 3 to 11 For illustration, the image processing schematic diagrams (I) to (IX) of the image to be tested in the present invention are shown respectively, as shown in the figures: the circuit image processing method disclosed in this embodiment mainly includes the following steps.

[0046] Step S01: Obtaining a test image of the test sample. The test sample WP is photographed by the image capture device 20 to obtain a test image GF of the test sample WP. The photographed test image GF is as follows: Figure 3 When photographing the sample WP, in order to enhance the circuit characteristics, a specific light source may be provided based on the material or structure of the circuit to enhance the circuit image in the image GF. The selection and configuration of these light sources are not limited in the present invention.

[0047] Step S02: Binarize the image to be tested and find the detection area. When the detection device 40 receives the image to be tested GF, the image to be tested GF is first binarized to obtain a binary image. In one embodiment, before the image to be tested GF is binarized, the contrast can be improved first, and the optimal binarization threshold can be quickly found through ISO, so that the image to be tested GF is binarized by the binarization threshold found by ISO, so that the image to be tested GF is easier to be detected and analyzed. After obtaining the binary image, the detection device 40 finds the detection area in the binary image, and the detection area includes but is not limited to, for example, mainly through BLOB processing (Binary Large Object) and TOPHAT processing (Top-hat Transform). BLOB processing is used to separate the connected areas in the image to be tested GF, and calculate the area and / or length and width of the connected areas marked by BLOB in the binary image, and delete objects that do not match the area and / or size by setting a threshold. TOPHAT processing is mainly used to filter out thin and narrow line areas. Most line segment features have thin and narrow features that are larger than object features, thereby better highlighting the shape and structure of the destination area.

[0048] In one embodiment, the step of searching for the detection area includes: firstly, the binary image is processed by BLOB to mark the connected area as an object, and the objects whose area and / or length and width in the binary image do not reach the threshold are filtered out; then, the binary image is processed by TOPHAT to obtain a TOPHAT-processed image TH, such as Figure 4 As shown in the figure, the white area is the area processed by TOPHAT, and at least one narrow line area is obtained by TOPHAT. Then, the binary image is processed by Scale-Add, and the difference between the ratio of the object and at least one narrow area is used (for example, any object with a ratio too large relative to at least one narrow area is deleted) to find the detection object that needs to be retained as the detection area to generate a new binary image BH, as shown in Figure 5 As shown, the binarized image only retains the thin and narrow objects, and the pixels with larger values ​​in the new binary image BH represent the detection area (white area), while the pixels with smaller values ​​represent the background area (black area). It should be noted here that the pixel inversion process in the binary image to make the detection area a black area and the background area a white area is the choice of those skilled in the art, and these simple changes do not deviate from the scope of protection of the present invention.

[0049] Step S03: Next, the detection area is subjected to backbone thinning processing to obtain a backbone line image. Figure 6As shown in FIG. 1 , after obtaining the detection area, the detection device 40 further performs backbone thinning on the image in the detection area and obtains a backbone line image. The detection device 40 can set the number of erosions and distinguish the object area (gray) and the line segment backbone area (white) through the internal points and the backbone points. Figure 6 (a) is a portion of the image IM1 after erosion. Next, the detection device 40 can determine whether it is a line again by comparing the number of object areas and line areas by partitioning. If it is determined that it is not a line, it will be filtered out. Figure 6 (b) is the image IM1-the filtered image IM2. Regarding the number of erosions, in general, if the line width is 4 units (for example, in pixels), the number of erosions can be set to 4, so that the end object (circular PAD) is retained. The actual number of erosions is not limited to the present invention and can be changed according to actual processing requirements.

[0050] Step S04: Next, multiple point clouds of the image to be tested are established based on the backbone line image. Next, after step S03, the detection device 40 establishes multiple point clouds of the image to be tested GF based on the backbone line image; wherein each point cloud includes the line segment angle data and point type data to which it belongs. The density of the point cloud can be the largest, for example, one feature point corresponds to one pixel point, or the point cloud is arranged at equal distances, which is not limited in the present invention. When establishing the point cloud, the line segment angle data and point type data to which it belongs are recorded for each feature point for use in the subsequent defect identification. In one embodiment, the line segment angle data includes 0 degrees, 45 degrees, 90 degrees and 135 degrees. The line segment angle data is basically determined based on the line layout of the sample to be tested WP. Depending on the different line layouts, the recorded line segment angle data can also be adjusted accordingly, which is not limited in the present invention. In one embodiment, the point type data can be, for example, but not limited to, endpoints, connection points, bend points and corner points, etc., which are not limited in the present invention.

[0051] Step S05: Finally, based on the line segment angle data and point type data of each point cloud, determine whether the feature point of the corresponding point cloud is a defect. There are several methods for determining whether the feature point of the corresponding point cloud is a defect. Please refer to the following Figures 7 to 11 , respectively, with accompanying drawings for explanation.

[0052] Method 1: When a feature point corresponding to a point cloud is determined as an endpoint, if the endpoint is not connected to any object, it is determined to be a defect. Figure 7 As shown in FIG. 1 , the feature points marked as endpoints in the point cloud will be connected to the corresponding object Q1; if they are not connected to the corresponding object Q1 or have two directions on the backbone line, such as Figure 8 As shown, it is very likely that it is a foreign object D1 in the line segment, and the corresponding feature point will be determined as a defect.

[0053] Method 2: When the feature points corresponding to the point cloud have three directions, it is determined to be a connection point defect. Specifically, Fig. 9 As shown, in the point cloud, when a feature point has three directions, such as the backbone line connected to feature point A1 has three directions of left, right, and bottom, and the backbone line connected to feature point A2 has three directions of left, right, and top, generally such feature points are very likely to be defects, such as open circuits or concave defects.

[0054] Method 3: Next, based on the line segment angle of the feature point corresponding to the point cloud, a mask of the corresponding angle is run, and the feature point highlighted after running the mask is determined to be a quasi-defect. In one embodiment, the angle mask can be, for example, a Sobel mask, which is not limited in the present invention. Specifically, the mask includes four types: vertical mask, horizontal mask, 45-degree mask, and 135-degree mask. The vertical mask is as follows: Fig.10 As shown in (a), the masked kernel matrix has the first row, first to third columns all -1, the second row, first to third columns all 0, and the third row, first to third columns all 1. The horizontal mask is Fig.10 As shown in (b), the first column, first to third rows of the masked kernel matrix are all -1, the first to third rows of the second column are all 0, and the first to third rows of the third column are all 1. The 135-degree mask is as follows: Fig.10 As shown in (c), the masked kernel matrix has the first row and third column, the second row and second column, and the third row and first column are all 1, the second row and third column, and the third row and second column are all 1.5, the third row and third column are 0, the third row and fourth column, and the fourth row and third column are all -1.5, and the third row and fifth column, the fourth row and fourth column, and the fifth row and third column are all -1. The 45-degree mask is as follows: Fig.10 As shown in (d), the masked kernel matrix has the first row, third column, second row, fourth column, and third row, fifth column all being 1, the second row, third column, and third row, fourth column all being 1.5, the third row, third column all being 0, the third row, second column, and fourth row, third column all being -1.5, the third row, first column, fourth row, second column, and fifth row, third column all being -1. By running the above mask in the direction of the backbone line, the detection gradient strength can be obtained, as Fig.11 As shown, when the gradient intensity is combined with the backbone line, it can be seen that the lighter defect D2 can also be highlighted.

[0055] When the vertical mask is running, the horizontal line segments in the backbone line will be shielded, and other types of line segments will be highlighted. Therefore, when the feature points highlighted by the vertical mask are marked as 90 degrees, they can be determined as quasi-defects; when the horizontal mask is running, the vertical line segments in the backbone line will be shielded, and other types of line segments will be highlighted. Therefore, when the feature points highlighted by the horizontal mask are marked as 0 degrees, they can be determined as quasi-defects; when the 45-degree mask is running, the 135-degree line segments in the backbone line will be shielded, and other types of line segments will be highlighted. Therefore, when the feature points highlighted by the 45-degree mask are marked as 45 degrees, they can be determined as quasi-defects; when the 135-degree mask is running, the 45-degree line segments in the image to be tested will be shielded, and other types of line segments will be highlighted. Therefore, when the feature points highlighted by the 135-degree mask are marked as 135 degrees, they can be determined as quasi-defects.

[0056] In method three, when a feature point or a set of feature points is determined to be a quasi-defect, the detection device 40 then calculates the grayscale average value and standard deviation of the entire line segment object to which the feature point determined to be a quasi-defect belongs. When the grayscale value of the feature point is higher than the grayscale average value of the entire line segment plus a preset multiple of the standard deviation, the feature point is determined to be a defect. If not, the feature point is determined not to be a defect.

[0057] Step S06: Frame the area around the selected defect as the region of interest, and inversely match the region of interest with the mother image. Feature points that are not located are confirmed as final defects. Specifically, through the above-mentioned methods one to three, a preset distance around the defect that is finally selected is framed as the region of interest, and the framed region of interest is inversely matched with the mother image (Inverse Matching). Feature points that are not located (that is, features that cannot find similar blocks in the mother image) are confirmed as final defects. It is worth noting that step S06 is only a review mechanism. In other embodiments, step S06 can be omitted, and only executing steps S01 to S05 can also complete the image detection of the circuit of the present invention, which is not limited in the present invention.

[0058] In summary, the present invention can perform defect detection based on the image to be tested, without the need for a CAM file and no longer highly dependent on the master image. It can be directly checked based on the original image, saving the time for calibration and alignment, thereby improving the efficiency of detection and reducing computing consumption.

[0059] The present invention has been described in detail above. The above is only one preferred embodiment of the present invention and should not be used to limit the scope of the present invention. That is, all equivalent changes and modifications made according to the scope of the patent application of the present invention should still fall within the scope of the patent of the present invention.

Claims

1. A line image detection method, characterized in that: include: Obtaining an image of a sample to be tested; Binarize the image to be tested and find a detection area; Performing backbone thinning processing on the detection area to obtain a backbone line image; Establishing a plurality of point clouds of the image to be tested according to the backbone line image; as well as According to the line segment angle data and the point type data of each point cloud, it is determined whether a feature point corresponding to the point cloud is a defect.

2. The line image detection method according to claim 1, characterized in that: When the feature point corresponding to the point cloud is determined to be a defect, a region of interest is selected around the defect, and the region of interest is reversely compared with a mother image, and the feature point that is not located is confirmed as the final defect.

3. The line image detection method according to claim 1, characterized in that: Before binarizing the image to be tested, a corresponding binarization threshold is found through ISO.

4. The line image detection method according to claim 1, characterized in that: The image to be tested is processed by BLOB and TOPHAT to highlight line features so as to find the detection area.

5. The line image detection method according to claim 4, characterized in that: The step of searching the detection zone comprises: The image to be tested after binarization is first processed by the BLOB to mark the connected areas as objects, and filter out the objects whose area and / or length and width in the binarized image do not reach a threshold value; The binary image is processed by TOPHAT to filter out at least one narrow line area; The binary image is processed by Scale-Add, and the difference in proportion between the object and the at least one narrow area is used to find the detection object that needs to be retained as the detection area.

6. The line image detection method according to claim 1, characterized in that: The backbone thinning process includes eroding the detection area according to a preset number of times.

7. The line image detection method according to claim 1, characterized in that: The line segment angle data includes 0 degrees, 45 degrees, 90 degrees and 135 degrees, and the point type data includes endpoints, connection points, bend points and corner points.

8. The line image detection method according to claim 1, characterized in that: The step of determining whether the feature point corresponding to the point cloud is a defect based on the line segment angle data and the point type data includes: when the feature point corresponding to the point cloud is determined to be an endpoint, if the endpoint is not connected to any object, it is determined to be a defect.

9. The line image detection method according to claim 1, characterized in that: The step of determining whether the feature point corresponding to the point cloud is a defect based on the line segment angle data and the point type data includes: when the feature point corresponding to the point cloud has three directions, it is determined to be a connection point and not a defect.

10. The line image detection method according to claim 1, characterized in that: The step of determining whether the feature point corresponding to the point cloud is a defect based on the line segment angle data and the point type data includes: running a mask of a corresponding angle based on the line segment angle of the feature point corresponding to the point cloud, and the feature point highlighted after running the mask is determined to be a quasi-defect.

11. The line image detection method according to claim 10, characterized in that: When it is determined to be the quasi-defect, a grayscale average value and a standard deviation are calculated for the entire line segment object to which the feature point determined to be the quasi-defect belongs, and when the grayscale value of the feature point is higher than the grayscale average value plus a preset multiple of the standard deviation, the feature point is determined to be a defect.

12. An automatic optical inspection system, characterized in that: include; A testing platform, used to set a sample to be tested; At least one image capture device, disposed on one side of the detection platform, for photographing the sample to be detected on the detection platform; as well as A detection device is connected to the image capture device, and after the image capture device obtains an image of the sample to be tested, the method as claimed in any one of claims 1 to 11 is executed to complete defect detection.

Citation Information

Patent Citations

  • Method for detecting defects of printed circuit board (PCB) based on skeleton extraction and range conversion

    CN103049753A

  • Method and system for detecting flaw

    CN106097371A

  • Optical detection system and detection method based on side-in type multi-angle light source structure

    CN111855679A

  • Method for detecting on-off of etched circuit

    CN112116591A

  • Apparatus and method for checking line defects on panel

    TW202242790A