A visual inspection method and system for IC packaging substrate appearance defects

Abstract

The present application relates to the field of image processing technology, and specifically to a method and system for visual inspection of appearance defects of IC package carriers, the method comprising: determining the regional characteristic value of each superpixel block in each convex hull, determining the filtering window of each pixel point in the grayscale image of the package carrier, and inspecting the filtered package carrier according to the regional concentration of each superpixel block in each convex hull, the color brightness difference value and the area of ​​all connected domains in each superpixel block in each convex hull. The present application determines the window size of each pixel point on the IC package carrier in the filtering algorithm by analyzing the characteristics of the normal texture area, the oxidation defect area and the noise appearing in the collected image in the gold surface area of ​​the IC package carrier, thereby improving the accuracy of oxidation defect detection in the gold surface area of ​​the IC package carrier.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and system for visually inspecting appearance defects of an IC packaging substrate. Background Art

[0002] As a high-end PCB board, IC package substrate is an indispensable part of chip packaging. Compared with traditional PCB boards, the process of IC package substrate is more complicated, and various defects are easily generated during the manufacturing process. In order to ensure the quality and reliability of IC package substrate, it is necessary to perform appearance defect detection on IC package substrate. With the development of IC package substrate manufacturing process, its line width and line spacing have also become smaller. In order to detect tiny appearance defects in IC package substrate, it is necessary to use a microscope imaging system to collect IC package substrate images, which will cause the normal texture, noise and appearance defects in the gold surface area of ​​the IC package substrate image to be magnified at the same time. At this time, the texture and noise of the gold surface area in the IC package substrate image may be similar to some appearance defects, such as oxidation defects. If the gold surface texture and noise in the IC package substrate image are not filtered, it will be easy to misjudge the gold surface texture and noise in the IC package substrate image as appearance defects.

[0003] Commonly used image filtering methods include median filtering, mean filtering, Gaussian filtering, bilateral filtering, etc. The filtering effects of these methods are usually related to the window size, but the window size in these methods is usually a fixed value selected by experience, which cannot adapt to the complexity of texture and oxidation defects in the gold surface area of ​​the IC package substrate. It is easy to cause the texture of the gold surface area in the IC package substrate image to not be effectively filtered out, or the oxidation defects in the gold surface area are over-filtered, thereby reducing the accuracy of defect detection on the IC package substrate. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of this application is to provide a method and system for visual inspection of appearance defects of IC packaging substrates. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for visually inspecting appearance defects of an IC package substrate, the method comprising the following steps:

[0006] Obtain a grayscale image of the package substrate, and use double threshold segmentation to obtain a target area in the grayscale image of the package substrate;

[0007] The convex hull algorithm is used to obtain each convex hull in the target area, and each convex hull is divided into multiple super-pixel blocks;

[0008] Obtaining a chromaticity statistical histogram of each superpixel block in each convex hull, and determining a color difference value of each superpixel block in each convex hull according to the similarity between the chromaticity statistical histograms of each pixel block in each convex hull and all superpixel blocks in all other convex hulls;

[0009] Obtaining a brightness statistical histogram of each superpixel block in each convex hull, obtaining a brightness difference value of each superpixel block according to the color difference value determination method, and determining a color brightness difference value of each superpixel block in each convex hull in combination with the color difference value;

[0010] Extracting the connected domain of each superpixel block in each convex hull, determining the regional concentration of each superpixel block in each convex hull according to the distribution of the distance between any two connected domains in each superpixel block in each convex hull, and determining the regional characteristic value of each superpixel block in each convex hull in combination with the color brightness difference value and the area of ​​all connected domains in each superpixel block in each convex hull;

[0011] According to the brightness values ​​of all pixels in the target area, the texture filtering degree of each pixel in the grayscale image of the packaging carrier is determined in combination with the regional characteristic value, and the filtering window of each pixel in the grayscale image of the packaging carrier is determined in combination with the distribution of pixels in the grayscale image of the packaging carrier, and the filtered packaging carrier is inspected.

[0012] Preferably, the color difference value of each superpixel block in each convex hull is the reciprocal of the mean of the similarities between the chromaticity statistical histogram of each superpixel block in each convex hull and the chromaticity statistical histograms of all superpixel blocks in all other convex hulls.

[0013] Preferably, the color brightness difference value of each super pixel block in each convex hull is the average of the color difference value and the brightness difference value of each super pixel block in each convex hull.

[0014] Preferably, the regional concentration of each superpixel block in each convex hull is the average of the distances between any two connected domains in each superpixel block in each convex hull.

[0015] Preferably, the expression of the regional characteristic value of each superpixel block in each convex hull is: ; In the formula, Represents the regional feature value of the jth superpixel block in the i-th convex hull; Represents the color brightness difference value of the jth superpixel block in the i-th convex hull; represents the regional concentration of the jth superpixel block in the i-th convex hull; It represents the mean area of ​​all connected domains in the jth superpixel block in the i-th convex hull.

[0016] Preferably, the method for determining the texture filtering degree of each pixel point in the grayscale image of the package carrier is:

[0017] Determine a pixel set based on the brightness values ​​of all pixels in the target area of ​​the package substrate grayscale image;

[0018] The expression of the texture filtering degree of each pixel in the grayscale image of the package substrate is: ; In the formula, Indicates the texture filtering degree of the kth pixel in the grayscale image of the package substrate; represents the regional feature value of the superpixel block where the k-th pixel point in the grayscale image of the package substrate is located; F represents the pixel point set; norm() represents the normalization function; Indicates a preset constant greater than 0.

[0019] Preferably, the method for determining the pixel point set is:

[0020] The brightness values ​​corresponding to all pixels in all convex hulls in the target area of ​​the package substrate grayscale image in the HSV channel are used as the input of the threshold segmentation algorithm to obtain the brightness threshold, and the set consisting of all pixels in all convex hulls in the target area with brightness values ​​less than the brightness threshold is recorded as the pixel set.

[0021] Preferably, the expression of the filter window of each pixel point in the grayscale image of the package carrier is: ; In the formula, Represents the filter window of the kth pixel in the grayscale image of the package substrate; represents the texture filtering degree of the kth pixel in the grayscale image of the package substrate; D represents the set of all pixels in the target area; , is a preset constant greater than 0, where ; round() is the rounding function.

[0022] Preferably, the testing of the filtered package carrier includes:

[0023] The package substrate grayscale image is used as an input of the filtering algorithm, wherein the filtering window of each pixel in the package substrate grayscale image is used as a window in the filtering algorithm, and the filtered package substrate grayscale image is output;

[0024] The optical automated inspection system is used to compare the filtered package substrate grayscale image with the standard template image of the IC package substrate to detect defects in the package substrate grayscale image.

[0025] In a second aspect, an embodiment of the present application further provides a visual inspection system for appearance defects of an IC packaging substrate, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of any one of the above-mentioned methods when executing the computer program.

[0026] This application has at least the following beneficial effects:

[0027] The present application analyzes the normal texture area, oxidation defect area and noise appearing in the collected image in the gold surface area of ​​the IC package carrier, and constructs the regional feature value of the superpixel block in combination with the color brightness difference value of each superpixel block in each convex hull. The beneficial effect is that the distinction between the normal texture area, the oxidation defect area and the noise pixel points in the package carrier grayscale image of the IC package carrier is improved, and the oxidation defect area in the gold surface area on the surface of the IC package carrier is more accurately identified. The pixel points corresponding to the grayscale image of the package carrier are identified.

[0028] The present application adaptively obtains the window size of each pixel point in the packaging substrate grayscale image of the IC packaging substrate in the filtering algorithm based on the texture filtering degree constructed based on the regional eigenvalue, utilizes the filtering algorithm to filter the texture of the gold surface area in the packaging substrate grayscale image of the IC packaging substrate, and completes the detection of the appearance defects of the IC packaging substrate based on the filtered packaging substrate grayscale image. The beneficial effect lies in that when using the filtering algorithm to filter the texture of the gold surface area in the packaging substrate grayscale image of the IC packaging substrate, it is possible to adaptively select a suitable window size for each pixel point in the packaging substrate grayscale image, and under the premise of ensuring that the texture of the gold surface area in the packaging substrate grayscale image of the IC packaging substrate is effectively filtered out, the detailed texture information of the oxidation defects in the gold surface area is effectively retained.

[0029] The present application determines the window size of each pixel point on the IC package substrate in the filtering algorithm by analyzing the normal texture area, oxidation defect area and noise appearing in the gold surface area of ​​the IC package substrate, thereby improving the accuracy of oxidation defect detection in the gold surface area of ​​the IC package substrate. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A flowchart of a method for visually inspecting appearance defects of an IC package substrate provided by one embodiment of the present application;

[0032] Figure 2 A schematic diagram of a regional feature value extraction process provided by an embodiment of the present application;

[0033] Figure 3 A schematic diagram of a filtering window acquisition process provided by an embodiment of the present application;

[0034] Figure 4 A diagram showing the filtering effect of a package substrate provided in one embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following is a detailed description of the IC package substrate appearance defect visual inspection method and system proposed by the present application, its specific implementation, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0036] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0037] The specific scheme of a method and system for visual inspection of appearance defects of an IC packaging substrate provided by the present application is described in detail below with reference to the accompanying drawings.

[0038] See also Figure 1 , which shows a flowchart of a method for visually inspecting appearance defects of an IC package substrate provided by an embodiment of the present application, the method comprising the following steps:

[0039] Step S1: collecting a surface image of an IC package substrate and preprocessing the collected image.

[0040] A microscope imaging system is used to collect the surface image of the IC package substrate, and the collected surface image is converted into a package substrate grayscale image and an HSV image respectively. The grayscale image of the IC package substrate is recorded as the package substrate grayscale image and the HSV image, and the chromaticity value of each pixel in the package substrate grayscale image in the chromaticity channel of the HSV image and the brightness value in the brightness channel are obtained respectively.

[0041] The conversion of the RGB image into the package substrate grayscale image and the HSV image is a well-known technique, and the specific process will not be described in detail.

[0042] The surface of the IC package substrate generally only includes a dark background area, a gold surface area and a white text area. Under the illumination of the light source of the microscope imaging system, the brightness difference between the dark background area and the gold surface area and the white text area is very obvious, so that the pixel points corresponding to the dark background area in the grayscale image of the package substrate have a smaller brightness value, while the pixel points corresponding to the gold surface area and the white text area in the grayscale image of the package substrate have a larger brightness value.

[0043] Therefore, the brightness values ​​corresponding to all pixels in the grayscale image of the packaging substrate in the HSV channel are used as the input of the threshold segmentation algorithm to obtain the first segmentation threshold, and the area corresponding to the pixels with brightness values ​​greater than the first segmentation threshold in the grayscale image of the packaging substrate is recorded as the foreground area.

[0044] It should be noted that there are many commonly used threshold segmentation algorithms. In this embodiment, the Otsu threshold segmentation algorithm is used to perform binarization processing on the image. The implementer may also use other threshold segmentation algorithms in combination with specific circumstances, and this embodiment does not make special restrictions. Among them, the Otsu threshold segmentation algorithm is a well-known technology, and the specific segmentation process will not be repeated.

[0045] Furthermore, since there is an obvious color difference between the gold surface area and the white text area on the surface of the IC package carrier, the corresponding pixel points of the white text area in the foreground image have a smaller chromaticity value, while the corresponding pixel points of the gold surface area in the foreground image have a larger chromaticity value. Therefore, the pixel values ​​of all pixels in the foreground area in the grayscale image of the package carrier are replaced with chromaticity values, and used as the input of the Otsu threshold segmentation algorithm to obtain the second segmentation threshold, and the area corresponding to the pixels in the foreground area whose chromaticity values ​​are greater than the second segmentation threshold in the grayscale image of the package carrier is recorded as the target area.

[0046] Step S2: Determine the color difference value of each superpixel block in each convex hull according to the similarity between the chromaticity statistical histograms of each pixel block in each convex hull and all superpixel blocks in all other convex hulls, and determine the color brightness difference value of each superpixel block in each convex hull in combination with the brightness difference value of each superpixel block.

[0047] During the packaging process of the IC packaging substrate, the gold surface area is directly or indirectly exposed to the air. The oxygen in the air will chemically react with the metal surface. During the packaging process, especially when using a high temperature or high humidity environment, the rate of the oxidation reaction may accelerate. High temperature can increase the kinetic energy of molecules, and high humidity provides sufficient oxygen supply for the oxidation reaction. Therefore, the metal part of the IC packaging substrate may have oxidation defects.

[0048] Therefore, in order to analyze whether there are oxidation defects in the gold surface area, the connected domain extraction algorithm is used to extract the contours of all connected domains in the target area to obtain the contour of each connected domain.

[0049] Since the contour of the gold surface area on the surface of the IC package substrate is usually relatively regular, a convex hull algorithm is used to calculate the convex hull of each connected domain in the target area to obtain all convex hulls in the target area.

[0050] It should be noted that there are many commonly used connected domain extraction algorithms. In this embodiment, an eight-neighborhood connected domain extraction algorithm is used to extract the contour in the target area. The implementer may also use other connected domain extraction algorithms to extract the contour in the image, and this embodiment does not impose any special restrictions.

[0051] Among them, the eight-neighborhood connected domain extraction algorithm and the convex hull algorithm are both well-known technologies, and the specific process will not be repeated here.

[0052] Furthermore, since the normal texture areas in the gold surface area on the surface of the IC package carrier usually have similar color distributions, and the metal color in the area where oxidation defects occur in the gold surface area becomes darker, there is a large difference in color and brightness between the normal texture area and the oxidation defect area in the gold surface area.

[0053] Therefore, based on the similarity between the chromaticity statistical histograms of each pixel block in each convex hull and all superpixel blocks in all other convex hulls, the color difference value of each superpixel block in each convex hull is determined, and based on the similarity between the brightness statistical histograms of each pixel block in each convex hull and all superpixel blocks in all other convex hulls, the brightness difference value of each superpixel block in each convex hull is determined to determine whether there are oxidation defects on the surface of the package carrier. Specifically:

[0054] First, each convex hull in the target area is used as the input of the superpixel segmentation algorithm, and all superpixel blocks of each convex hull in the target area are output. The superpixel segmentation algorithm is a well-known technology, and the specific process of image segmentation is not described in detail.

[0055] Furthermore, the chromaticity statistical histogram and brightness statistical histogram of each superpixel block in each convex hull are obtained to characterize the color distribution and brightness distribution of the pixels in the superpixel block. The process of obtaining the statistical histogram is a well-known technology, and the specific process will not be repeated.

[0056] Furthermore, the chromaticity statistical histogram and the brightness statistical histogram of all superpixel blocks of each convex hull are normalized for subsequent acquisition of similarities between different histograms. The normalization of the histogram is a well-known technique, and the specific process will not be described in detail.

[0057] It can be obtained that the color difference value of each superpixel block in each convex hull is the reciprocal of the mean of the similarity between the chromaticity statistical histogram of each superpixel block in each convex hull and the chromaticity statistical histogram of all superpixel blocks in all other convex hulls.

[0058] It should be noted that there are many methods for measuring the similarity between two histograms. In this embodiment, the similarity between two chromaticity statistical histograms is measured by calculating the inverse of the Bhattacharyya coefficient between the two chromaticity statistical histograms. In actual application, as other implementation methods, the implementer may also measure the similarity between the chromaticity statistical histograms by calculating the inverse of the contrast between the histograms. Regarding the method for measuring the similarity between two histograms, this embodiment does not impose any special restrictions.

[0059] The calculation process of the Bhattacharyya coefficient is a well-known technology, and the specific calculation process will not be repeated here.

[0060] The smaller the similarity between the chromaticity statistical histogram of the superpixel block and the chromaticity statistical histograms of all superpixel blocks in all other convex hulls, the larger the color difference value of the superpixel block, and the more likely it is that oxidation defects appear in the gold surface area corresponding to the superpixel block; conversely, the greater the similarity between the chromaticity statistical histogram of the superpixel block and the chromaticity statistical histograms of all other superpixel blocks in all other convex hulls, the smaller the color difference value of the superpixel block, and the smaller the possibility of oxidation defects appearing in the gold surface area corresponding to the superpixel block.

[0061] Furthermore, with respect to the brightness statistical histogram of the super-pixel block, the brightness difference value of each super-pixel block in each convex hull is obtained according to the color difference value calculation method.

[0062] Furthermore, the average of the color difference value and brightness difference value of each superpixel block in each convex hull is recorded as the color-brightness difference value of each superpixel block in each convex hull, which is used to characterize the degree of difference in color and brightness between the gold surface area corresponding to the superpixel block and the gold surface areas corresponding to all superpixel blocks in all convex hulls in the target area.

[0063] If the color difference value of the superpixel block is larger and the brightness difference value is larger, then the color-brightness difference value of the corresponding superpixel block is larger, indicating that the oxidation defect is more likely to occur in the gold surface area corresponding to the superpixel block; conversely, if the color difference value of the superpixel block is smaller and the brightness difference value is smaller, then the color-brightness difference value of the corresponding superpixel block is smaller, indicating that the possibility of oxidation defects occurring in the gold surface area corresponding to the superpixel block is smaller.

[0064] Step S3: Determine the regional concentration of each superpixel block in each convex hull according to the distribution of the distance between any two connected domains in each superpixel block in each convex hull, and determine the regional characteristic value of each superpixel block in each convex hull by combining the color brightness difference value, the regional concentration and the area of ​​all connected domains in each superpixel block in each convex hull.

[0065] The normal area where the noise is located in the gold surface area in the surface image of the IC package carrier usually has a large color and brightness difference from the normal texture area in the gold surface area in the surface image. However, the noise is usually randomly generated during the acquisition and transmission of the surface image of the IC package carrier, and the oxidation of the metal surface on the gold surface area is usually diffusive, which makes the noise area in the super-pixel block not only small in area but also more dispersed in distribution, while the oxidation defect area in the super-pixel block is not only large in area but also more concentrated in distribution.

[0066] Therefore, based on the above analysis, the image of each superpixel block in each convex hull is used as the input of the edge detection algorithm, the edge of each superpixel block in each convex hull is output, and the connected domain extraction algorithm is used to extract all connected domains in the edge to obtain all connected domains of each superpixel block in each convex hull.

[0067] It should be noted that there are many commonly used edge detection algorithms and connected domain extraction algorithms. In this embodiment, the Canny edge detection algorithm is used to extract the edges in each superpixel block, and the eight-neighborhood connected domain algorithm is used to extract the connected domain in each superpixel block. Implementers may also use other edge detection algorithms and connected domain extraction algorithms to obtain the edges and connected domains in each superpixel block. This embodiment does not impose any special restrictions.

[0068] Among them, the Canny edge detection algorithm and the eight-neighborhood connected domain extraction algorithm are both well-known technologies. The specific process of extracting edge curves in superpixel blocks using the Canny edge detection algorithm and extracting connected domains using the eight-neighborhood connected domain are not described in detail.

[0069] Further, according to the distribution of the distance between any two connected domains in each superpixel block in each convex hull, the regional concentration of each superpixel block in each convex hull is determined, and the regional characteristic value of each superpixel block in each convex hull is determined in combination with the color brightness difference value, the regional concentration and the area of ​​all connected domains in each superpixel block in each convex hull, so as to eliminate the interference of noise on oxidation defects, specifically:

[0070] (1) The number of all pixels in each connected domain of each superpixel block in each convex hull is taken as the area of ​​each connected domain.

[0071] (2) Furthermore, the mean distance between any two connected domains in each superpixel block in each convex hull is analyzed and recorded as the regional concentration of each superpixel block in each convex hull, which is used to characterize the concentration of the distribution between the noise areas or oxidation defect areas corresponding to all connected domains in the superpixel block.

[0072] It should be noted that there are many methods for calculating the distance between any two convex hulls. In this embodiment, the halcon software is used to calculate the distance between any two connected domains in each superpixel block in each convex hull. The implementer may also adopt other methods for calculating the distance between convex hulls according to the specific situation. This embodiment does not impose any special restrictions.

[0073] Among them, the method used by the halcon software to calculate the distance between connected domains is a well-known technology, and its specific process will not be repeated here.

[0074] (3) Further, based on the color brightness difference value of each superpixel block, the area of ​​all connected domains in each superpixel block and the regional concentration, the regional characteristic value of each superpixel block in each convex hull is determined to characterize the possibility of oxidation defects occurring in the gold surface area corresponding to the superpixel block, specifically:

[0075] The expression of the regional characteristic value of each superpixel block in each convex hull is: ; In the formula, Represents the regional feature value of the jth superpixel block in the i-th convex hull; Represents the color brightness difference value of the jth superpixel block in the i-th convex hull; represents the regional concentration of the jth superpixel block in the i-th convex hull; It represents the mean area of ​​all connected domains in the jth superpixel block in the i-th convex hull.

[0076] The greater the difference in color and brightness between the gold surface area corresponding to the superpixel block and the gold surface areas corresponding to all other superpixel blocks in the target area, that is, the greater the color and brightness difference value of the superpixel block, and the larger the area of ​​each connected domain in the gold surface area corresponding to the superpixel block, and the greater the regional concentration of the superpixel block, the more likely it is that the gold surface area corresponding to the superpixel block has oxidation defects, that is, the larger the regional characteristic value of the corresponding superpixel block; conversely, the smaller the difference in color and brightness between the gold surface area corresponding to the superpixel block and the gold surface areas corresponding to all other superpixel blocks in the target area, that is, the smaller the color and brightness difference value of the superpixel block, and the smaller the area of ​​each connected domain in the gold surface area corresponding to the superpixel block, and the smaller the regional concentration of the superpixel block, the smaller the possibility of oxidation defects in the gold surface area corresponding to the superpixel block, that is, the smaller the regional characteristic value of the corresponding superpixel block.

[0077] Preferably, the schematic diagram of the regional feature value extraction process provided in this embodiment is as follows: Figure 2 shown.

[0078] Step S4: Determine the texture filtering degree of each pixel in the grayscale image of the package carrier according to the brightness values ​​of all pixels in the target area in combination with the regional feature value.

[0079] The area where oxidation defects appear in the gold surface area on the surface of the IC package substrate has a darker color than the normal texture area. Therefore, the brightness values ​​corresponding to all pixels in all convex hulls in the target area in the HSV channel are used as the input of the threshold segmentation algorithm to obtain the brightness threshold, and the set consisting of all pixels with brightness values ​​less than the brightness threshold in all convex hulls in the target area of ​​the package substrate grayscale image is recorded as a pixel set, which is used to characterize the set consisting of all pixels corresponding to the area where oxidation defects appear in the gold surface area on the surface of the IC package substrate and the pixels generated by noise in the package substrate grayscale image.

[0080] It should be noted that there are many commonly used threshold segmentation algorithms. In this embodiment, the maximum inter-class variance is used to classify the pixels in all convex hulls in the target area. The implementer may also use other threshold segmentation algorithms in combination with specific circumstances, and this embodiment does not make special restrictions. Among them, the maximum inter-class variance is a well-known technology, and the specific process will not be repeated.

[0081] Further, based on the brightness values ​​of all pixels in the target area, combined with the regional feature values, the texture filtering degree of each pixel in all convex hulls in the target area is determined, which is used to characterize the filtering degree of the pixel when the texture of the gold surface area in the surface image of the IC package carrier is filtered using the image filtering method, specifically:

[0082] The expression of the texture filtering degree of each pixel in the grayscale image of the package substrate is: ; In the formula, Indicates the texture filtering degree of the kth pixel in the grayscale image of the package substrate; represents the regional feature value of the superpixel block where the k-th pixel in the grayscale image of the package substrate is located; k represents the k-th pixel in the grayscale image of the package substrate; F represents a pixel set; norm() represents a normalization function; Indicates a preset constant greater than 0, used to prevent the denominator from being 0.

[0083] It should be noted that for pixels in the grayscale image of the package substrate that do not belong to the pixel set, there is no regional characteristic value. In order to filter out normal areas without oxidation defects, the pixels that do not belong to the pixel set are assigned a larger value of 1 to filter them out.

[0084] Since it is necessary to filter the texture of the gold surface area in the IC package carrier surface image, if the pixel point exists in the pixel point set, the larger the regional characteristic value of the superpixel block corresponding to the pixel point is, the more likely the pixel point is the area where oxidation defects appear in the gold surface area on the IC package carrier surface. In order to retain the texture detail information of the area where oxidation defects appear in the gold surface area on the IC package carrier surface, the filtering degree of the pixel point should be smaller; if the pixel point is not a point in the pixel point set, the pixel point is to effectively filter out the normal texture detail information of the gold surface area in the IC package carrier surface image. The larger the filtering degree of the pixel point is, the value of 1 is assigned to the corresponding pixel point.

[0085] Step S5: Determine the filtering window for each pixel in the grayscale image of the package carrier according to the texture filtering degree of each pixel in the target area and the distribution of the pixels in the grayscale image of the package carrier, and detect the filtered package carrier.

[0086] According to the texture filtering degree of each pixel in the target area and the distribution of pixels in the grayscale image of the package substrate, the filtering window of each pixel in the grayscale image of the package substrate is determined. The specific calculation process is as follows: The expression of the filtering window of each pixel in the grayscale image of the package substrate is: ; In the formula, Represents the filter window of the kth pixel in the grayscale image of the package substrate; represents the texture filtering degree of the kth pixel in the grayscale image of the package substrate; D represents the set of all pixels in the target area; , is a preset constant greater than 0, used to control the size of the pixel window in the filtering algorithm, where: , used to emphasize the processing of the target area; round() is a rounding function, used to round the value in brackets.

[0087] It should be noted that , The value of is set manually. , The values ​​of are 3 and 9 respectively, and the implementer can also set them according to the specific situation. This embodiment does not impose any special restrictions.

[0088] In order to retain the texture detail information in the area where oxidation defects appear in the gold surface area on the surface of the IC package substrate, and to effectively filter out the normal texture detail information of the gold surface area in the surface image of the IC package substrate, if the pixel point in the grayscale image of the package substrate belongs to the target area, and the greater the texture filtering degree of the corresponding pixel point, the larger the window size of the pixel point in the filtering algorithm should be; and if the pixel point in the grayscale image of the package substrate does not belong to the target area, in order to filter out the noise pixel points that appear in the grayscale image of the package substrate except the image area corresponding to the gold surface area on the surface of the IC package substrate, Acts as a filtering window for pixels outside the target area in the grayscale image of the package substrate.

[0089] Preferably, the filter window acquisition process diagram provided in this embodiment is as follows: Figure 3 shown.

[0090] Step S6: Based on the filtering window of each pixel in the grayscale image of the package carrier, the filtered package carrier is inspected in combination with the filtering algorithm.

[0091] Based on step S5, the filtering window of each pixel point in the grayscale image of the packaging carrier is obtained. Therefore, in combination with the filtering algorithm, the grayscale image of the packaging carrier is used as the input of the filtering algorithm, wherein the filtering window of each pixel point in the grayscale image of the packaging carrier is used as the window in the filtering algorithm, and the filtered grayscale image of the packaging carrier is output.

[0092] It should be noted that there are many commonly used filtering algorithms. In this embodiment, a median filtering algorithm is used to filter the noise in the grayscale image of the package carrier. In actual application, as other implementation methods, the implementer may also adopt other filtering methods such as Gaussian filtering and mean filtering. This embodiment does not impose any special restrictions on the choice of filtering algorithm.

[0093] Among them, the median filtering algorithm is a well-known technology, and the specific filtering process will not be described in detail.

[0094] The automated optical inspection (AOI) system is used to compare the grayscale image of the package substrate with the standard template image of the IC package substrate that has been pre-imported into the AOI system. The defects in the grayscale image of the package substrate are displayed or marked through a display or automatic marking to complete the detection of appearance defects of the IC package substrate.

[0095] Preferably, the packaging substrate filtering effect diagram provided in this embodiment is as follows Figure 4 As shown, a represents the grayscale image of the package carrier before filtering, b represents the grayscale image of the package carrier after filtering by adjusting the filter window, and c represents the grayscale image of the package carrier after filtering without adjusting the filter window.

[0096] Based on the same inventive concept as the above method, an embodiment of the present application also provides a visual inspection system for appearance defects of an IC package substrate, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of any one of the above-mentioned methods for visual inspection of appearance defects of an IC package substrate when executing the computer program.

[0097] It should be noted that the above sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above is a description of a specific embodiment of this specification. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0099] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for visually inspecting appearance defects of IC packaging substrates, characterized in that: The method comprises the following steps: Obtain a grayscale image of the package substrate, and use double threshold segmentation to obtain a target area in the grayscale image of the package substrate; The convex hull algorithm is used to obtain each convex hull in the target area, and each convex hull is divided into multiple super-pixel blocks; Obtaining a chromaticity statistical histogram of each superpixel block in each convex hull, and determining a color difference value of each superpixel block in each convex hull according to the similarity between the chromaticity statistical histograms of each pixel block in each convex hull and all superpixel blocks in all other convex hulls; Obtaining a brightness statistical histogram of each superpixel block in each convex hull, obtaining a brightness difference value of each superpixel block according to the color difference value determination method, and determining a color brightness difference value of each superpixel block in each convex hull in combination with the color difference value; Extracting the connected domain of each superpixel block in each convex hull, determining the regional concentration of each superpixel block in each convex hull according to the distribution of the distance between any two connected domains in each superpixel block in each convex hull, and determining the regional characteristic value of each superpixel block in each convex hull in combination with the color brightness difference value and the area of ​​all connected domains in each superpixel block in each convex hull; According to the brightness values ​​of all pixels in the target area, the texture filtering degree of each pixel in the grayscale image of the package carrier is determined in combination with the regional feature value, and the filtering window of each pixel in the grayscale image of the package carrier is determined in combination with the distribution of the pixels in the grayscale image of the package carrier, and the filtered package carrier is inspected; The expression of the regional characteristic value of each super pixel block in each convex hull is: ; In the formula, Represents the regional feature value of the jth superpixel block in the i-th convex hull; Represents the color brightness difference value of the jth superpixel block in the i-th convex hull; represents the regional concentration of the jth superpixel block in the i-th convex hull; It represents the mean area of ​​all connected domains in the jth superpixel block in the i-th convex hull.

2. A visual inspection method for appearance defects of IC packaging substrates as claimed in claim 1, characterized in that: The color difference value of each superpixel block in each convex hull is the reciprocal of the mean of the similarity between the chromaticity statistical histogram of each superpixel block in each convex hull and the chromaticity statistical histogram of all superpixel blocks in all other convex hulls.

3. The IC package substrate appearance defect visual inspection method according to claim 1, characterized in that: The color brightness difference value of each super pixel block in each convex hull is the average of the color difference value and the brightness difference value of each super pixel block in each convex hull.

4. The IC package substrate appearance defect visual inspection method according to claim 1, characterized in that: The regional concentration of each superpixel block in each convex hull is the average of the distances between any two connected domains in each superpixel block in each convex hull.

5. The IC package substrate appearance defect visual inspection method according to claim 1, characterized in that: The method for determining the texture filtering degree of each pixel point in the grayscale image of the package carrier is as follows: Determine a pixel set based on the brightness values ​​of all pixels in the target area of ​​the package substrate grayscale image; The expression of the texture filtering degree of each pixel in the grayscale image of the package substrate is: ; In the formula, Indicates the texture filtering degree of the kth pixel in the grayscale image of the package substrate; Represents the regional feature value of the superpixel block where the k-th pixel point in the grayscale image of the package substrate is located; F represents a set of pixels; norm() represents a normalization function; Indicates a preset constant greater than 0.

6. A method for visually inspecting appearance defects of IC packaging substrates as claimed in claim 5, characterized in that: The method for determining the pixel point set is: The brightness values ​​corresponding to all pixels in all convex hulls in the target area of ​​the package substrate grayscale image in the HSV channel are used as the input of the threshold segmentation algorithm to obtain the brightness threshold, and the set consisting of all pixels in all convex hulls in the target area with brightness values ​​less than the brightness threshold is recorded as the pixel set.

7. The IC package substrate appearance defect visual inspection method according to claim 1, characterized in that: The expression of the filter window of each pixel in the grayscale image of the package carrier is: ; In the formula, Represents the filter window of the kth pixel in the grayscale image of the package substrate; represents the texture filtering degree of the kth pixel in the grayscale image of the package substrate; D represents the set of all pixels in the target area; , is a preset constant greater than 0, where ; round() is the rounding function.

8. The IC package substrate appearance defect visual inspection method according to claim 1, characterized in that: The step of testing the filtered package carrier includes: The package substrate grayscale image is used as an input of the filtering algorithm, wherein the filtering window of each pixel in the package substrate grayscale image is used as a window in the filtering algorithm, and the filtered package substrate grayscale image is output; The optical automated inspection system is used to compare the filtered package substrate grayscale image with the standard template image of the IC package substrate to detect defects in the package substrate grayscale image.

9. A visual inspection system for appearance defects of IC packaging substrates, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method for visually inspecting appearance defects of an IC packaging substrate as described in any one of claims 1 to 8 are implemented.

Images