A method, device and storage medium for detecting the pin size of a semiconductor device

Abstract

The present invention discloses a method, device and storage medium for detecting the pin size of a semiconductor device, including: preprocessing a target image by using Gaussian filtering and adaptive threshold segmentation; within a pre-set ROI region of the target image, fitting a reference line according to the target edge; within the ROI region of each pin, fitting a pin vertex according to the mean value of the pin edge, and drawing a perpendicular line from the pin vertex to the reference line to obtain the pin length; performing angle correction on the target image according to the reference line, obtaining a forward view of the pin region from the corrected target image, and obtaining the pin contour; obtaining the centroid of each sub-contour in the pin contour, and replacing the pin vertex coordinates with the centroid coordinates, and obtaining the pin pitch according to the centroid spacing. The present invention not only improves the accuracy of edge line fitting, but also speeds up the algorithm detection rate, greatly reduces the influence of pin edge fluctuations on detection, reduces the dependence of detection on the lighting conditions, and improves the detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of digital image processing, and particularly to a method, device, and storage medium for detecting the pin size of a semiconductor device. Background Art

[0002] Currently, the appearance size detection of industrial devices mostly relies on human eye detection. However, human eye detection depends on human experience and has subjectivity and instability, and is gradually being replaced by various computer vision detection algorithms in recent years.

[0003] Computer vision algorithms based on traditional features usually include steps such as image preprocessing, artificial feature selection and design, feature extraction, and appearance defect or size detection. Among them, image preprocessing includes methods such as filtering and denoising, image enhancement, and image correction. This step aims to improve the image quality, highlight specific features, and serve for subsequent detection; artificial feature selection and design is the core of traditional computer vision algorithms. Common features include edge features, texture features, and color features, etc. The final accuracy and efficiency of the detection algorithm are directly affected by the selected features.

[0004] In recent years, computer vision algorithms based on traditional features have been increasingly widely used in the field of industrial vision detection, and their defects are also obvious. First, since traditional computer vision detection methods rely on artificial feature selection, the detection effect of the algorithm is directly affected by the selected features. Therefore, how to select appropriate features and design feature extraction algorithms for specific detection targets is one of the problems that need to be considered. Second, traditional computer vision algorithms are sensitive to external factors, especially the lighting factor. In many cases, it is difficult to detect the target to be detected that does not meet the lighting requirements. Summary of the Invention

[0005] In order to overcome the above-mentioned disadvantages and deficiencies of the prior art, the purpose of the present invention is to provide a method, device, and storage medium for detecting the pin size of a semiconductor device.

[0006] The purpose of the present invention is achieved by the following technical solutions:

[0007] A method for detecting the pin size of a semiconductor device, comprising:

[0008] Preprocessing the target image by using Gaussian filtering and adaptive threshold segmentation method;

[0009] Within the ROI (region of interest) area preset in the target image, fitting a reference line according to the target edge;

[0010] Within the ROI area of each pin, fitting the pin vertex according to the mean value of the pin edge, and drawing a perpendicular line from the pin vertex to the reference line to obtain the pin length;

[0011] Perform angle correction on the target image according to the reference straight line, obtain the pin area map of the corrected target image, and obtain the pin contour;

[0012] Obtain the centroid of each sub-contour in the pin contour, replace the pin vertex coordinates with the centroid coordinates, and obtain the pin pitch according to the centroid spacing.

[0013] Furthermore, the target image is preprocessed by using Gaussian filtering and adaptive threshold segmentation method, specifically:

[0014] Slice the target image using the ROI template to obtain image I, determine the size detection area of the semiconductor device, and then use Gaussian filtering to denoise the image;

[0015] Use the OTSU algorithm to obtain the optimal threshold T for target image segmentation;

[0016] Binarize the target image using the optimal threshold T.

[0017] Furthermore, the use of the OTSU algorithm to obtain the optimal threshold T for target image segmentation is specifically:

[0018] Find the maximum gray value and minimum gray value of the target image, denoted as max and min respectively. Let the threshold variable T traverse the interval [min, max]. For each traversed T value, record T as the current threshold, and then divide the image gray level into two categories according to whether the gray value is greater than the current threshold, and calculate the between-class variance g of these two categories. When the between-class variance reaches the maximum value, the corresponding T value is the optimal threshold.

[0019] Furthermore, the pin vertices are fitted according to the pin edge mean value within the ROI area of each pin, and the pin length is obtained by drawing a perpendicular line from the pin vertex to the reference straight line, specifically:

[0020] Within the four ROI areas preset on the edge of the semiconductor device, extract the target edge by the Sobel operator and calculate the edge mean value to obtain four edge fitting mean values a1, a2, a3, a4, and use the least squares method to fit the edge straight line to obtain the reference straight line;

[0021] Within the ROI area preset for each semiconductor device pin, calculate the mean point (x i , y i ) of the pin edge according to the pin edge, and this pin edge mean point is used as the pin vertex P i ;

[0022] Draw a perpendicular line from the pin vertex P i to the reference straight line, and denote the foot of the perpendicular as F i , then the line segment P i F iThe length is the length of the pins of the semiconductor device.

[0023] Further, when calculating the ordinate y of the mean point i one maximum value and one minimum value are discarded.

[0024] Further, obtaining the forward image of the pin area from the corrected target image to obtain the pin contour specifically includes:

[0025] Performing angle correction on the image by the offset angle θ of the reference straight line to ensure that the pins of the semiconductor device are vertically upward, and obtaining the corrected target image I';

[0026] Using the pin area ROI template to slice the corrected target image to obtain the pin area image I0;

[0027] Performing morphological processing of erosion first and then dilation on the pin area image I0 to obtain the connected area;

[0028] Performing contour scanning on the connected area to obtain the pin contour C, and the sub-elements in C are the contours of each pin.

[0029] Further, obtaining the centroid of each sub-contour in the pin contour and replacing the pin vertex coordinates with the centroid coordinates, and obtaining the pin pitch according to the centroid pitch specifically includes:

[0030] Obtaining the zero-order moment M 00 and the first-order moment M 01 and M 10 of all sub-contours in the pin contour C to obtain the centroid coordinates (x c , y c ) of the sub-contour;

[0031] Subtracting the abscissa of the centroid of adjacent pin contours to obtain the pin pitch d i .

[0032] Further, the expression of the between-class variance g is:

[0033] g = w1(u1 - u) 2 + w2(u2 - u) 2

[0034] where u is the gray value, u1 is the gray mean of class 1, w1 is the proportion of class 1, u2 is the gray mean of class 2, and w2 is the proportion of class 2.

[0035] A detection device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the method for detecting the pin size of the semiconductor device according to any one of claims 1-8.

[0036] A computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the semiconductor device pin size detection method described above is implemented.

[0037] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0038] (1) The present invention combines a preset ROI (region of interest) with edge extraction to fit the edge straight line, which not only improves the accuracy of edge straight line fitting but also speeds up the algorithm detection rate.

[0039] (2) The present invention selects the centroid of the pin contour instead of the pin vertex as the key feature for detecting the pin pitch, and improves the feature extraction method, solving the problem of difficult pin vertex detection caused by the uneven edge and shape of the pin and insufficient external conditions (especially lighting conditions), and greatly improving the accuracy, robustness, and anti-interference ability of the pin pitch detection algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is the step flowchart of the present invention;

[0041] Figure 2 is the algorithm flowchart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] The following combines embodiments to further elaborate on the present invention in detail, but the implementation manners of the present invention are not limited thereto.

[0043] Embodiment 1

[0044] As Figure 1 - Figure 2 shown, a semiconductor device pin size detection method, where the pin size detection includes pin length and pin pitch detection. The method includes the following steps:

[0045] Step 1: Obtain an image of the semiconductor device, and sequentially perform preprocessing on the image using Gaussian filtering and OTSU adaptive threshold segmentation algorithms, including:

[0046] S1.1 Use an ROI template to slice the target image to obtain image I, determine the size detection area of the semiconductor device, and then use Gaussian filtering to denoise the image;

[0047] Specifically: Use an ROI template to intercept the original image to obtain the area to be detected. Since the original image has a high resolution and contains a lot of irrelevant background and interference, first use the ROI template to intercept and determine the position of the semiconductor device pins, which can reduce the calculation amount.

[0048] S1.2 Use the OTSU algorithm to obtain the optimal threshold T for target image segmentation, specifically as follows:

[0049] Find the maximum and minimum gray values of the target image, denoted as max and min respectively. Let the threshold variable T traverse the interval [min, max]. For each traversed T value, record T as the current threshold, and then divide the image gray values into two categories according to whether the gray value is greater than the current threshold. Calculate the between-class variance g of these two categories. When the between-class variance reaches the maximum value, the corresponding T value is the optimal threshold.

[0050] The expression of the between-class variance g is shown in Equation (1):

[0051] g = w1(u1 - u) 2 + w2(u2 - u) 2 (1)

[0052] where u is the gray value, u1 is the gray mean of category 1, w1 is the proportion of category 1, u2 is the gray mean of category 2, and w2 is the proportion of category 2;

[0053] S1.3 Binarize the target image using the optimal threshold T.

[0054] Step 2: Within the pre-set ROI area of the target image, fit the reference line according to the target edge, including the following steps:

[0055] S2.1 In the four pre-set ROI areas on the edge of the semiconductor device, extract the target edge by the Sobel operator and calculate the edge mean value to obtain four edge fitting mean values a1, a2, a3, and a4. Use the least squares method to fit the edge line to obtain the reference line L. The overall squared error E of the reference line fitting process is shown in Equation (2):

[0056]

[0057] where the line to be fitted is Ax + By + C = 0, and by finding the values of A, B, and C when E is minimized, the fitted reference line L can be obtained;

[0058] S2.2 In the pre-set ROI area of each semiconductor device pin, extract the pin edge by the Sobel operator and calculate the mean point (x i , y i ). When calculating y i , first discard one maximum value and one minimum value to reduce the influence of edge fluctuations. The mean point of the pin edge is used as the pin vertex P i ;

[0059] S2.3 Draw a perpendicular line from the pin vertex P i to the reference line L, and denote the foot of the perpendicular as Fi , then the length of the line segment P i F i is the length of the pin of the semiconductor device.

[0060] Pin pitch detection includes the following steps:

[0061] Step 3: Perform angle correction on the target image according to the reference straight line, obtain the pin area map of the corrected target image, and obtain the pin contour. Specifically:

[0062] S3.1 Perform angle correction on the image by the offset angle θ of the reference straight line L to ensure that the pins of the semiconductor device are vertically upward, and obtain the corrected target image I';

[0063] S3.2 Use the pin area ROI template to slice the corrected target image to obtain the pin area image I0;

[0064] S3.3 Perform morphological processing of erosion first and then dilation on the pin area image I0 to smooth the pin edges and remove possible noise to obtain the connected area;

[0065] S3.4 Perform contour scanning on the connected area to obtain the pin contour C of the semiconductor device, and the sub-elements in C are the contours of each pin.

[0066] Step 4: Obtain the centroid of each sub-contour in the pin contour C, replace the pin vertex coordinates with the centroid coordinates, and obtain the pin pitch according to the centroid spacing. Specifically:

[0067] S4.1 Obtain the centroid of each sub-contour from the zero-order moment M 00 and the first-order moment M 01 and M 10 . The expression of the zero-order moment is shown in Equation (3), and the expressions of the first-order moments are shown in Equations (4) and (5);

[0068]

[0069]

[0070]

[0071] where V(i,j) is the gray value at the image coordinate (i,j). From the zero-order moment and the first-order moment, the centroid coordinates (x c , y c ) of the sub-contour can be obtained according to Equation (6).

[0072]

[0073] S4.2 Subtract the abscissa of the centroid of adjacent pin contours to obtain the pin pitch di 。

[0074] In this embodiment, when fitting the reference straight line, 4 ROI regions need to be preset in advance, all of which are located near the lower edge of the semiconductor device. The purpose is to determine the approximate range of the lower edge of the semiconductor device. The specific positions of the ROI regions are obtained through experimental tests. When obtaining the pin vertices, ROI regions also need to be set. Since the semiconductor device has 60 pins, there are 60 ROI regions. These ROI regions are used to determine the approximate positions of each pin, and the specific positions are also obtained through experimental tests. After all the ROI regions are set, they are applicable to all images.

[0075] This method divides the detection of the pin dimensions of the semiconductor device into pin length detection and pin pitch detection. First, preprocess the image, such as filtering to remove noise, binarization, and angle correction, and then perform pin length detection and pin pitch detection respectively. For pin length detection, within the preset ROI regions, the reference straight line and the pin vertices are respectively fitted according to the target edge, and the pin length can be obtained from the pin vertices and the reference straight line. For pin pitch detection, first obtain the positive image of the pin region through image slicing, then obtain the pin contour, calculate the centroid of each pin according to the pin contour, use the centroid to fit and replace the pin vertices, and finally the pin pitch can be obtained from the centroid spacing. The present invention adopts a combination of preset ROI and edge extraction to fit the edge straight line, which not only improves the accuracy of edge straight line fitting but also speeds up the algorithm detection rate. The present invention selects the centroid of the pin contour to replace the pin vertices as the key feature for detecting the pin pitch, greatly reducing the influence of the pin edge undulation on the detection, reducing the dependence of the detection on the lighting conditions, having high detection accuracy for the pin dimensions, and strong robustness.

[0076] Embodiment 2

[0077] An embodiment of the present invention provides a device. Based on Embodiment 1, it includes at least one processor and a memory communicatively connected to the at least one processor. Among them, the memory stores instructions executable by the at least one processor. When the instructions are executed by the at least one processor, the at least one processor is enabled to execute a method for detecting appearance defects according to some embodiments of the present invention. A method for detecting the pin size of a semiconductor device according to an embodiment of the present invention, wherein the pin size detection includes detecting the pin length and the pin pitch. A detection device according to an embodiment of the present invention, the processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of a runnable device of a detection method, and uses various interfaces and lines to connect various parts of the runnable device of the entire detection method. The memory can be used to store computer programs and / or modules. The processor realizes various functions of a runnable device of a method for detecting the pin size of a semiconductor device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.), etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0078] Embodiment 3

[0079] An embodiment of the present invention provides a computer-readable storage medium. Based on Embodiment 1, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute a method for detecting the pin size of a semiconductor device according to some embodiments of the present invention. A computer-readable storage medium according to an embodiment of the present invention realizes pin size detection, and the detection includes detecting the pin length and the pin pitch.

[0080] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the described embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A method for detecting the pin size of a semiconductor device, characterized in that, Including: Preprocessing the target image by using Gaussian filtering and adaptive threshold segmentation method; Within the pre-set ROI area of the target image, fitting a reference line according to the target edge; Within the ROI area of each pin, fitting the pin vertex according to the mean value of the pin edge, and drawing a perpendicular line from the pin vertex to the reference line to obtain the pin length. Specifically: Within the four ROI areas pre-set at the edge of the semiconductor device, extracting the target edge by the Sobel operator and calculating the edge mean value to obtain four edge fitting mean values a1, a2, a3, a4, and using the least square method to fit the edge line to obtain the reference line L; Within the ROI region preset for each semiconductor device pin, calculate the mean point (x i , y i ) according to the pin edge, and this pin edge mean point is taken as the pin vertex P i ; From the pin vertex P i draw a perpendicular line to the reference straight line L, and mark the foot of the perpendicular as F i , then the length of the line segment P i F i is the length of the pin of the semiconductor device; Performing angle correction on the target image according to the reference line, obtaining the pin area map of the corrected target image, and obtaining the pin contour; Specifically: The image is angle-corrected by the offset angle θ of the reference straight line L to ensure that the pins of the semiconductor device are vertically upward, and the corrected target image I is obtained. ＇ ; Slicing the corrected target image by using the pin area ROI template to obtain the pin area image I0; Performing morphological processing of erosion first and then dilation on the pin area image I0 to obtain the connected area; Performing contour scanning on the connected area to obtain the pin contour C, and the sub-elements in C are the contours of each pin; Obtaining the centroid of each sub-contour in the pin contour, replacing the pin vertex coordinates with the centroid coordinates, and obtaining the pin pitch according to the centroid pitch; Specifically: Obtain the zero-order moment M of all sub-contours in the pin profile C 00 and the first-order moment M 01 and M 10 , and obtain the centroid coordinates (x c , y c ) of the sub-contour; Subtract the abscissa of the centroid of the adjacent pin contours to obtain the pin pitch d i .

2. The method for detecting the pin size of a semiconductor device according to claim 1, wherein The preprocessing of the target image by using Gaussian filtering and adaptive threshold segmentation method is specifically: Slicing the target image by using the ROI template to obtain the image I, determining the size detection area of the semiconductor device, and then using Gaussian filtering to denoise the image; Using the OTSU algorithm to obtain the optimal threshold T for target image segmentation; Binarizing the target image by using the optimal threshold T.

3. The method for detecting the pin size of a semiconductor device according to claim 2, wherein The using the OTSU algorithm to obtain the optimal threshold T for target image segmentation is specifically: Finding the maximum gray value and the minimum gray value of the target image, respectively denoted as max and min, letting the threshold variable T traverse the interval [min, max], for each traversed T value, recording T as the current threshold, and then classifying the image gray scale into two categories according to whether the gray value is greater than the current threshold, calculating the between-class variance g of these two categories, and when the between-class variance obtains the maximum value, the corresponding T value is the optimal threshold.

4. The method for detecting the pin size of a semiconductor device according to claim 1, wherein When calculating the ordinate y of the mean point i discard one maximum value and one minimum value.

5. The method for detecting the pin size of a semiconductor device according to claim 3, wherein The expression of the between-class variance g is: g = w1(u1 - u) 2 + w2(u2 - u) 2 Where u is the gray value, u1 is the gray mean value of category 1, w1 is the proportion of category 1, u2 is the gray mean value of category 2, and w2 is the proportion of category 2.

6. A detection device, characterized in that, Including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the semiconductor device pin size detection method according to any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, A computer program is stored thereon. When the computer program is executed by the processor, it implements the semiconductor device pin size detection method according to any one of claims 1-5.

Images