A semiconductor wire bonding device recognition method based on image multi-feature matching

Abstract

The application relates to a semiconductor device lead bonding method based on image multi-feature matching, belongs to the image recognition technical field, solves the problem of fast matching of semiconductor devices in lead bonding, and comprises the following steps: establishing a bonding device image multi-feature template library, pre-processing a device image, extracting contour features of the image, extracting texture features of the image to obtain parameters H i Bonding device matching, the application can establish different template libraries according to different chips, does not need manual identification of semiconductor devices, and greatly improves the production efficiency of a lead bonding machine.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image recognition, and particularly relates to a semiconductor lead bonding device recognition method based on image multi-feature matching. BACKGROUND

[0002] A wire bonding machine is a mechatronic device used for electrical connection between wafers and wafers, wafers and electrodes in the field of microelectronic packaging, and finally realizes the connection of micron-level metal wires from one end of a device electrode to the other end, which is similar to building a bridge between devices by using the metal wire to realize the current passing between the finished devices. Taking a light emitting diode as an example, the electrical connection process of the chip IC is completed by the bonding machine. In order to realize the power supply of the positive and negative electrodes of the diode to make the chip of the lamp bead emit light, the metal wire needs to be used to connect the positive electrode of the diode and the positive electrode of the chip, and the negative electrode of the diode and the negative electrode of the chip, so as to realize a complete current loop.

[0003] In the era when domestic automation technology is not mature, the wire bonding machine is manually used to realize the above-mentioned function steps. When operating, the worker needs to manually weld the wire by using a microscope. The disadvantage is that the welding quality is uneven and the efficiency is low. With the introduction of foreign automation technology, domestic manufacturers began to imitate the design ideas of foreign countries, and domestic automatic wire bonding machines gradually became the market mainstream. The automatic wire bonding machine can realize automatic welding. The worker only needs to set the welding path in advance. After setting, only the feeding and discharging operation is needed, which greatly improves the production efficiency.

[0004] At present, most of the wire bonding matching methods at home and abroad directly select the invariant moment method for matching. However, since the profile of the semiconductor device on the chip is relatively simple and the internal texture is similar, there may be a very high similarity in texture between two semiconductor devices with similar profiles. If only the invariant moment matching method is used to identify the semiconductor device, the error rate will be high, which will reduce the yield of the chip. Moreover, the matching method of using invariant moment alone is slow in identifying some semiconductor devices with large volume and complex texture, which consumes too much time and affects the production efficiency. SUMMARY

[0005] The research target of the present application is to solve the automatic matching and recognition problem of a wire bonder based on machine vision when realizing wire bonding between semiconductor devices, and to provide a semiconductor wire bonding device recognition method based on image multi-feature matching. Through the combination of the principle of wire bonding machine process and the image matching algorithm, the matching and recognition of the semiconductor devices on the chip that need to be wire bonded are automatically completed. Through the processing of the semiconductor device image captured by the industrial camera by the algorithm, the corresponding number of the current bonding device image in the Modu template library is obtained, and the bonding mode can be obtained according to the number.

[0006] To solve the above technical problems, the technical scheme adopted by the present application is as follows: a semiconductor wire bonding device recognition method based on image multi-feature matching: the method comprises the following steps:

[0007] S1. Establishment of a multi-feature template library of bonding device images

[0008] A template library folder Modu is established, a text file named Data is established in Modu, and the images of the library devices and the corresponding image feature texts are stored in the folder Modu:

[0009] 1) The industrial camera is moved to the mark point position on the chip by manually driving the mechanical mechanism, the mark point position coordinates are stored in the Data text, the mechanical mechanism is driven to move the industrial camera to the first library device position, the position coordinates are stored in the Data text, and the image of the first library device captured by the industrial camera is named Modu0-QS0-0 and stored in the folder Modu;

[0010] 2) Move the mouse and frame the first library device in the screen with the mouse, the range of the frame is slightly larger than the library device; the mouse frame image is first processed according to steps S2, S3, the contour perimeter corresponding to the maximum contour area and the center distances d1, d2, d3, d4 and other information are stored in Data, and the texture information obtained after S4 processing is stored in Data;

[0011] 3) The mechanical mechanism is driven to move the industrial camera to the positions of other library devices in the chip in turn, and the images of all other library devices Modu0-QS0-1, Modu0-QS0-2, … are stored in Modu, and the position coordinates of each library device and the information obtained by processing the corresponding mouse frame image according to S2, S3, S4 are stored in Data, and finally the multi-feature template library folder Modu of the library device images is formed;

[0012] S2. Device image preprocessing

[0013] Let the captured image be X, the image X1 is obtained by gray-scale processing of the color image X according to formula (1), and the image X2 is obtained by binary processing of the image X1 according to formula (2):

[0014] X1 = 0.2989 * X R + 0.587 * X G + 0.114 * X B (1)

[0015]

[0016] where X R , X G , X B are the red, green, blue components of the color image X respectively, (x, y) are the horizontal and vertical coordinates of the pixel in the image, and T is the threshold value of binarization.

[0017] S3. Extracting the contour features of the image

[0018] 1) Perform Gaussian filtering on the preprocessed image X2 to obtain X3, where the generating equation of the 3x3 Gaussian filter kernel used is:

[0019]

[0020] 2) Calculate the gradient intensity matrix G of X3, where c is the current position point in X3, and each element G xy According to formulas (4), (5), (6), and (7):

[0021]

[0022]

[0023]

[0024]

[0025] where G x , G y represent the gradient values of pixel point c in the x and y directions respectively, S x , S y are the templates, * is the convolution symbol, A is a 3x3 window, sum represents the sum of all elements in the matrix, G xy is the gradient value of pixel point c, θ represents the gradient direction, and arctan is the inverse tangent function.

[0026] 3) Perform non-maximum suppression on the gradient intensity matrix G to obtain the image X5:

[0027] For the gradient intensity matrix G, g1, g2, g3, g4 are four adjacent pixels in the gradient direction of the current position point c, the gradient intensity value of g1 is M(g1), the gradient intensity value of g2 is M(g2), dtmp1 is the interpolation point of g1 and g2 in the gradient direction of c, dtmp2 is the interpolation point of g3 and g4 in the gradient direction of c, the gradient intensity value M(dtmp1) of dtmp1 and the gradient intensity value M(dtmp2) of dtmp2 are respectively calculated according to formula (8), (9):

[0028] M(dtmp1) = w*M(g2) + (1-w)*M(g1) (8)

[0029] M(dtmp2) = w*M(g3) + (1-w)*M(g4) (9)

[0030] Wherein w = tan(θ);

[0031] If the gradient intensity value M(c) of point c is greater than the gradient intensity values M(dtmp1) and M(dtmp2) of the interpolation points, the gradient intensity value of point c is kept unchanged, otherwise the gradient intensity value of point c is set to 0, and the image obtained by processing G above is denoted as X5;

[0032] 4) The gradient intensity image X6 is obtained by performing double threshold processing on the image X5:

[0033] For the image X5, two threshold values are set: the low threshold minV and the high threshold maxV, the pixel point with the gradient intensity value higher than maxV is assigned a value of 255, and the pixel point with the gradient intensity value lower than minV is assigned a value of 0; for the point with the gradient intensity value between the low threshold and the high threshold, it is further judged whether the gradient intensity values of the eight adjacent pixel points of the point are 255, if not, the value is assigned to 0; if yes, the value is assigned to 255;

[0034] 5) The contour area, the perimeter and the center point distances d1, d2, d3, d4 of the image X6 are calculated;

[0035] All contours of X6 are found, and the contour area is operated as follows: the area in the contour is calculated, and the contour with smaller area is deleted; the maximum value x max and the minimum value x min of the horizontal coordinate of the contour point with the maximum area are found, the maximum value y max and the minimum value y mini of the vertical coordinate are found, and four extreme coordinate points p1(x min , y min ), p2(x max , y min ), p3(x max , y max ), and p4(x min , ymax According to formulas (10) and (11), the center coordinates of the extreme value coordinates are obtained as follows:

[0036]

[0037]

[0038] According to formulas (12), (13), (14), and (15), the distances d1, d2, d3, and d4 between the center coordinates and the four extreme points are obtained:

[0039]

[0040]

[0041]

[0042]

[0043] Where d1 is the distance from the center point m(x, y) to the point p1(x). min ,y min The distance from m(x, y) to point p2(x) is d2. max ,y min The distance from point m to point p3(x) is d3, where d3 is the distance from point m to point p3(x). max ,y max The distance from point m to point p4(x) is d4. min ,y max The distance;

[0044] S4. Extract texture features from the image to obtain parameter H. i

[0045] 1) Contour filtering of gradient intensity images:

[0046] After extracting contour features from an input texture image in step S3, the maximum x1, minimum x2, maximum y1, and minimum y2 of the horizontal coordinates corresponding to the contour with the largest area on its gradient intensity image X6 are found. Then, the contours in image X6 are filtered: if the coordinates (x1, x2, y1, and y2) of a certain contour point are... i y i ) satisfies formulas (16) and (17):

[0047] x2 <x i <x1 (16)

[0048] y2 <y i <y1 (17)

[0049] If the outline is not retained, it will be retained; otherwise, it will be deleted. Image X6 is filtered by outline to obtain image G1.

[0050] 2) Calculate the 7 values h of the Hu moment invariant for the image G1 i as texture features

[0051] Let the pixel value at position (x, y) be I(x, y), then the moment of force of the image is:

[0052] M ij =∑ x ∑ y x i y i I(x,y) (18)

[0053] where i and j are integers (e.g. 0, 1, 2,...);

[0054] the centroid of the image

[0055]

[0056] The central moment is obtained according to formula (20):

[0057]

[0058] The central distance is normalized according to formula (21):

[0059]

[0060] h is calculated according to formula (22) i :

[0061]

[0062] The seven values h of Hu are converted into H i according to formula (23) i :

[0063] H i = -sign(h i ) log |h i | i is an integer taking values from (1, 7) (23)

[0064] where log is the logarithmic function;

[0065] S5. Multi-feature matching and recognition of images of bonded devices

[0066] 1) Correct the position coordinates of the current bonded device:

[0067] Move the camera from the starting position of the machine table, find the position of the mark point on the current chip, and calculate the position difference between the current mark point position coordinate and the mark point position coordinate when the library is built; using the position difference, the relative position of the bonded device and the mark point, correct the position coordinate of the current bonded device;

[0068] 2) Take a picture of the bonded device and perform image preprocessing:

[0069] Drive the mechanical mechanism to move the industrial camera to the position of the corrected bonded device, take a picture of each bonded device, and then perform image preprocessing of S2;

[0070] 3) Bonded device contour matching:

[0071] First, perform preprocessing of step S2 and contour feature extraction of S3 on each bonded device image;

[0072] Since the field of view of the camera is larger than the bonded device, there are multiple contours in the camera image in addition to the bonded device contour. For this, according to formula (24), find the best matching contour k for all contours obtained in step S3 m (0 < m ≤ U), so that the center distance is minimized:

[0073]

[0074] Where d m，i is the extreme center distance of each contour obtained in step S3, d Data is the extreme center distance stored in the Data text at the current position, and U is the total number of contours in the image X6;

[0075] Second, compare the perimeter and area of the obtained best contour with the contour area and perimeter in the Data text at the same position in the device library. If the absolute value of the perimeter error is less than 50 pixels and the absolute value of the area error is less than 300 pixels, obtain the bonded device number stored in the Data text, perform bonded device texture data matching, otherwise, the matching fails;

[0076] 4) Bonded device texture data matching:

[0077] Obtain the texture parameters of the current bonded device image by processing step S4, and then calculate the texture matching value W according to formula (25):

[0078]

[0079] Where is the texture parameter of the current bonded device, is the texture parameter stored in the Data text with the number obtained in step 3, i.e. the seven values of Hu moments Hi ;

[0080] If W < 0.0001, the matching is successful, and the number of the bonded device stored in the library is obtained.

[0081] The advantages and positive effects of the present application are:

[0082] 1. Since the profiles of most semiconductor devices on the chip are relatively simple, the profile is a relatively obvious feature of the semiconductor device, and at the same time, there are differences in the internal texture of each semiconductor device, so single profile feature or single texture feature cannot achieve accurate identification, resulting in a high error rate in identifying semiconductor devices by single feature. In the present application, the peripheral profile of the image is identified by using the profile feature first, which can quickly determine whether the profile of the current bonded device exists in the picture taken by the camera, thereby improving the identification efficiency to a certain extent. If the profile of the current bonded device exists, the internal texture of the semiconductor device is determined by using the texture feature to determine whether it matches the current bonded device.

[0083] 2. The larger the volume of the bonded device and the more complex the internal texture, the longer the processing time when extracting the texture information of the image, and other problems. In the present application, in order to solve this problem, the image processing method in step S3 is used for the picture taken by the camera, so that a picture with high complexity becomes an image with only simple lines, greatly improving the speed of obtaining the texture information of the bonded device, and greatly reducing the time cost in this process. The effect will be more obvious as the volume of the semiconductor device becomes larger and the texture becomes more complex.

[0084] 3. The semiconductor lead bonding device identification method based on image multi-feature matching provided by the present application effectively solves the problem of low accuracy of single-feature technology in identifying bonded devices, and can accurately identify bonded devices with similar profiles on the chip, greatly improving the accuracy of identifying bonded devices, and enabling the lead process to be automated and efficient. BRIEF DESCRIPTION OF DRAWINGS

[0085] Figure 1 is a flowchart of the present application.

[0086] Figure 2 is a bonded device image taken by the camera of the present application.

[0087] Figure 3 is the best matching profile image of the bonded device of the present application.

[0088] Figure 4 is a Modu template library screenshot image of the present application.

[0089] Figure 5 is a Data text screenshot image of the present application.

[0090] Figure 6 is a schematic diagram of non-maximum suppression of gradient intensity matrix G. DETAILED DESCRIPTION

[0091] In order to make the purpose, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below.

[0092] A semiconductor lead bonding device recognition method based on image multi-feature matching, comprising the following steps:

[0093] S1. Establishment of a multi-feature template library of bonding device images;

[0094] A template library folder Modu is established, a text file named Data is established in Modu, and the images of the library devices and the corresponding image feature texts are stored in the folder Modu:

[0095] 1) The industrial camera is moved to the mark point position on the chip by manually driving the mechanical mechanism, the mark point position coordinates are stored in the Data text, the mechanical mechanism is driven to move the industrial camera to the first library device position, the position coordinates are stored in the Data text, and the image of the first library device taken by the industrial camera is named Modu0-QS0-0 and stored in the folder Modu;

[0096] 2) Move the mouse and frame the first library device (the range of the selection frame is slightly larger than the library device) in the screen with the mouse; the mouse framed image is first processed according to steps S2 and S3, the contour perimeter corresponding to the maximum contour area and the center distances d1, d2, d3 and d4 are stored in Data, and the texture information obtained after S4 processing is stored in Data;

[0097] 3) The mechanical mechanism is driven to move the industrial camera to the positions of other library devices in the chip in sequence, and the images of all other library devices Modu0-QS0-1, Modu0-QS0-2, … are stored in Modu, and the position coordinates of each library device and the information obtained by processing the mouse framed image according to S2, S3 and S4 are stored in Data, and finally the multi-feature template library folder Modu of library device images is formed.

[0098] S2, device image preprocessing

[0099] Let the taken image be X, the image X1 is obtained by grayscale processing of the color image X according to formula (1), and the image X2 is obtained by binaryzation processing of the image X1 according to formula (2):

[0100] X1 = 0.2989 * XR +0.587*X G +0.114*X B (1)

[0101]

[0102] Where X R X G X B , respectively, are the red, green and blue components of the color image X, (x,y) are the horizontal and vertical coordinates of the pixels in the image, and T is the binarization threshold;

[0103] S3. Extract the contour features of the image.

[0104] 1) Gaussian filtering is applied to the preprocessed image X2 to obtain X3. The generation equation of the 3x3 Gaussian filter kernel used is:

[0105]

[0106] 2) Calculate the gradient intensity matrix G for X3, where c is the current position point in X3, and each element G in G is a gradient intensity matrix. xy Calculated according to formulas (4), (5), (6), and (7):

[0107]

[0108]

[0109]

[0110]

[0111] Among them, G x G y S represents the gradient values ​​of pixel c in the x and y directions, respectively. x S y The template is `*`, the convolution symbol is `A`, the window size is `3x3`, `sum` represents summing all elements in the matrix, and `G` represents the convolution symbol. xy Let θ be the gradient value of pixel c, θ represent the gradient direction, and arctan be the arctangent function;

[0112] 3) Non-maximum suppression is applied to the gradient intensity matrix G to obtain image X5:

[0113] like Figure 6As shown, for the gradient intensity matrix G, let g1, g2, g3, and g4 be the four adjacent pixels of the current position point c along the gradient direction, with the gradient intensity value of g1 being M(g1) and the gradient intensity value of g2 being M(g2). dtmp1 is the interpolation point of g1 and g2 along the gradient direction c, and dtmp2 is the interpolation point of g3 and g4 along the gradient direction c. According to formulas (8) and (9), the gradient intensity values ​​M(dtmp1) of dtmp1 and M(dtmp2) of dtmp2 are calculated respectively:

[0114] M(dtmp1)=w*M(g2)+(1-w)*M(g1) (8)

[0115] M(dtmp2)=w*M(g3)+(1-w)*M(g4) (9)

[0116] Its w = tan(θ),

[0117] If the gradient intensity value M(c) at point c is greater than the gradient intensity values ​​M(dtmp1) and M(dtmp2) at the interpolation point, then the gradient intensity value at point c is kept unchanged; otherwise, the gradient intensity value at point c is set to 0. The image obtained by applying the above non-maximum suppression to G is denoted as X5.

[0118] 4) Perform double thresholding on image X5 to obtain gradient intensity image X6:

[0119] For image X5, set two thresholds: a low threshold minV and a high threshold maxV. Assign a value of 255 to pixels with gradient intensity values ​​higher than maxV and a value of 0 to pixels with gradient intensity values ​​lower than minV. For pixels with gradient intensity values ​​between the low and high thresholds, check if there are any pixels with gradient intensity values ​​of 255 in the eight neighboring pixels. If not, assign a value of 0; if so, assign a value of 255.

[0120] 5) Calculate the contour area, perimeter, and center point distances d1, d2, d3, and d4 of image X6. d1, d2, d3, and d4 are features shared by all contours. They can not only quickly determine the contour position of the current bonding device, but also provide simple and fast calculations while accurately reflecting the contour characteristics of semiconductor devices.

[0121] Find the contour of X6, and perform the following operations on the contour region: calculate the area within the contour; find the maximum x-coordinate among the contour points. max and minimum value x min , the maximum value of the ordinate y max and minimum value y mini Four extreme coordinate points p1(x) were obtained. min ,y min p2(x) max ,ymin ), p3(x max , y max ), p4(x min , y max ); the center coordinates of the extreme value coordinates are obtained according to formulas (10), (11):

[0122]

[0123]

[0124] The distances d1, d2, d3, d4 of the center coordinates and the four extreme points are obtained according to formulas (12), (13), (14), (15):

[0125]

[0126]

[0127]

[0128]

[0129] wherein d1 is the distance from the center point m(x, y) to the point p1(x min , y min ), d2 is the distance from m(x, y) to the point p2(x max , y min ), d3 is the distance from the point m to the point p3(x max , y max ), and d4 is the distance from the point m to the point p4(x min , y max ).

[0130] Table 1 shows the values of the contours d1, d2, d3, d4 in the bonded device image

[0131] d1 [d2] [d3] [d4] 8.6713 14.7672 10.2481 20.3264 13.1639 15.1887 16.6096 14.7807 22.9123 16.3885 27.7422 24.5362 5.2431 5.5435 4.4667 4.6872 9.4561 10.4566 8.324 8.2415

[0132] S4. Texture feature extraction is performed on the image to obtain the parameter H i

[0133] The maximum value x1 and the minimum value x2 of the abscissa and the maximum value y1 and the minimum value y2 of the ordinate of the best matching contour coordinate points on the image X6 are obtained. Let x i , y i be the abscissa and the ordinate of the remaining contour coordinate points in the image X6. Then:

[0134] x2 < x i < x1 (16)

[0135] y2 < yi <y1 (17)

[0136] If there exists a contour point whose x-coordinate satisfies (16) and y-coordinate satisfies (17), then retain the contour; otherwise, delete the contour and obtain image G1.

[0137] For an input image G1, let the pixel value at position (x,y) be I(x,y). Find the seven values ​​h of its Hu invariant moments. i Among them, the torque of the image:

[0138] M ij =∑ x ∑ y x i y i I(x,y) (18)

[0139] Where i and j are integers (e.g., 0, 1, 2, ...).

[0140] Image Center of mass:

[0141]

[0142] The central moment is obtained according to formula (20):

[0143]

[0144] The center distance is normalized according to formula (21):

[0145]

[0146] h is calculated according to formula (22) i :

[0147]

[0148] The seven values ​​h of Hu are obtained through formula (23). i Convert to H i :

[0149] H i =-sign(h i )log|h i | (i is an integer value between (1, 7)) (23)

[0150] Where log is the logarithmic function.

[0151] Image multi-feature matching and recognition of S5 bonding devices

[0152] 1) Correct the position coordinates of the current bonded device:

[0153] Move the camera from the starting position of the machine, find the position of the mark point on the current chip, and calculate the position difference between the current mark point position coordinate and the mark point position coordinate when the library is built; using the position difference, the relative position of the bonded device and the mark point, correct the position coordinate of the current bonded device.

[0154] 2) Take a picture of the bonded device and perform image preprocessing:

[0155] Drive the mechanical mechanism to move the industrial camera to the position of the corrected bonded device, take a picture of each bonded device, and then perform image preprocessing of S2;

[0156] 3) Bonded device contour matching:

[0157] First, perform the preprocessing of step S2 and the contour feature extraction of S3 for each bonded device image. Since the field of view of the camera is larger than the bonded device, there are multiple contours in the camera image in addition to the bonded device contour. For this, for all contours of the gradient intensity image X6 in step S3, find the best matching contour k according to formula (24) m (0 < m < U), so that the center distance is minimized:

[0158]

[0159] Where d m，i is the extreme center distance of each contour obtained in step S3, d Data is the extreme center distance stored in the Data text corresponding to the current position, and U is the total number of contours in the image X6.

[0160] The pictures of the two bonded devices on the chip taken by the camera are named P1 and P2. After processing the images P1 and P2 through steps S2 and S3, the d1, d2, d3, and d4 corresponding to the minimum k value are obtained, and the best matching contour of the bonded device on the image P1 and P2 is determined:

[0161] Table 2 shows the calculation results of the minimum k value

[0162] d1 [d2] [d3] [d4] k [P2] 18.6532 18.6532 24.0365 24.0136 21.3021 0 [P2] 13.1639 15.1887 16.6096 14.7807 0.0021

[0163] Secondly, compare the perimeter and area of the obtained best contour with the contour area and perimeter in the Data text at the same position. If the absolute value of the error of the perimeter and area is within 50 and 300 respectively, obtain the bonded device number stored in the Data text, and perform step 4 of S5. Otherwise, the matching fails.

[0164] Match the perimeter and area information of the bonded device contour on the image P1 and P2 with the data in the Data:

[0165] Table 3 Area and Perimeter of the Best Matched Contour Display

[0166] image [P2] [P2] area 145947 63132 perimeter 1500 999

[0167] Table 4 Area and Perimeter Matching Results in Data Display

[0168] image [P2] [P2] area error 120 89 perimeter error 20 32 whether in error range yes yes contour matching success yes yes

[0169] 4) Bonding device texture data matching:

[0170] The image of the current bonding device is processed by step S4 to obtain the texture parameters, and then the texture matching value W is calculated according to formula (25):

[0171]

[0172] Wherein is the texture parameter of the current bonding device, is the texture parameter of the same library device number in the Data text in the template library, that is, the seven values H of the Hu moment. i .

[0173] If W < 0.0001, the matching is successful, and the number of the bonding device stored in the library is obtained.

[0174] The H i value of the bonding device on the image P1, P2 is obtained:

[0175] Table 5 H i value results of images P1, P2 are displayed

[0176] [P2] [P2] 1.19369 2.15062 3.46168 5.32814 4.91179 6.26544 7.11953 6.35606 13.2855 12.6668 9.19571 9.02637 13.2859 -14.5871

[0177] The texture parameter information of the bonding device on the image P1, P2 is calculated with the data in Data, and whether the matching is successful is judged according to the value of W:

[0178] Table 6 Image P1, P2 texture information matching result display

[0179] W 0.0000034 0.00000073 matching success yes yes number acquisition Modu0-QS0-4 Modu0-QS0-8

[0180] Due to the large field of view of the camera, there can be multiple semiconductor devices in the current bonding device image taken, which can cause the number of contours in the image to be large. After the image is processed by steps S2 and S3, the complexity of the image can be simplified, the information of the image contour is more prominent, the contour information of the semiconductor device is easily obtained, and the texture features of the image are also easily extracted subsequently.

[0181] If the image processing in step S3 is not performed on the semiconductor device image taken by the camera, and the texture feature is directly extracted from the original image (equivalent to the method of matching only using invariant moments in the background art), the complexity of the image processing and the time consumed for extracting the texture information will be greatly increased, and it is more susceptible to the intensity of the light source. After the image processing in step S3, the image taken by the camera with certain complexity is changed into an image with only simple texture lines, not only the processing speed of the image is greatly increased, but also the sensitivity of the semiconductor device to the light source is reduced in the actual application, and the correct rate of the recognition is improved.

[0182] In addition, the running speed of the method of the present application and the existing single-feature invariant moment are compared in Table 7. From the above table, it can be seen that the matching speed is obviously improved by using the multi-feature matching in the present application.

[0183] Table 7: Matching time comparison data

[0184]

[0185]

[0186] The embodiments of the present application are described above in detail, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present application.

Claims

1. A method for identifying semiconductor wire bonding devices based on multi-feature image matching, characterized in that, The method includes the following steps: S1. Establishment of a multi-feature template library for bonding device images Create a template library folder named Modu. Inside Modu, create a text file named Data and save the images of the library devices and their corresponding image feature text into the Modu folder. 1) Manually drive the mechanical mechanism to move the industrial camera to the marked point position on the chip, and store the coordinates of the marked point position in the Data text file; drive the mechanical mechanism to move the industrial camera to the position of the first library device, store the position coordinates in the Data text file, and name the image of the first library device captured by the industrial camera as Modu0-QS0-0 and store it in the folder Modu. 2) Move the mouse and select the first library device in the screen with a mouse box. The selection box should be slightly larger than the library device. The image selected by the mouse box is first processed by steps S2 and S3. The contour perimeter and center distance d1, d2, d3, and d4 information corresponding to the maximum contour area are stored in Data. The texture information obtained after processing by S4 is then stored in Data. 3) Drive the mechanical mechanism to move the industrial camera to the positions of other library devices in the chip in sequence, take pictures of all other library devices Modu0-QS0-1, Modu0-QS0-2, ... and store them in Modu. Then, store the position coordinates of each library device and the information obtained by processing the corresponding mouse selected image through S2, S3, and S4 in Data, and finally form a multi-feature template library folder Modu of library device images. S2. Device Image Preprocessing Let the captured image be X. According to formula (1), the color image X is converted to grayscale to obtain image X1. According to formula (2), image X1 is binarized to obtain image X2. X1=0.2989*X R +0.587*X G +0.114*X B (1) Where X R X G X B , respectively, are the red, green and blue components of the color image X, (x,y) are the horizontal and vertical coordinates of the pixels in the image, and T is the binarization threshold; S3. Extract the contour features of the image. 1) Gaussian filtering is applied to the preprocessed image X2 to obtain X3. The generation equation of the 3x3 Gaussian filter kernel used is: 2) Calculate the gradient intensity matrix G for X3, where c is the current position point in X3, and each element G in G is a gradient intensity matrix. xy Calculated according to formulas (4), (5), (6), and (7): Among them, G x G y S represents the gradient values ​​of pixel c in the x and y directions, respectively. x S y The template is `*`, the convolution symbol is `A`, the window size is `3x3`, `sum` represents summing all elements in the matrix, and `G` represents the convolution symbol. xy Let θ be the gradient value of pixel c, θ represent the gradient direction, and arctan be the arctangent function; 3) Non-maximum suppression is applied to the gradient intensity matrix G to obtain image X5: For the gradient intensity matrix G, let g1, g2, g3, and g4 be the four adjacent pixels of the current position point c along the gradient direction, with the gradient intensity value of g1 being M(g1) and the gradient intensity value of g2 being M(g2). Let dtmp1 be the interpolation point of g1 and g2 along the gradient direction c, and dtmp2 be the interpolation point of g3 and g4 along the gradient direction c. Calculate the gradient intensity values ​​M(dtmp1) and M(dtmp2) of dtmp1 and dtmp2 respectively using formulas (8) and (9): M(dtmp1)=w*M(g2)+(1-w)*M(g1) (8) M(dtmp2)=w*M(g3)+(1-w)*M(g4) (9) Where w = tan(θ); If the gradient intensity value M(c) of point c is greater than the gradient intensity values ​​M(dtmp1) and M(dtmp2) of the interpolation point, then the gradient intensity value of point c is kept unchanged; otherwise, the gradient intensity value of point c is set to 0. The image obtained by G after the above processing is denoted as X5. 4) Perform double thresholding on image X5 to obtain gradient intensity image X6: For image X5, set two thresholds: a low threshold minV and a high threshold maxV. Assign a value of 255 to pixels with gradient intensity values ​​higher than maxV and a value of 0 to pixels with gradient intensity values ​​lower than minV. For pixels with gradient intensity values ​​between the low threshold and the high threshold, check if there are any pixels with gradient intensity values ​​of 255 in the eight neighboring pixels of that pixel. If not, assign a value of 0; if so, assign a value of 255. 5) Find the area, perimeter, and distances d1, d2, d3, and d4 between the center points of the image X6. Find all contours within X6 and perform the following operations on the contour regions: calculate the area within the contours, delete contours with smaller areas; find the maximum x-coordinate of the contour point with the largest area. max and minimum value x min , the maximum value of the ordinate y max and minimum value y mini Four extreme coordinate points p1(x) were obtained. min ,y min p2(x) max ,y min p3(x) max ,y max p4(x) min ,y max According to formulas (10) and (11), the center coordinates of the extreme value coordinates are obtained as follows: According to formulas (12), (13), (14), and (15), the distances d1, d2, d3, and d4 between the center coordinates and the four extreme points are obtained: Where d1 is the distance from the center point m(x, y) to the point p1(x). min ,y min The distance from m(x, y) to point p2(x) is d2. max ,y min The distance from point m to point p3(x) is d3, where d3 is the distance from point m to point p3(x). max ,y max The distance from point m to point p4(x) is d4. min ,y max The distance; S4. Extract texture features from the image to obtain parameter H. i 1) Contour filtering of gradient intensity images: After extracting contour features from an input texture image in step S3, the maximum x1, minimum x2, maximum y1, and minimum y2 of the horizontal coordinates corresponding to the contour with the largest area on its gradient intensity image X6 are found. Then, the contours in image X6 are filtered: if the coordinates (x1, x2, y1, and y2) of a certain contour point are... i y i ) satisfies formulas (16) and (17): x2 <x i <x1 (16) y2 <y i <y1 (17) If the outline is not retained, it will be retained; otherwise, it will be deleted. Image X6 is filtered by outline to obtain image G1. 2) Calculate the seven values ​​h of the Hu invariant moments for the image G1. i As texture features Let the pixel value at position (x,y) be I(x,y), then the moment of the image is: M ij =∑ x ∑ y x i y i I(x,y) (18) Where i and j are integers (e.g., 0, 1, 2...). Image centroid The central moment is obtained according to formula (20): The center distance is normalized according to formula (21): h is calculated according to formula (22) i : The seven values ​​h of Hu are obtained through formula (23). i Convert to H i : H i =-sign(h i )log|h i |i takes the integer value of (1,7) (23) Where log is the logarithmic function; S5. Image Multi-Feature Matching and Recognition of Bonding Devices 1) Correct the position coordinates of the current bonded device: Move the camera from the starting position of the machine to find the position of the current mark point on the chip, and calculate the position difference between the current mark point position coordinates and the mark point position coordinates when the library was built; use the position difference and the relative position of the bonding device and the mark point to correct the position coordinates of the current bonding device. 2) Take pictures of the bonded devices and perform image preprocessing: Drive the mechanical mechanism to move the industrial camera to the position of the corrected bonding device, take pictures of each bonding device, and then perform S2 image preprocessing. 3) Bonded device contour matching: First, each bonding device image undergoes preprocessing in step S2 and contour feature extraction in step S3. Since the field of view of the camera is larger than the bonding device, there are multiple contours in the image captured by the camera in addition to the contour of the bonding device. For this reason, for all the contours obtained from the bonding image through step S3, the best-matching contour k is found according to formula (24) m (0 < m ≤ U), such that the sum of the center distances is minimized: Where d m，i d is the extreme center distance of each contour obtained in step S3. Data The distance between the extreme center of the library device stored in the Data text corresponding to the current position, and U is the total number of contours in image X6; Secondly, the perimeter and area of ​​the obtained best contour are compared with the contour area and perimeter in the Data text at the same location in the device library. If the absolute value of the error in the perimeter is less than 50 pixels and the absolute value of the error in the area is less than 300 pixels, the bonding device number stored in the Data text is obtained and bonding device texture data matching is performed; otherwise, the matching fails. 4) Bonded device texture data matching: The image of the current bonding device is processed in step S4 to obtain texture parameters, and then the texture matching value W is calculated according to formula (25): in For the texture parameters of the current combined device, The texture parameters H obtained in step 3 are stored in the Data text as the seven values ​​of the Hu moment. i ; If W < 0.0001, the match is successful, and the number of the bonding device stored in the library is obtained.

Images