High-precision measurement method for geometric parameters of chip mounting

Abstract

The invention relates to a high-precision measurement method for chip mounting geometric parameters, and relates to the technical field of chip mounting geometric parameter measurement. Comprising the following steps: carrying out distortion correction on a patch element image; carrying out image preprocessing on the patch element image subjected to distortion correction; according to the sub-pixel edge detection algorithm based on the Canny-Franklin moment, firstly, Canny edge coarse positioning is improved, and then sub-pixel edge extraction is carried out based on the Franklin moment; edge fitting and size measurement: after sub-pixel-level accurate positioning is carried out on the edge of the element by using an improved Canny-Franklin moment method, straight line fitting is carried out on the edge through a least square fitting straight line algorithm; and carrying out element edge pixel size, deviation angle and centroid calculation. According to the invention, the positioning precision and the detection efficiency in the chip mounting process can be improved.

Description

Technical Field

[0001] The invention relates to the technical field of chip mounting geometric parameter measurement, in particular to a high-precision chip mounting geometric parameter measurement method. Background Art

[0002] With the rapid development of semiconductor technology and the electronics industry, the chip manufacturing industry is tending to achieve higher integration, smaller size and lower cost. Correspondingly, the accuracy and quality standards of chip mounting are also improved. At present, the detection and positioning tasks in the chip mounting process at home and abroad are mainly completed by computer vision systems. With its automated characteristics, visual inspection technology can replace many traditional manual inspection tasks and effectively solve many common problems in manual inspection.

[0003] In the process of mounting, the precise positioning of the target chip and the accuracy of edge detection are crucial. Pixel-level detection algorithms include first-order differential methods and second-order differential methods. First-order differentials include Prewitt operators, Roberts operators, and Sobel operators, and second-order differentials include Canny operators and Laplacian operators. However, traditional pixel-level edge detection technology can no longer achieve higher measurement standards in terms of positioning accuracy.

[0004] Therefore, how to provide a high-precision measurement method for chip mounting geometric parameters that can improve the measurement accuracy of dimensional parameters has become a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In order to solve at least one technical problem in the background technology, the present invention provides a high-precision measurement method for chip mounting geometric parameters, which is beneficial to improving the positioning accuracy and detection efficiency in the chip mounting process.

[0006] To achieve the above object, the present invention provides a high-precision measurement method for chip mounting geometric parameters, comprising the following steps:

[0007] S1, performs distortion correction on the patch element image;

[0008] S2, performing image preprocessing on the distortion-corrected patch element image;

[0009] S3, sub-pixel edge detection algorithm based on Canny-Franklin moment: firstly improve the Canny edge rough positioning, and then extract the sub-pixel edge based on Franklin moment;

[0010] S4, edge fitting and size measurement: After using the improved Canny-Franklin moment method to accurately locate the edge of the component at the sub-pixel level, a least squares straight line fitting algorithm is used to fit the edge; the pixel size, offset angle and centroid of the component edge are calculated.

[0011] Furthermore, in step S2, the method for performing image preprocessing on the distortion-corrected patch element image comprises the following steps:

[0012] S21, image grayscale conversion;

[0013] S22, hybrid filtering denoising, firstly uses median filtering on the image to remove most of the high-frequency noise, and then uses bilateral filtering to further process the initially smoothed image;

[0014] S23, image segmentation, uses the OTSU method to perform threshold segmentation on the patch image.

[0015] Furthermore, in step S21, a weighted average method is used to convert the color image into a grayscale image, and the formula is as follows:

[0016]

[0017] Among them, (i,j) is the position coordinate of a pixel in the image; Gray(i,j) is the grayscale value of the image, G(i,j), B(i,j) and R(i,j) are the values ​​of the green, blue and red channels of the color image respectively.

[0018] Furthermore, the redundant interference area after image segmentation is optimized, and the optimization process is as follows:

[0019] S231, using 7×7 circular structure elements, based on morphological opening operation to eliminate isolated noise points and small interference areas on the binary image;

[0020] S232, filling the hole area in the image with the same pixel value as the surrounding adjacent area, so as to reduce the missing part in the image and make the target object more complete;

[0021] S233, by setting an appropriate area threshold, delete the connected domains whose area is smaller than the threshold P.

[0022] Furthermore, the process of improving Canny edge rough positioning is as follows:

[0023] 1) Use a hybrid filtering denoising method to smooth the image while retaining edge details and improving image quality;

[0024] 2) Select a 3×3 gradient detection template for detection. The gradient detection template is shown as follows:

[0025]

[0026] Among them, H x is the horizontal gradient convolution kernel, which is used to detect the edge changes of the image in the horizontal direction. y is the vertical gradient convolution kernel, which is used to detect the edge changes of the image in the vertical direction. 45° is the gradient convolution kernel along the 45° direction, which is used to detect the edge of the image in the 45° direction. 135° is the gradient convolution kernel along the 135° direction, which is used to detect the edge of the image in the 135° direction;

[0027] 3) Non-maximum suppression: After calculating the image gradient amplitude and direction, each pixel in the image is traversed one by one to eliminate non-edge pixels. When the gradient value of the pixel is the largest in the adjacent area in the gradient direction, it is retained as an edge point; otherwise, the pixel will be ignored.

[0028] 4) Adaptive threshold selection: Adaptive high and low threshold selection is realized during edge connection through the OTSU algorithm.

[0029] Furthermore, the Franklin moment is created as follows:

[0030] First, the linearly independent function set {a n (x) = 1, 0 ≤ x ≤ 1}

[0031]

[0032] Where: a0 is a constant term, a1 is a linear term, a i is a function term generated by recursion, i ≥ 2; a i =(2i-1-2 p ) / 2 p , p is a positive integer, p is all the integers that satisfy 2 p ≤ the maximum value of 2i-1;

[0033] The above linearly independent function group {a n (x) = 1, 0 ≤ x ≤ 1}, and then perform Gram-Schmidt orthogonalization to obtain the Franklin function system The expressions of the first three basis functions are:

[0034]

[0035] Franklin moments are defined based on Franklin functions. The nth Franklin function is 0≤x≤1, n=0,1,2,… Given an image function f(x,y), 0≤x,y≤1, its (n+m)-order Franklin moment is defined as follows

[0036]

[0037] in, is the Franklin basis function in the y direction, used for orthogonal decomposition in the y-axis direction.

[0038] Furthermore, the method of sub-pixel edge extraction based on Franklin moment is as follows:

[0039] First, calculate the difference f of the edge pixels in the row and column directions r , f c :

[0040]

[0041] Where: f(u,v) represents the pixel value of a certain point in the image, u is the vertical coordinate of the image, v is the horizontal coordinate of the image, i, j are the index values ​​of the row and column where the current pixel is located;

[0042] When f r ≥f c When , the Franklin moments of different orders are calculated as follows:

[0043] F 00 =h+k(1-l) (10)

[0044]

[0045] when hour,

[0046]

[0047] when hour,

[0048]

[0049] Combining the above formulas, we get

[0050]

[0051] in, l is the theoretical distance from the origin to the edge; k is the grayscale difference; h is the background grayscale; F 00 It is a special case of Franklin moment, representing the zero-order zero-order moment; F 01 and F 02are the first-order and second-order moments of Franklin moments; l1 and l2 are two theoretical distances calculated based on Franklin moments, representing the theoretical distance from a reference point in the image to the edge;

[0052] The final sub-pixel coordinate formula is:

[0053]

[0054] Where (x', y') is the sub-pixel coordinate of the edge of the image, and (x, y) is the coordinate of the origin;

[0055] When f r <f c When , the Franklin moments of different orders are calculated as follows:

[0056] F 00 =h+k(1-l) (16)

[0057]

[0058] when hour,

[0059]

[0060] when hour,

[0061]

[0062] Combining the above formulas, we get

[0063]

[0064] in,

[0065] The final sub-pixel coordinate formula is:

[0066]

[0067] Where (x', y') is the sub-pixel coordinate of the edge of the image, and (x, y) is the coordinate of the origin.

[0068] Furthermore, the edge data is segmented before the edge is fitted with a line by the least squares straight line fitting algorithm, as follows:

[0069] First, find the four boundary extreme values ​​X from the sub-pixel edge contour point set S max , Y max , X min , Y min , and use this to find the coordinates of the contour midpoint (X m ,Ym ); then find all X=X in the point set S min Then find the point with the smallest Y (X min ,Y0); if

[0070] |Y0-Y min |≤T (22)

[0071] If the component has no offset angle, it is considered that the component has an offset angle.

[0072] When the element has no offset angle, the sub-pixel precision edge contour point set is initially divided into four subsets according to formula (23);

[0073]

[0074] Then, each contour subset is traversed in the x or y direction, and the mode is calculated as K1, K2, K3, and K4; then, the deviation threshold is set as d, and the subset is further filtered according to formula (24), thereby completing the data division process;

[0075]

[0076] Among them, S t3 is the sub-pixel contour point set of the upper edge, S t1 is the sub-pixel contour point set of the lower edge, S t2 is the sub-pixel contour point set of the right edge, S t4 is the sub-pixel contour point set of the left edge;

[0077] When the component has an offset angle, traverse the point set S to obtain the coordinates of the four boundary extreme points (X min ,Y1),(X1,Y min ), (X max ,Y2),(X2,Y max ); When the component has an offset angle, the coordinates of the boundary extreme points are not equal and unique. Based on this feature, the edge is segmented, and the four edge point sets to be fitted are segmented according to formula (25);

[0078]

[0079] Furthermore, the method for calculating the pixel size, offset angle and centroid of the component edge is as follows:

[0080] Assume that the four fitted straight lines are line1, line2, line3, and line4, and specify S t3 , S r3 The edge of the point set fitting is the upper edge line1, S t1 , Sr4 The edge of the point set fitting is the lower edge line3; the calculation formula of the intersection point is based on the parametric equation of the two straight lines; the coordinates of the intersection point between the specific straight lines are as follows

[0081]

[0082] Then calculate the average distance between the two straight lines; the calculation of the average distance is based on sampling and averaging the vertical distance between the two straight lines within a given range; the average distance D1 between line1 and line3, and the average distance D2 between line2 and line4 are shown as follows:

[0083]

[0084] Where: d i (line1, line3) is the vertical distance between the i-th sampling point in line1 and line3, N is the total number of sampling points; d j (line2, line4) is the vertical distance between the jth sampling point in line2 and line4, and M is the total number of sampling points;

[0085] The angle is calculated based on the slope α of the straight line and converted into an angle system. When the condition of formula (22) is not satisfied, the component has an offset angle. The angle between the upper edge line line1 and the lower edge line line3 is as follows:

[0086]

[0087] Where α1 and α2 are the slopes of line1 and line3 respectively; let the offset angle of the element be the average angle

[0088] θ mean =(θ up +θ bottom ) / 2 (29)

[0089] For a standard square device, the centroid is calculated by averaging the four intersection points, as shown below:

[0090]

[0091] Among them, (x c ,y c ) is the centroid, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the coordinates of the four intersection points respectively;

[0092] Then compensate the centroid coordinates and calculate the coordinates of the four intersection points after compensation, as shown in the following formula:

[0093]

[0094] The final optimized centroid coordinates are:

[0095]

[0096] The beneficial effects of the present invention are:

[0097] Aiming at the problems of offset detection, centroid positioning and size measurement, the present invention combines the improved Canny operator with the Franklin moment to detect sub-pixel edges, and adopts different strategies for edge segmentation according to whether there is an offset angle. The interference of edge noise points is removed by the RANSAC algorithm, and finally the least squares method is used to fit the edge to realize the calculation of component edge pixel size, offset angle and centroid, which is beneficial to improve the positioning accuracy and detection efficiency in the chip mounting process. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 is a flow chart of the method of the present invention;

[0099] Figure 2 A schematic diagram of the chessboard calibration process of the present invention;

[0100] Figure 3 is a hybrid filter graph of the present invention;

[0101] Figure 4 This is the threshold segmentation effect diagram of the present invention;

[0102] Figure 5 This is the edge detection effect diagram of the present invention;

[0103] Figure 6 This is the ideal step model diagram of fr≥fc of the present invention;

[0104] Figure 7 This is the ideal step model diagram of fr<fc of the present invention;

[0105] Figure 8 It is the non-offset edge segmentation map of the present invention;

[0106] Fig. 9 The present invention has an offset edge segmentation map;

[0107] Fig.10 This is a comparison diagram of edge straight line fitting detection of the present invention;

[0108] Fig.11 It is a simulation diagram of different angles of the present invention;

[0109] Fig.12 It is a schematic diagram of the detection result of the simulated image of the present invention. DETAILED DESCRIPTION

[0110] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0111] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0112] In the present application, the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings. These terms are mainly used to better describe the present application and its embodiments, and are not used to limit the indicated devices, elements or components to have a specific orientation, or to be constructed and operated in a specific orientation.

[0113] In addition, some of the above terms may be used to express other meanings in addition to indicating orientation or positional relationship. For example, the term "on" may also be used to express a certain dependency or connection relationship in some cases. For those of ordinary skill in the art, the specific meanings of these terms in this application can be understood according to specific circumstances.

[0114] In addition, the terms "installed", "set", "provided with", "connected", "connected", and "socketed" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection, or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be an internal connection between two devices, elements, or components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0115] To achieve the above purpose, Figure 1 As shown, the present invention provides a high-precision measurement method for chip mounting geometric parameters, comprising the following steps:

[0116] S1, performs distortion correction on the patch element image;

[0117] In order to ensure high-precision measurement of workpiece images, distortion correction is required. The purpose of distortion correction is to obtain the compensation pixel position corresponding to each pixel in the workpiece image, compensate for the distorted image, and thus ensure high-precision measurement of geometric parameters. Through distortion correction, the influence of image distortion on the measurement results can be effectively eliminated, and the measurement accuracy of dimensional parameters can be improved.

[0118] The Zhang Zhengyou calibration method is used to correct the distorted image, and a circular calibration plate is used in the calibration process. The circular calibration plate can provide a good reference benchmark during the image calibration process, which helps to improve the accuracy of the calibration. The specific conditions of the calibration process and the correction error are as follows: Figure 2 As shown in Table 1, these charts can provide a more intuitive understanding of the calibration effect and correction accuracy. After multiple measurement experiments, the measurement accuracy can be improved by 2 to 3um when distortion compensation is performed.

[0119]

[0120]

[0121] Table 1

[0122] S2, image preprocessing of the distortion-corrected patch element image:

[0123] S21, image grayscale conversion;

[0124] Since the measurement algorithm is concerned with the grayscale information of the image and not the color information, and the grayscale image can simplify calculations, increase processing speed, remove color interference, and improve contrast and details.

[0125] To convert a color image into a grayscale image, a weighted average method is used, and the formula is as follows:

[0126]

[0127] Among them, (i,j) is the position coordinate of a pixel in the image; Gray(i,j) is the grayscale value of the image, G(i,j), B(i,j) and R(i,j) are the values ​​of the green, blue and red channels of the color image respectively.

[0128] S22, hybrid filtering denoising, firstly uses median filtering on the image to remove most of the high-frequency noise, and then uses bilateral filtering to further process the initially smoothed image;

[0129] When filtering the image, it is necessary to ensure that more edge details are retained to avoid weakening edge information, thereby avoiding missing some weak edges and isolated edges, which will affect the contour fitting effect. In order to remove the noise caused by uneven illumination and component wear in the patch component image, a hybrid filtering denoising method is used.

[0130] Median filtering can effectively smooth images and eliminate most noise, but it may cause loss of edge details. Bilateral filtering is a nonlinear filtering technology that can simultaneously consider the spatial information and pixel intensity information of the image. While removing noise, it can effectively retain the edges and details of the image. Combining the advantages and disadvantages of the two filtering methods, median filtering is first applied to the image to remove most of the high-frequency noise, and then bilateral filtering is used to further process the initially smoothed image. By adopting this hybrid filtering denoising method, it can not only reduce noise, but also protect edge details, avoid affecting the final measurement results and image processing effects due to information loss during the filtering process, and ensure that the contour fitting effect of the image is not affected. The effect of the patch component image after hybrid filtering is shown in Figure 3 ( Figure 3 (a) is the effect diagram of hybrid filtering. Figure 3 (b) is the mixed filtering edge effect diagram), the image edge integrity is good.

[0131] S23, image segmentation, uses the OTSU method to perform threshold segmentation on the patch image.

[0132] In actual detection, the edge can be found by image segmentation. The Otsu method (OTSU) is a classic and effective threshold segmentation method, which can adaptively determine the optimal threshold to maximize the pixel variance between the target object and the background. Therefore, the OTSU method is used to perform threshold segmentation on the patch image.

[0133] However, the image obtained by the OTSU method may still contain some interference information caused by surface defects of components, uneven illumination and other uncertain factors. These interference information will cause the edge of the interference area to be misjudged as the target edge during edge detection, which is not conducive to subsequent size measurement. Therefore, after threshold segmentation, these interference information needs to be properly processed to improve detection accuracy. The optimization process for redundant interference areas is mainly as follows:

[0134] 1) Using 7×7 circular structural elements, based on morphological opening operation, isolated noise points and small interference areas on the binary image are eliminated. This method can effectively remove small noise and make the image clearer.

[0135] 2) Fill the hole areas in the image with the same pixel values ​​as the surrounding adjacent areas to reduce the missing parts in the image and make the target object more complete. This can ensure the integrity of the image and improve the accuracy of subsequent processing.

[0136] 3) By setting an appropriate area threshold and deleting the connected domains with an area smaller than the threshold P, small connected areas caused by noise and other interference factors can be eliminated, and only the main part of the target is retained. Figure 4 The optimization effect is that the threshold is selected as 60% of the number of image lines, where Figure 4 (a) is the segmentation effect diagram of OTSU method. Figure 4 (b) is the optimization effect diagram.

[0137] S3, sub-pixel edge detection algorithm based on Canny-Franklin moment: firstly improve the Canny edge rough positioning, and then extract the sub-pixel edge based on Franklin moment;

[0138] In order to detect the contour more accurately, an edge detection method is used. Edge detection based on the Canny operator is a commonly used edge detection method. Considering the wear, uneven lighting and offset angle that may appear on the edge of the patch component image, an optimization algorithm is proposed based on the existing algorithm to achieve accurate extraction of the component edge.

[0139] The process of improving Canny edge coarse positioning is as follows:

[0140] 1) The hybrid filtering denoising method proposed in this paper is used to replace the traditional Gaussian filtering to remove noise in the image. The hybrid filtering method includes a combination of median filtering and bilateral filtering, which can effectively use the hybrid filtering denoising method to smooth the image while retaining edge details and improving image quality;

[0141] 2) Select a 3×3 gradient detection template for detection, and add detection in the 45° and 135° directions on this basis. This improved template can better detect component edges with offset angles, improve detection accuracy, and effectively suppress noise. The gradient detection template is shown in the following formula:

[0142]

[0143] Among them, H x is the horizontal gradient convolution kernel, which is used to detect the edge changes of the image in the horizontal direction. y is the vertical gradient convolution kernel, which is used to detect the edge changes of the image in the vertical direction. 45° is the gradient convolution kernel along the 45° direction, which is used to detect the edge of the image in the 45° direction. 135° is the gradient convolution kernel along the 135° direction, which is used to detect the edge of the image in the 135° direction;

[0144] 3) Non-maximum suppression: After calculating the image gradient amplitude and direction, each pixel in the image is traversed one by one to eliminate non-edge pixels. When the gradient value of the pixel is the largest in the adjacent area in the gradient direction, it is retained as an edge point; otherwise, the pixel will be ignored.

[0145] 4) Adaptive threshold selection: A suitable threshold can effectively reduce noise interference in edge detection and reduce edge irregularity. The OTSU algorithm is used to select adaptive high and low thresholds for edge connection. The OTSU algorithm obtains the threshold T with the largest variance between the target and the background, which is the optimal threshold and is set as the high threshold T. h , the lower threshold is set to T l , through experimental analysis, the relationship between the two is T l =0.3T h .

[0146] If the gradient value of the current edge pixel is greater than the high threshold T h , it is marked as a strong edge. Otherwise, the pixel is suppressed and not retained. If the gradient value of the current edge pixel is between the high threshold T h With low threshold T l If there are strong edge points in the 8-neighborhood of these virtual edge points, they will be marked as virtual edges (need to be temporarily retained). For these virtual edge points, if there are strong edge points in their 8-neighborhood, the virtual edge points will be re-determined as strong edges. This process is repeated until all edge points are connected. Figure 5 Comparison of the detection results between the traditional Canny operator and the algorithm in this paper ( Figure 5 (a) To improve the Canny operator detection effect diagram, Figure 5 (b) is the effect of traditional Canny detection. When there is interference on the edge of the image, the improved Canny operator has better edge detection effect.

[0147] The Franklin function proposed by Philip Franklin is defined in L 2 A continuous orthogonal function system on [0,1] obtained by orthogonalizing a set of linearly independent truncated power bases.

[0148] The Franklin moment is created as follows:

[0149] First, the linearly independent function set {a n (x) = 1, 0 ≤ x ≤ 1}

[0150]

[0151] Where: a0 is a constant term, a1 is a linear term, a i is a function term generated by recursion, i ≥ 2; a i=(2i-1-2 p ) / 2 p , p is a positive integer, p is all the integers that satisfy 2 p ≤ the maximum value of 2i-1;

[0152] The above linearly independent function group {a n (x) = 1, 0 ≤ x ≤ 1}, and then perform Gram-Schmidt orthogonalization to obtain the Franklin function system The expressions of the first three basis functions are:

[0153]

[0154] Franklin moments are defined based on Franklin functions. The nth Franklin function is 0≤x≤1, n=0,1,2,… Given an image function f(x,y), 0≤x,y≤1, its (n+m)-order Franklin moment is defined as follows

[0155]

[0156] in, is the Franklin basis function in the y direction, used for orthogonal decomposition in the y-axis direction.

[0157] Franklin moments provide a quantitative description of image features. As an orthogonal moment, it allows lossless decomposition of images, which means that different Franklin moments are independent and uncorrelated. This property ensures that the rich features of the image can be captured by the smallest set of moments, thereby achieving effective dimensionality reduction of the feature space.

[0158] The method of sub-pixel edge extraction based on Franklin moment is as follows:

[0159] The ideal edge model is Figure 6 and Figure 7 As shown in the figure. The square in the figure represents a single pixel unit, and the intersection of the straight line L and the square is the ideal edge. The grayscale values ​​on both sides of L are h and h+k, k is the grayscale difference, and l is the theoretical distance from the origin to the edge.

[0160] First, calculate the difference f of the edge pixels in the row and column directions r , f c :

[0161]

[0162] Where: f(u,v) represents the pixel value of a certain point in the image, u is the vertical coordinate of the image, v is the horizontal coordinate of the image, i, j are the index values ​​of the row and column where the current pixel is located;

[0163] according to Figure 6 , when f r ≥f c When , the Franklin moments of different orders are calculated as follows:

[0164] F 00 =h+k(1-l) (10)

[0165]

[0166] when hour,

[0167]

[0168] when hour,

[0169]

[0170] Combining the above formulas, we get

[0171]

[0172] Where l is the theoretical distance from the origin to the edge; k is the grayscale difference; h is the background grayscale; F 00 It is a special case of Franklin moment, representing the zero-order zero-order moment; F 01 and F 02 are the first and second order moments of Franklin moments; l1 and l2 are two theoretical distances calculated based on Franklin moments, which represent the theoretical distance from a reference point (such as the center of a pixel) to the edge in the image. When l1 and l2 fall within a specific range, the exact position of the sub-pixel edge can be calculated by a specific formula;

[0173] The final sub-pixel coordinate formula is:

[0174]

[0175] Where (x', y') is the sub-pixel coordinate of the edge of the image, (x, y) is Figure 6 The coordinates of the origin;

[0176] according to Figure 7 , when f r <f c When , the Franklin moments of different orders are calculated as follows:

[0177] F 00=h+k(1-l) (16)

[0178]

[0179] when hour,

[0180]

[0181] when hour,

[0182]

[0183] Combining the above formulas, we get

[0184]

[0185] in,

[0186] The final sub-pixel coordinate formula is:

[0187]

[0188] Where (x', y') is the sub-pixel coordinate of the edge of the image, (x, y) is Figure 7 The coordinates of the origin.

[0189] S4, edge fitting and size measurement: After using the improved Canny-Franklin moment method to accurately locate the edge of the component at the sub-pixel level, a least squares straight line fitting algorithm is used to fit the edge; the pixel size, offset angle and centroid of the component edge are calculated.

[0190] Before fitting the edge with a line using the least squares straight line fitting algorithm, the edge data is segmented as follows:

[0191] Considering the two cases where the component has an offset angle and no offset angle. First, find the four boundary extreme values ​​X from the sub-pixel edge contour point set S. max , Y max , X min , Y min , and use this to find the coordinates of the contour midpoint (X m ,Y m ); then find all X=X in the point set S min Then find the point with the smallest Y (X min ,Y0); if

[0192] |Y0-Y min |≤T (22)

[0193] It is considered that the element has no offset angle, otherwise, it is considered that the element has an offset angle; the present invention sets the value of T to 30.

[0194] When the element has no offset angle, the sub-pixel precision edge contour point set is initially divided into four subsets according to formula (23);

[0195]

[0196] Then, each contour subset is traversed in the x or y direction, and the mode is calculated as K1, K2, K3, and K4; then, the deviation threshold is set as d, and the subset is further filtered according to formula (24), thereby completing the data division process;

[0197]

[0198] Among them, S t3 is the sub-pixel contour point set of the upper edge, S t1 is the sub-pixel contour point set of the lower edge, S t2 is the sub-pixel contour point set of the right edge, S t4 is the sub-pixel contour point set of the left edge;

[0199] The segmentation effect is as follows Figure 8 As shown. Figure 8 It can be seen that edge segmentation is sometimes unstable and there may be points that affect the fitting edge.

[0200] When the component has an offset angle, since the direction of the offset angle is unknown, a new segmentation strategy is proposed to adapt to the characteristics of the component image when the offset angle occurs. Traverse the point set S to obtain the coordinates of the four boundary extreme points (X min ,Y1),(X1,Y min ), (X max ,Y2),(X2,Y max ); When the component has an offset angle, the coordinates of the boundary extreme points are not equal and unique. Based on this feature, the edge is segmented, and the four edge point sets to be fitted are segmented according to formula (25);

[0201]

[0202] This new segmentation method can effectively cope with the changes in component images caused by the offset angle and improve the robustness and accuracy of image processing. Fig. 9 The edge segmentation results of component images at different angles are presented.

[0203] The patch edge may be damaged, dirty or the edge segmentation effect is not ideal, which may cause deviation points in the edge line point set. These deviation points are called outliers. The existence of outliers will interfere with the fitting of edge lines and affect the accuracy of dimension measurement. Therefore, the least squares method is combined with the RANSAC algorithm to fit the four edge lines to ensure high-precision dimension measurement. The fitting effect is as follows Fig.10 As shown in the figure, the yellow line represents the fitting result using only the least squares method, and the green line represents the optimized fitting of the least squares method combined with the RANSAC algorithm.

[0204] The method for calculating the pixel size, offset angle and centroid of the component edge is as follows:

[0205] Due to manufacturing errors and imaging errors, the fitted straight lines 1 and 3, and straight lines 2 and 4 cannot be guaranteed to be completely parallel, so the pixel size of the component is determined by calculating the average distance. The pixel length and width of the component can be obtained by calculating the average distance between straight lines 1 and 3, and between 2 and 4 in the image.

[0206] Assume that the four fitted straight lines are line1, line2, line3, and line4, and specify S t3 , S r3 The edge of the point set fitting is the upper edge line1, S t1 , S r4 The edge of the point set fitting is the lower edge line3; the calculation formula of the intersection point is based on the parametric equation of the two straight lines; the coordinates of the intersection point between the specific straight lines are as follows

[0207]

[0208] Then calculate the average distance between the two straight lines; the calculation of the average distance is based on sampling and averaging the vertical distance between the two straight lines within a given range; the average distance D1 between line1 and line3, and the average distance D2 between line2 and line4 are shown as follows:

[0209]

[0210] Where: d i (line1, line3) is the vertical distance between the i-th sampling point in line1 and line3, N is the total number of sampling points; d j (line2, line4) is the vertical distance between the jth sampling point in line2 and line4, and M is the total number of sampling points;

[0211] The angle is calculated based on the slope α of the straight line and converted into an angle system. When the condition of formula (22) is not satisfied, the component has an offset angle. The angle between the upper edge line line1 and the lower edge line line3 is as follows:

[0212]

[0213] Where α1 and α2 are the slopes of line1 and line3 respectively; let the offset angle of the element be the average angle

[0214] θ mean =(θ up +θ bottom ) / 2 (29)

[0215] For a standard square device, the centroid is calculated by averaging the four intersection points, as shown below:

[0216]

[0217] Among them, (x c ,y c ) is the centroid, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the coordinates of the four intersection points respectively;

[0218] Then compensate the centroid coordinates and calculate the coordinates of the four intersection points after compensation, as shown in the following formula:

[0219]

[0220] The final optimized centroid coordinates are:

[0221]

[0222] Aiming at the problems of offset detection, centroid positioning and size measurement, the present invention combines the improved Canny operator with the Franklin moment to detect sub-pixel edges, and adopts different strategies for edge segmentation according to whether there is an offset angle. The interference of edge noise points is removed by the RANSAC algorithm, and finally the least squares method is used to fit the edge to realize the calculation of component edge pixel size, offset angle and centroid, which is beneficial to improve the positioning accuracy and detection efficiency in the chip mounting process.

[0223] Application examples:

[0224] Simulation experiment

[0225] Offset angle and center of mass simulation verification

[0226] In order to verify the effectiveness of the proposed algorithm in calculating the offset angle and centroid coordinates in the square edge image, the simulated camera resolution is 1624×1240, and an image containing a square is generated; the angles between the linear edge of the image and the vertical direction are 0°, 5°, 15°, 25°, 35°, 45°, 55°, 65°, 75°, and 85°, respectively, and the centroid coordinates are (812, 620). The image is Fig.11 shown.

[0227] The centroid calculation and angle detection are performed on the simulated image using the algorithm in this paper, the Canny-Zernike algorithm, and the pixel-level algorithm. The results are as follows: Fig.12 shown. Fig.12 (a) In terms of angle detection, the algorithm of the present invention performs the most stable and accurate among all groups, and its absolute error remains at a low level. The specific error value fluctuates between 0 and 0.05, indicating that the measurement accuracy is relatively high. In contrast, the absolute error of the Canny-Zernike algorithm is slightly higher, usually fluctuating between 0.05 and 0.15, showing a certain stability but slightly inferior to the accuracy of the algorithm of the present invention. The absolute error of the pixel-level algorithm fluctuates greatly, especially in the third and eighth groups, where the absolute errors reach 0.5 and 0.6, respectively, which are significantly higher than the other two algorithms. Fig.12 (b) In terms of centroid detection, the absolute error of centroid detection is the Euclidean distance between the detected coordinates and the ideal centroid coordinates; the performance of the algorithm of the present invention and the Canny-Zernike algorithm in each group is relatively close, and the absolute errors are kept at a low level, and the absolute errors in each group are stable between 0 and 0.5, showing a high degree of consistency and accuracy. The absolute error of the Canny-Zernike algorithm is slightly higher, between 0.3 and 0.7, and slightly increased in a few groups. In contrast, the error of the pixel-level algorithm fluctuates greatly, especially in the 3rd and 10th groups, where the absolute errors reach 6 and 2 respectively, indicating poor stability and accuracy in some cases. Therefore, the method of the present invention can effectively perform sub-pixel edge detection of square images to meet measurement requirements.

[0228] SMD component size measurement

[0229] The experimental test uses Hikvision MV-CS020-10GM industrial camera and ML-M2518HR lens of Molybdenum to fix it through the experimental frame to capture images perpendicular to the patch image. The camera resolution is 1624×1240 and the lens focal length is 25mm. Secondly, the white ring LED light source is used to evenly illuminate the workpiece to be inspected, and a high-definition image with low noise is captured. The software platform of the measurement system is built on the Windows 10 64-bit Ultimate operating system; the application layer software uses the C++ programming language and is carried out in the Qt5.14.2 development environment; OpenCV4.7.0 is integrated to support image processing and analysis functions.

[0230] System calibration

[0231] In the image, the size is in pixels. In order to obtain the actual physical size of the patch component, the visual inspection system in the text needs to be calibrated. The present invention uses the standard parts method for calibration. The present invention calibrates with standard parts of known physical size to ensure that the measurement results are accurate and stable. The standard parts are kept consistent with the working environment of the patch component image acquisition; this means that if the experimental environment changes, the calibration value needs to be recalculated, and the pixel equivalent of the system is: 0.0176mm / pix.

[0232] Results Analysis

[0233] When measuring the size, 6 patch components of different sizes were selected as samples for the system measurement accuracy experiment. In order to evaluate the measurement accuracy of the measurement method for components of various sizes, a Mitutoyo digital display vernier caliper with a resolution of 0.001mm was used to measure the length and width of a series of components of different sizes. Each component was measured 10 times and its average value was taken as the actual size. At the same time, in order to test the time efficiency of the algorithm, the running time of the algorithm of the present invention, the Canny-Zernike algorithm and the pixel-level algorithm were recorded in the size measurement, as shown in Table 2; the size measurement data are shown in Tables 3 and 4, Table 3 is the length measurement result, and Table 4 is the width measurement result.

[0234] It can be seen from Table 2 that the algorithm of the present invention has the fastest running speed, the shortest time consumption, and the best real-time performance. Compared with the Canny-Zernike moment algorithm, the running time is reduced by about 21%, and the running time difference with the pixel-level algorithm is very small.

[0235]

[0236] Table 2

[0237] It can be seen from Tables 3 and 4 that the maximum absolute error of the Canny-Zernike algorithm in measurement is 0.04mm and the minimum is 0.016mm; the maximum absolute error of the pixel-level algorithm is 0.101mm and the minimum absolute error is 0.058mm; the maximum absolute error of the algorithm of the present invention is 0.007mm and the minimum absolute error is 0.002mm. The experiment proves that the measurement method of this study meets the requirements of detection accuracy and has practical application significance.

[0238]

[0239] Table 3

[0240]

[0241]

[0242] Table 4

[0243] Conclusion: Aiming at the problems of offset detection, centroid positioning and dimensional measurement accuracy in the chip mounting process, the present invention proposes a high-precision measurement method for chip mounting geometric parameters. To verify the effectiveness of the method, three measurement algorithms were compared respectively. The results show that when the offset angle and centroid are measured by simulation experiments, the angle error of the method of the present invention is less than 0.06°, and the centroid error is less than 0.6 pixels, showing the highest accuracy and stability, with a small error value and low variation. In the actual workpiece measurement, the relative error is within ±0.1%, and the measurement error is less than ±0.008mm, reaching micron-level accuracy, showing strong robustness and consistency. The algorithm running time is shortened by about 21% compared with Canny-Zernike. The results show that the system has good stability and adaptability.

[0244] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any slight modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A high-precision measurement method for chip mounting geometric parameters, characterized in that: The steps include: S1, performs distortion correction on the patch element image; S2, performing image preprocessing on the distortion-corrected patch element image; S3, sub-pixel edge detection algorithm based on Canny-Franklin moment: firstly improve the Canny edge rough positioning, and then extract the sub-pixel edge based on Franklin moment; S4, edge fitting and size measurement: After using the improved Canny-Franklin moment method to accurately locate the edge of the component at the sub-pixel level, a least squares straight line fitting algorithm is used to fit the edge; the pixel size, offset angle and centroid of the component edge are calculated.

2. The high-precision measurement method for chip mounting geometric parameters according to claim 1, characterized in that: In step S2, the method for performing image preprocessing on the distortion-corrected patch element image comprises the following steps: S21, image grayscale conversion; S22, hybrid filtering denoising, firstly uses median filtering on the image to remove most of the high-frequency noise, and then uses bilateral filtering to further process the initially smoothed image; S23, image segmentation, uses the OTSU method to perform threshold segmentation on the patch image.

3. A high-precision measurement method for chip mounting geometric parameters according to claim 1 or 2, characterized in that: In step S21, the color image is converted into a grayscale image using a weighted average method, and the formula is as follows: Among them, (i,j) is the position coordinate of a pixel in the image; Gray(i,j) is the grayscale value of the image, G(i,j), B(i,j) and R(i,j) are the values ​​of the green, blue and red channels of the color image respectively.

4. A high-precision measurement method for chip mounting geometric parameters according to claim 3, characterized in that: The redundant interference area after image segmentation is optimized. The optimization process is as follows: S231, using 7×7 circular structure elements, based on morphological opening operation to eliminate isolated noise points and small interference areas on the binary image; S232, filling the hole area in the image with the same pixel value as the surrounding adjacent area, so as to reduce the missing part in the image and make the target object more complete; S233, by setting an appropriate area threshold, delete the connected domains whose area is smaller than the threshold P.

5. A high-precision measurement method for chip mounting geometric parameters according to claim 4, characterized in that: The process of improving Canny edge coarse positioning is as follows: 1) Use a hybrid filtering denoising method to smooth the image while retaining edge details and improving image quality; 2) Select a 3×3 gradient detection template for detection. The gradient detection template is shown as follows: Among them, H x is the horizontal gradient convolution kernel, which is used to detect the edge changes of the image in the horizontal direction. y is the vertical gradient convolution kernel, which is used to detect the edge changes of the image in the vertical direction. 45° is the gradient convolution kernel along the 45° direction, which is used to detect the edge of the image in the 45° direction. 135° is the gradient convolution kernel along the 135° direction, which is used to detect the edge of the image in the 135° direction; 3) Non-maximum suppression: After calculating the image gradient amplitude and direction, each pixel in the image is traversed one by one to eliminate non-edge pixels. When the gradient value of the pixel is the largest in the adjacent area in the gradient direction, it is retained as an edge point; otherwise, the pixel will be ignored. 4) Adaptive threshold selection: Adaptive high and low threshold selection is realized during edge connection through the OTSU algorithm.

6. A high-precision measurement method for chip mounting geometric parameters according to claim 5, characterized in that: The Franklin moment is created as follows: First, the linearly independent function set {a n (x) = 1, 0 ≤ x ≤ 1} Where: a0 is a constant term, a1 is a linear term, a i is a function term generated by recursion, i ≥ 2; a i =(2i-1-2 p ) / 2 p , p is a positive integer, p is all the integers that satisfy 2 p ≤ the maximum value of 2i-1; The above linearly independent function group {a n (x) = 1, 0 ≤ x ≤ 1}, and then perform Gram-Schmidt orthogonalization to obtain the Franklin function system The expressions of the first three basis functions are: Franklin moments are defined based on Franklin functions. The nth Franklin function is 0≤x≤1, n=0,1,2,… Given an image function f(x,y), 0≤x,y≤1, its (n+m)-order Franklin moment is defined as follows in, is the Franklin basis function in the y direction, used for orthogonal decomposition in the y-axis direction.

7. A high-precision measurement method for chip mounting geometric parameters according to claim 6, characterized in that: The method of sub-pixel edge extraction based on Franklin moment is as follows: First, calculate the difference f of the edge pixels in the row and column directions r , f c : Where: f(u,v) represents the pixel value of a certain point in the image, u is the vertical coordinate of the image, v is the horizontal coordinate of the image, i, j are the index values ​​of the row and column where the current pixel is located; When f r ≥f c When , the Franklin moments of different orders are calculated as follows: F 00 =h+k(1-l) (10) when hour, when hour, Combining the above formulas, we get in, l is the theoretical distance from the origin to the edge; k is the grayscale difference; h is the background grayscale; F 00 It is a special case of Franklin moment, representing the zero-order zero-order moment; F 01 and F 02 are the first-order and second-order moments of Franklin moments; l1 and l2 are two theoretical distances calculated based on Franklin moments, representing the theoretical distance from a reference point in the image to the edge; The final sub-pixel coordinate formula is: Where (x', y') is the sub-pixel coordinate of the edge of the image, and (x, y) is the coordinate of the origin; When f r <f c When , the Franklin moments of different orders are calculated as follows: F 00 =h+k(1-l) (16) when hour, when hour, Combining the above formulas, we get in, The final sub-pixel coordinate formula is: Where (x', y') is the sub-pixel coordinate of the edge of the image, and (x, y) is the coordinate of the origin.

8. A high-precision measurement method for chip mounting geometric parameters according to claim 7, characterized in that: In step S4, the edge data is segmented before performing a straight line fitting on the edge using a least squares straight line fitting algorithm, as follows: First, find the four boundary extreme values ​​X from the sub-pixel edge contour point set S max , Y max , X min , Y min , and use this to find the coordinates of the contour midpoint (X m ,Y m ); then find all X=X in the point set S min Then find the point with the smallest Y (X min ,Y0); if |Y0-Y min |≤T (22) If the component has no offset angle, it is considered that the component has an offset angle. When the element has no offset angle, the sub-pixel precision edge contour point set is initially divided into four subsets according to formula (23); Then, each contour subset is traversed in the x or y direction, and the mode is calculated as K1, K2, K3, and K4; then, the deviation threshold is set as d, and the subset is further filtered according to formula (24), thereby completing the data division process; Among them, S t3 is the sub-pixel contour point set of the upper edge, S t1 is the sub-pixel contour point set of the lower edge, S t2 is the sub-pixel contour point set of the right edge, S t4 is the sub-pixel contour point set of the left edge; When the component has an offset angle, traverse the point set S to obtain the coordinates of the four boundary extreme points (X min ,Y1),(X1,Y min ), (X max ,Y2),(X2,Y max ); When the component has an offset angle, the coordinates of the boundary extreme points are not equal and unique. Based on this feature, the edge is segmented, and the four edge point sets to be fitted are segmented according to formula (25); 9. A high-precision measurement method for chip mounting geometric parameters according to claim 8, characterized in that: The method for calculating the pixel size, offset angle and centroid of the component edge is as follows: Assume that the four fitted straight lines are line1, line2, line3, and line4, and specify S t3 , S r3 The edge of the point set fitting is the upper edge line1, S t1 , S r4 The edge of the point set fitting is the lower edge line3; the calculation formula of the intersection point is based on the parametric equation of the two straight lines; the coordinates of the intersection point between the specific straight lines are as follows Then calculate the average distance between the two straight lines; the calculation of the average distance is based on sampling and averaging the vertical distance between the two straight lines within a given range; the average distance D1 between line1 and line3, and the average distance D2 between line2 and line4 are shown as follows: Where: d i (line1, line3) is the vertical distance between the i-th sampling point in line1 and line3, N is the total number of sampling points; d j (line2, line4) is the vertical distance between the jth sampling point in line2 and line4, and M is the total number of sampling points; The angle is calculated based on the slope α of the straight line and converted into an angle system. When the condition of formula (22) is not satisfied, the component has an offset angle. The angle between the upper edge line line1 and the lower edge line line3 is as follows: Where α1 and α2 are the slopes of line1 and line3 respectively; let the offset angle of the element be the average angle i mean =(θ up +θ bottom ) / 2 (29) For a standard square device, the centroid is calculated by averaging the four intersection points, as shown below: Among them, (x c ,y c ) is the centroid, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the coordinates of the four intersection points respectively; Then compensate the centroid coordinates and calculate the coordinates of the four intersection points after compensation, as shown in the following formula: The final optimized centroid coordinates are:

Images