A visual identification method for BGA

Abstract

The embodiment of the present invention provides a visual recognition method for BGA, comprising the following steps: S1: image pre-processing, performing binarization, opening operation and closing operation on the image; S2: image contour extraction, least squares fitting circle, extracting all balls; S3: filtering interference points; S4: filtering MARK points; S5: rotating all balls; S6: generating result matrix; S7: comparison; S8: iterative linear difference; S9: calculating MARK position; S10: calculating component position and angle, so as to effectively, quickly and accurately locate BGA, which is applied in the production process of placement machine to avoid poor production quality, thereby making the machine more humane and intelligent. The embodiment of the present invention facilitates effective, fast and accurate positioning of BGA, which is applied in the production process of placement machine to avoid poor production quality, thereby making the machine more humane and intelligent.

Description

Technical Field

[0001] The present invention relates to the field of visual recognition technology, and more specifically, to a BGA visual recognition method. Background Art

[0002] In the modern SMT production process, the positioning of components is achieved through visual recognition. BGA components are very important and expensive components among patch components. In production, not only must their extremely high position accuracy and angle accuracy be guaranteed, but also their correct direction and correct ball state must be guaranteed. Due to the diversity of their ball states, some SMT equipment cannot calculate the ball state and can only locate by the outermost part of the ball. If the internal ball is missing, it cannot be detected, which will cause product defects and this defect cannot be discovered in time, causing great harm to production and may also cause great losses to customers.

[0003] SUMMARY OF THE INVENTION

[0004] In order to overcome the deficiencies of the prior art, the present invention provides a BGA visual recognition method for solving the problems of low positioning accuracy, inability to identify direction, and inability to determine whether a ball is defective by using component shape features.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a BGA visual recognition method, comprising the following steps:

[0006] S1: Image pre-processing, binarization, opening and closing operations on the image;

[0007] S2: Image contour extraction, least squares fitting circle, extracting all spheres;

[0008] S3: filter out interference points;

[0009] S4: filter out MARK points;

[0010] S5: rotate all balls;

[0011] S6: Generate result matrix;

[0012] S7: Alignment;

[0013] S8: iterative linear interpolation;

[0014] S9: Calculate the MARK position;

[0015] S10: Calculate component position and angle.

[0016] Specifically, the image pre-processing step includes:

[0017] The image was binarized using Otsu automatic thresholding;

[0018] Use open operation to filter out interference points;

[0019] The shape of the BGA ball imaging is closed using a closing operation.

[0020] Specifically, image contour extraction, least squares fitting circle, and extraction of all spheres, the steps include:

[0021] For contour fitting circles with actual areas greater than 25, the score is calculated based on the aspect ratio, and the score of each circle is:

[0022] Among them, box.size represents the size of the minimum circumscribed rectangle of the outline, height represents the height of the minimum circumscribed rectangle of the outline, and width represents the width of the minimum circumscribed rectangle of the outline;

[0023] When the score of the circle is greater than 0.7, the circle is considered to be the target ball.

[0024] Specifically, the step of filtering out interference points includes:

[0025] Calculate the minimum enclosing rectangle of all balls;

[0026] Filter out the 4 balls closest to the minimum circumscribed rectangle;

[0027] Repeat the iteration three times until the area of ​​the minimum enclosing rectangle is the same as the previous one or the difference is less than 5%, then terminate the process early.

[0028] Specifically, the step of filtering out the MARK points includes:

[0029] Calculate the minimum enclosing rectangle for all balls;

[0030] Calculate the ball closest to the four vertices as the candidate ball for filtering;

[0031] Exclude the balls closest to the four vertices and calculate the minimum enclosing rectangle again;

[0032] Determine whether the ball closest to the four vertices is within the minimum circumscribed rectangle;

[0033] When the balls closest to the four vertices are all within the minimum circumscribed rectangle, the balls closest to the vertices are filtered out.

[0034] Specifically, all the balls are rotated, and the steps include:

[0035] Calculate the angles of the four sides formed by the outermost sphere respectively;

[0036] After rotation, calculate the coordinates of the four corner points and sort them from top left to bottom right as corner[4];

[0037] Among them, corner[4] represents the coordinates of the four corner points.

[0038] Specifically, the result matrix is ​​generated, and the steps include:

[0039] The theoretical coordinates of the i-th row and j-th column are p i,j =(coner[0].x+pitch_x×j,coner[0].y-pitch_y×i)

[0040] Generate a 2D matrix b ij Represents the coordinates of the position ball at row i and column j.

[0041] Specifically, the comparison step includes:

[0042] A two-dimensional matrix to store balls a ij When it is 1, it means there is a ball at the position of the i-th row and j-th column. ij When it is 0, it means there is no ball in the i-th row and j-th column;

[0043] when (a ij =1∩b ij =(0,0))∪(a ij =0∩b ij ≠(0,0)), it indicates that the component is of poor quality and will not be mounted.

[0044] Specifically, iterative linear interpolation, the steps include:

[0045] Interpolate in the order (a)(b)(c)(d)(e).

[0046] (a)

[0047] (b)

[0048] (c)

[0049] (d) Vertical empathy

[0050] (e)p i,j =(2×p i,j-1 .xp i,j-2 .x,2×p i,j-1 .yp i,j-2 .y), the same for vertical direction

[0051] Among them, p i,j represents the coordinates of the interpolation position in the i-th row and j-th column, p i-1,j Indicates pi,j The coordinates of the points above the four spheres, p i+1,j Indicates p i,j The coordinates of the points below the four fields, p i,j-1 Indicates p i,j The coordinates of the left point of the four fields, p i,j+1 Indicates p i,j The coordinates of the right point of the four fields, p m,n represents the coordinates of the starting point, p m+k,n+k represents the coordinates of the end point, and k represents the number of interpolations.

[0052] Specifically, the MARK position is calculated, and the steps include:

[0053] Specify whether to have a MARK mark in the parameter settings;

[0054] Indicates the size of the MARK mark and the distance of the MARK mark from the nearest ball.

[0055] Specifically, the position and angle are calculated, and the steps include:

[0056]

[0057]

[0058] p i,j is the point in the i-th row and j-th column, deg1, deg2, deg3, and deg4 are the angles of the straight lines fitted to the four outermost edges.

[0059] The beneficial effects of the present invention are: S1: image pre-processing, performing binarization, opening operation and closing operation on the image; S2: image contour extraction, least squares fitting circle, extracting all balls; S3: filtering out interference points; S4: filtering out MARK points; S5: rotating all balls; S6: generating result matrix; S7: comparison; S8: iterative linear difference; S9: calculating MARK position; S10: calculating component position and angle, so as to effectively, quickly and accurately locate BGA, which is applied in the production process of placement machines to avoid poor production quality, thereby making the machine more humane and intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of a visual identification method of BGA.

[0061] Figure 2 It is a schematic diagram showing the effect of a visual identification method for BGA.

[0062] Figure 3 It is another schematic diagram of the effect of a visual identification method of BGA.

[0063] Figure 4It is another schematic diagram of the effect of a visual identification method of BGA.

[0064] Figure 5 It is another schematic diagram of the effect of a visual identification method of BGA.

[0065] Figure 6 It is another schematic diagram of the effect of a visual identification method of BGA.

[0066] Figure 7 It is another schematic diagram of the effect of a visual identification method of BGA.

[0067] Figure 8 It is another schematic diagram of the effect of a visual identification method of BGA.

[0068] Fig. 9 It is another schematic diagram of the effect of a visual identification method of BGA.

[0069] Fig.10 It is another schematic diagram of the effect of a visual identification method of BGA.

[0070] Fig.11 It is another schematic diagram of the effect of a visual identification method of BGA.

[0071] Fig.12 It is another schematic diagram of the effect of a visual identification method of BGA.

[0072] Fig.13 It is another schematic diagram of the effect of a visual identification method of BGA. DETAILED DESCRIPTION

[0073] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments:

[0074] Embodiment 1:

[0075] Figure 1 The implementation process of a BGA visual recognition method provided by the first embodiment of the present invention is shown. For the convenience of description, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0076] In step S101: image pre-processing, performing binarization, opening operation and closing operation on the image;

[0077] Specifically, the image pre-processing step includes:

[0078] The image was binarized using Otsu automatic thresholding;

[0079] Use open operation to filter out interference points;

[0080] The shape of the BGA ball imaging is closed using a closing operation.

[0081] In step S102: extract image contour, fit the circle by least squares, and extract all spheres;

[0082] Specifically, image contour extraction, least squares fitting circle, and extraction of all spheres, the steps include:

[0083] For contour fitting circles with actual areas greater than 25, the score is calculated based on the aspect ratio, and the score of each circle is:

[0084]

[0085] Among them, box.size is the size of the minimum bounding rectangle of the outline, height is the height of the minimum bounding rectangle of the outline, and width is the width of the minimum bounding rectangle of the outline;

[0086] When the score of the circle is greater than 0.7, the circle is considered to be the target ball.

[0087] In step S103: filtering out interference points;

[0088] Specifically, the step of filtering out interference points includes:

[0089] Calculate the minimum enclosing rectangle of all balls;

[0090] Filter out the 4 balls closest to the minimum circumscribed rectangle;

[0091] Repeat the iteration three times until the area of ​​the minimum enclosing rectangle is the same as the previous one (with a difference of 5%), and then terminate the process early.

[0092] In step S104: filter out MARK points;

[0093] Specifically, the step of filtering out the MARK points includes:

[0094] Calculate the minimum enclosing rectangle for all balls;

[0095] Calculate the ball closest to the four vertices as the candidate ball for filtering;

[0096] Exclude the balls closest to the four vertices and calculate the minimum enclosing rectangle again;

[0097] Determine whether the ball closest to the four vertices is within the minimum circumscribed rectangle;

[0098] When the balls closest to the four vertices are all within the minimum circumscribed rectangle, the balls closest to the vertices are filtered out.

[0099] In step S105: rotate all balls;

[0100] Specifically, all the balls are rotated, and the steps include:

[0101] Calculate the angles of the four sides formed by the outermost sphere respectively;

[0102] After rotation, calculate the coordinates of the four corner points and sort them from top left to bottom right as corner[4];

[0103] Among them, corner[4] represents the coordinates of the four corner points.

[0104] In step S106: generating a result matrix;

[0105] Specifically, the result matrix is ​​generated, and the steps include:

[0106] The theoretical coordinates of the i-th row and j-th column are p i,j =(coner[0].x+pitch_x×j,coner[0].y-pitch_y×i)

[0107] Generate a 2D matrix b ij Represents the coordinates of the position ball at row i and column j.

[0108] In step S107: comparison;

[0109] Specifically, the comparison step includes:

[0110] A two-dimensional matrix to store balls a ij When it is 1, it means there is a ball at the position of the i-th row and j-th column. ij When it is 0, it means there is no ball in the i-th row and j-th column.

[0111] when (a ij =1∩b ij =(0,0))∪(a ij =0∩b ij ≠(0,0)), it indicates that the component is of poor quality and will not be mounted.

[0112] In step S108: iterating the linear difference;

[0113] Specifically, iterative linear interpolation, the steps include:

[0114] Interpolate in the order (a)(b)(c)(d)(e).

[0115] (a)

[0116] (b)

[0117] (c)

[0118] (d) Vertical empathy

[0119] (e)p i,j =(2×p i,j-1 .xp i,j-2 .x,2×p i,j-1 .yp i,j-2 .y), the same for vertical direction

[0120] Among them, p i,j represents the coordinates of the interpolation position in the i-th row and j-th column, p i-1,j Indicates p i,j The coordinates of the points above the four spheres, p i+1,j Indicates p i,j The coordinates of the points below the four fields, p i,j-1 Indicates p i,j The coordinates of the left point of the four fields, p i,j+1 Indicates p i,j The coordinates of the right point of the four fields, p m,n represents the coordinates of the starting point, p m+k,n+k represents the coordinates of the end point, and k represents the number of interpolations.

[0121] In step S109: calculate the MARK position;

[0122] Specifically, the MARK position is calculated, and the steps include:

[0123] Specify whether to have a MARK mark in the parameter settings;

[0124] Indicates the size of the MARK mark and the distance of the MARK mark from the nearest ball.

[0125] In step S1010 : the component position and angle are calculated.

[0126] Specifically, the position and angle are calculated, and the steps include:

[0127]

[0128]

[0129] p i,j is the point in the i-th row and j-th column, deg1, deg2, deg3, and deg4 are the angles of the straight lines fitted to the four outermost edges.

[0130] A person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as ROM / RAM, disk, CD-ROM, etc.

[0131] Embodiment 2:

[0132] Figures 2 to 13 The effect schematic diagram of a BGA visual recognition method provided by the second embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0133] Step 1): Pre-treatment

[0134] like Figure 2 , 3 As shown, the image is first binarized using Otsu automatic thresholding. Because there may be some interference points in the image, an open operation is first used to filter them out, and then a closed operation is used to close the circle (the shape of the BGA ball imaging).

[0135] Step 2): Extract all balls

[0136] like Figure 4 As shown in the figure, image contour extraction, contour fitting circles with actual area greater than 25, and score calculation based on aspect ratio, the score of each circle is:

[0137]

[0138] Among them, box.size is the size of the minimum bounding rectangle of the outline, height is the height of the minimum bounding rectangle of the outline, and width is the width of the minimum bounding rectangle of the outline;

[0139] When the score is greater than 0.7, it is considered to be the target ball, and its position is the center of the fitting circle.

[0140] Step 3): Filter out interference points

[0141] like Figure 5 As shown, because there may be interference factors in the image and outside the component, they need to be filtered out. For all the balls extracted in the previous step, first calculate their minimum enclosing rectangle, and then filter out the four balls closest to the four sides of the rectangle. Then calculate the minimum enclosing rectangle formed by the remaining balls again, and filter out the four balls closest to the four sides of the rectangle again. This process is iterated 3 times. If the area of ​​the minimum enclosing rectangle this time is the same as the last time (the difference is less than 5%), the process ends early. The red solid circles are the filtered balls (you can see that some balls are "overkilled", which will be processed later).

[0142] Step 4): Filter out MARK points

[0143] like Figure 6As shown, the BGA component has only one MARK point, which is located at the corner of the component (may be at the lower left corner, upper left corner, upper right corner or lower right corner), and is generally in the shape of a triangle. The distance from the nearest ball is not fixed, some are closer, and some are farther. Calculate the minimum circumscribed rectangle for all balls, calculate the ball closest to the four vertices as the candidate balls for filtering, exclude these four balls and calculate the minimum circumscribed rectangle again, and calculate whether the four balls are within this rectangle. If they are all within, do not filter them out, otherwise filter out the ball closest to the vertex.

[0144] Step 5): Spin the ball

[0145] like Figure 7 , 8 As shown in the figure, the outermost sphere forms 4 edges. The angles of each edge are calculated and rounded to near 0 degrees. The average is calculated and then rotated. After the rotation, the coordinates of the 4 corner points are calculated and sorted from top left to bottom right as corner[4]. Corner[4] represents the coordinates of the 4 corner points, as shown in the figure. Figure 8 In the figure, red, green, blue and yellow circles are shown.

[0146] Step 6): Generate the result matrix

[0147] like Fig. 9 As shown, the theoretical coordinates of the i-th row and j-th column are

[0148] p i,j =(coner[0].x+pitch_x×j,coner[0].y-pitch_y×i)

[0149] Generate a two-dimensional matrix B, iterate all the balls, and fill the coordinates of each ball into the nearest matrix grid. If there is no ball near the grid, its coordinates are the default value (0,0). After all the balls are assigned, check again to see if there are repeated assignments. If there are, it means that there are extra balls inside the component, the component is of poor quality and will not be mounted.

[0150] b ij Represents the coordinates of the position ball at row i and column j.

[0151] Step 7): Compare

[0152] like Fig.10 As shown, in the parameter setting of BGA, there is a two-dimensional matrix for storing the state of the ball, as follows:

[0153] a ij When it is 1, it means there is a ball at the position of the i-th row and j-th column. ij When it is 0, it means there is no ball in the i-th row and j-th column.

[0154] If (aij =1∩b ij =(0,0))∪(a ij =0∩b ij ≠(0,0)), it means that the ball is missing where there should be a ball, or there are extra balls where there should not be a ball. The component is of poor quality and will not be mounted.

[0155] When all positions are normal, that is, there is no shortage of balls at the positions where the balls are located, or there are no extra balls at the positions where the balls are located, the next step can be carried out.

[0156] Step 8): Iterative linear interpolation

[0157] like Fig.11 As shown, since the position is calculated based on the positions of all the balls, if all the balls are not symmetrical about the origin, the calculation will be wrong. At this time, interpolation is required, and the interpolation priority decreases from left to right.

[0158]

[0159]

[0160]

[0161]

[0162] Vertical empathy

[0163] p i,j =(2×p i,j-1 .xp i,j-2 .x,2×p i,j-1 .yp i,j-2 .y) The same goes for vertical direction

[0164] Among them, p i,j represents the coordinates of the interpolation position in the i-th row and j-th column, p i-1,j Indicates p i,j The coordinates of the points above the four spheres, p i+1,j Indicates p i,j The coordinates of the points below the four fields, p i,j-1 Indicates p i,j The coordinates of the left point of the four fields, p i,j+1 Indicates p i,j The coordinates of the right point of the four fields, p m,n represents the coordinates of the starting point, p m+k,n+k represents the coordinates of the end point, and k represents the number of interpolations.

[0165] Step 8): Calculate the MARK position

[0166] like Fig.12 As shown, the parameter setting will clearly specify whether there is a MARK mark. If there is a MARK mark, the size of the MARK mark and the distance from the nearest ball will also be specified. Search for the MARK mark at the specified position according to the set parameters. If the MARK mark is found at the specified position, it means that the direction is correct and you can proceed to the next step.

[0167] Step 8): Calculate position and angle

[0168] like Fig.13 As shown, p i,j is the point at the i-th row and j-th column, deg1, deg2, deg3, and deg4 are the angles of the straight lines fitted to the four outermost edges (rounded to around 0 degrees), and the position and angle are calculated as follows.

[0169]

[0170]

[0171] Those skilled in the art will appreciate that the units and algorithm steps of the embodiments described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0172] Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. The above is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for visually identifying a BGA, characterized in that: The following steps are involved: S1: Image pre-processing, performing binarization, opening and closing operations on the image; S2: Image contour extraction, least squares fitting circle, extracting all spheres; Image contour extraction, least squares fitting circle, extracting all spheres, the steps include: For contour fitting circles with actual areas greater than 25, the score is calculated based on the aspect ratio, and the score of each circle is: ； in, Expressed as the size of the minimum circumscribed rectangle of the outline, It is expressed as the height of the minimum circumscribed rectangle of the outline. It is expressed as the width of the minimum circumscribed rectangle of the outline; when the score of the circle is greater than 0.7, it is considered that the circle is the target ball; S3: filtering out interference points; the step of filtering out interference points includes: Calculate the minimum enclosing rectangle of all balls; Filter out the 4 balls closest to the minimum circumscribed rectangle; Repeat the iteration three times until the area of ​​the minimum enclosing rectangle is the same as the previous one or the difference is less than 5%, then end the process early; S4: filtering out MARK points; the step of filtering out MARK points includes: Calculate the minimum enclosing rectangle for all balls; Calculate the ball closest to the four vertices as the candidate ball for filtering; Exclude the balls closest to the four vertices and calculate the minimum enclosing rectangle again; Determine whether the ball closest to the four vertices is within the minimum circumscribed rectangle; When the balls closest to the four vertices are all within the minimum circumscribed rectangle, the balls closest to the vertices are filtered out; S5: Rotate all balls; Rotate all balls, the steps include: Calculate the angles of the four sides formed by the outermost sphere respectively; After rotation, calculate the coordinates of the four corner points and sort them by top left, bottom right. ; in, Expressed as the coordinates of the 4 corner points; S6: Generate a result matrix; generating a result matrix, the steps include: No. OK The theoretical coordinates of the columns are Generate a 2D matrix , represents the coordinates of the position ball at the i-th row and j-th column; S7: comparison; the comparison step includes: A two-dimensional matrix to store balls , When it is 1, it means there is a ball at the position of row i and column j. When it is 0, it means there is no ball in the i-th row and j-th column; when , it indicates that the component is of poor quality and will not be mounted; S8: iterative linear interpolation: iterative linear interpolation, the steps comprising: Interpolate in the order of (a)(b)(c)(d)(e); （a） （b） （c） (d) , vertically similar (e) , vertically similar in, represents the coordinates of the interpolation position in the i-th row and j-th column, express The coordinates of the points above the four spheres, express The coordinates of the points below the four realms, express The coordinates of the left point of the four fields, express The coordinates of the right point of the four fields, represents the coordinates of the starting point, represents the coordinates of the end point, Indicates the number of interpolations; S9: Calculate the MARK position; Calculate the MARK position, the step includes: Specify whether to have a MARK mark in the parameter settings; Indicate the size of the MARK mark and the distance of the MARK mark from the nearest ball; S10: Calculating the position and angle of the component; Calculating the position and angle, the steps include: For the Line The points in the column, deg1, deg2, deg3, deg4 are the angles of the straight lines fitted to the four outermost sides.

2. A BGA visual recognition method according to claim 1, characterized in that: The image pre-processing step includes: The image was binarized using Otsu automatic thresholding; Use open operation to filter out interference points; The shape of the BGA ball imaging is closed using a closing operation.

Images