A positioning method and device based on Gaussian filtering using chip key points

Abstract

The present invention discloses a positioning method and device based on Gaussian filtering and using chip key points, in order to solve the problems of large positioning errors, weak robustness, low efficiency, etc. caused by deformation, incompleteness, and density of chip pins in existing placement machine positioning algorithms. This method makes an ideal template through key points, uses a Gaussian algorithm to filter the image acquired online and then extracts the corner points, then performs sub-pixel processing, and finally uses the ideal template and the image sub-pixel corner points to establish a function, uses the minimum distance method to converge the function, and obtains the rotation angle and offset. This method has high robustness for positioning chips of different packaging types that are deformed, incomplete, and dense. At the same time, this method selects key vertices for positioning, which has a small amount of calculation and high efficiency compared to the traditional full-image point set registration.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a positioning method and device using chip key points based on Gaussian filtering. Background Art

[0002] With the development of science and technology industry and chip technology, chip mounting technology has received great attention. Although computer vision positioning has been developed for a long time, most of the current algorithms for solving the problem of chip component positioning are centered around binary processing. As chip designs become more and more complex and pins become denser, the noise introduced by the deformation and incompleteness of chip pins reduces the accuracy and efficiency of registration.

[0003] There are many noise points in the corner extraction of deformed, incomplete and dense pins, which greatly affects the current chip positioning accuracy and efficiency. The present invention proposes to make an ideal template through the key vertices of the chip, and use the image processed by Gaussian filtering to align with the ideal template to accurately locate the pins. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a positioning method and device using chip key points based on Gaussian filtering.

[0005] To achieve the above technical objectives, the technical solution of the present invention is: a positioning method using chip key points based on Gaussian filtering, comprising the following steps:

[0006] (1) Obtain the design parameters of the chip from the chip design manual to obtain the location information of the key points of the chip;

[0007] (2) Establishing a template coordinate system with the chip element as the center as the origin of the template coordinate system;

[0008] (3) according to the position information of the key points of the chip obtained in step (1), calibrating the positions of the key points of the chip in the template coordinate system established in step (2);

[0009] (4) Collecting an image of the component to be positioned, establishing a component image coordinate system with the center of the component to be positioned as the origin; performing Gaussian filtering on the image of the component to be positioned, and performing corner point detection to obtain the corner point position;

[0010] (5) Using the corner point positions obtained in step (4), extract sub-pixel points to obtain the position of each sub-pixel point;

[0011] (6) The sub-pixel position obtained in step (5) is aligned with the chip key points calibrated in step (3), the rotation angle and translation amount of the chip are calculated, and the positioning of the chip key points is completed.

[0012] Furthermore, the step (1) is specifically as follows: the design parameters include the size of the chip and the location of the pins; the corner points of the outermost pins on each side of the chip are selected as key points, and according to the chip design manual, the distances between the key points and the central symmetry axes of the chip in the vertical and horizontal directions are obtained, the position information of all key points is obtained, and a key point set K is established.

[0013] Furthermore, first input the key vertex information of the component pin obtained in step (1); according to the template coordinate system established in step (2), select the origin of the coordinate system as the center of the component, and set the distance l between a key point k in the key point set K and the central symmetry axis in the vertical direction of the chip kx As the horizontal coordinate, the distance l between the key point k and the horizontal center symmetry axis of the chip ky As the ordinate, the key k coordinate is marked as k(l kx ,l ky ), k∈{Ⅰ,Ⅱ,Ⅲ…}, draw the key points in the template coordinate system and complete the calibration of the key vertex positions of the component.

[0014] Furthermore, the step (4) specifically includes the following sub-steps:

[0015] (4.1) Collect the image of the component to be located, and establish the component image coordinate system with the center of the component to be located as the origin;

[0016] (4.2) Scale the component image selected in step (4.1) by s times and then perform Gaussian filtering;

[0017] (4.3) extracting possible corner points of the image after Gaussian filtering in step (4.2) according to the window function;

[0018] (4.4) Filter the possible corner points found in step (4.3) to obtain the corner point positions.

[0019] Furthermore, the step (4.3) is specifically as follows: grayscale processing is first performed on the Gaussian filtered image, and a window function is drawn, and the formula is as follows:

[0020]

[0021] Where w(x,y) is a binary Gaussian function, u means that the window moves horizontally by u pixels each time, and v means that the window moves vertically by v pixels each time. The grayscale processed image is scanned using the window function. When the window function scans to point p(x P ,y p )When E(u,v) is greater than the set threshold, then the point p(x p ,y p ) are possible corner points.

[0022] Furthermore, the screening process in step (4.4) is specifically as follows: creating a possible corner point set M, wherein the point set M contains valid information of all possible corner points p, including the corner point coordinates p (x P ,y p ) and window function E(u,v);

[0023] Then calculate the distance between any two possible corner points, the formula is as follows:

[0024]

[0025] in, is the horizontal coordinate of the first corner point, is the horizontal coordinate of the second corner point, is the ordinate of the first corner point, is the ordinate of the second corner point.

[0026] If there is such a distance l P ≤2 times the pin width, then compare the E(u,v) of the two possible corner points, filter out the possible corner points with smaller E(u,v), and record the point set of possible corner points after filtering out as the accurate point set H, and get the positions of all corner points.

[0027] Furthermore, the step (5) is specifically as follows: sub-pixel processing is performed on the corner points in the accurate point set H obtained in step (4): any point p0 in the accurate point set H is selected to create an n×n window with p0 as the center, and each point in the window is a surrounding point of p0, denoted as p i ,p i The gray gradient of the point is denoted by G i [d x ,d y ]; G i Transpose Note the point p i The sub-pixel corner point is q, the inner product of the direction vector of the sub-pixel corner point with respect to the pixel corner point and the grayscale vector of the pixel corner point is 0, which is After introducing Gaussian weight correction The coordinates of the sub-pixel corner points q can be obtained by using the least squares method. All sub-pixel corner points constitute the point set Q.

[0028] Furthermore, the step (6) specifically includes the following sub-steps:

[0029] (6.1) Obtain the center of the key point set K established in step (3) The center of the point set Q formed by the sub-pixel corner points in step (5) The translation matrix is ​​calculated as T, and the formula is as follows:

[0030]

[0031] The translation matrix is

[0032] In the formula, is the center point of the key point set K, is any point in the key point set K, and n is the number of points in the key point set K;

[0033] In the formula, is the center point of the sub-pixel corner point set Q, is any point in the sub-pixel corner point set Q, and m is the number of points in the sub-pixel corner point set Q;

[0034] (6.2) The key point set K and the point set Q consisting of sub-pixel corner points are translated to the center point respectively: Objective Function Can be converted to Euclidean distance

[0035] (6.3) The sub-pixel point set Q undergoes a rotation matrix After transformation Calculate this time Solve for the rotation angle θ.

[0036] The present invention proposes a positioning device based on Gaussian filtering and utilizing chip key points, comprising one or more processors for implementing the above-mentioned positioning method based on Gaussian filtering and utilizing chip key points.

[0037] The present invention proposes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it is used to implement the above-mentioned positioning method based on Gaussian filtering and using chip key points.

[0038] The beneficial effects of the present invention are as follows: the method of the present invention has high robustness in positioning chips of different packaging types that are deformed, incomplete, and dense. In particular, for chips with a large number of incomplete pins, when making templates using the template matching method currently commonly used in the market, a large number of incomplete pins will greatly affect the success rate and accuracy of template creation. With this method, the incomplete pins have little effect on the production of templates, and the template matching method can be effectively used to achieve chip pose alignment. At the same time, this method selects key vertices for positioning, which has a small amount of calculation and high efficiency compared to the traditional full-image point set alignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the method of the present invention;

[0040] Figure 2A schematic diagram showing the selection of the corner points of the outermost pins on each edge of the chip as key points;

[0041] Figure 3 This is the schematic diagram of sub-pixel corner points;

[0042] Figure 4 It is a sub-pixel corner extraction map;

[0043] Figure 5 It is a schematic diagram of the structure of the device of the present invention. DETAILED DESCRIPTION

[0044] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0045] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0046] The following is a detailed description of the positioning method and device based on Gaussian filtering and using chip key points of the present invention in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0047] The present invention aims to solve the problems of large positioning error, weak robustness and low efficiency caused by deformation, incompleteness and density of chip pins in the existing placement machine positioning algorithm. A positioning method based on Gaussian filtering and using chip key points is proposed. The flow chart of the method of the present invention is as follows: Figure 1 As shown, the specific steps include:

[0048] (1): Obtain the design parameters of the chip through the chip design manual, where the design parameters include the size of the chip and the location of the pins.

[0049] The method for obtaining the position information of the key points of the chip in the embodiment of the present invention is specifically as follows: Figure 2 As shown in the figure, the corner point of the outermost pin of each edge of the chip is selected as the key point. According to the chip design manual, the distance between the key point and the central symmetry axis of the chip is obtained. The distance l between the key point I and the central symmetry axis of the chip in the vertical direction is recorded. Ix, the distance between key point I and the horizontal center symmetry axis of the chip l Iy , the distance l between key point II and the vertical center symmetry axis of the chip Ⅱx , the distance between key point II and the horizontal center symmetry axis of the chip is l Ⅱy , and so on, to obtain the location information of all key points.

[0050] (2): With the chip element as the center and the origin of the template coordinate system, the template coordinate system is established as a two-dimensional Cartesian coordinate system; wherein the X-axis is horizontal to the right and the Y-axis is vertically upward.

[0051] (3): Based on the position information of the key points of the chip obtained in step (1), the positions of the key points of the chip are calibrated in the template coordinate system established in step (2). Specifically:

[0052] First, input the key vertex information of the component pin obtained in step (1), wherein the key vertex information includes the two outermost corner points found in the pins on the same side of the component as key vertices; according to the template coordinate system established in step (2), select the origin of the coordinate system as the center of the component and establish a key point set K. Set the distance l between a key point k in the key point set K and the vertical central symmetry axis of the chip kx As the horizontal coordinate, the distance l between the key point k and the horizontal center symmetry axis of the chip ky As the ordinate, the key k coordinate is marked as k(l kx ,l ky ), k∈{Ⅰ,Ⅱ,Ⅲ…}, draw the key points in the template coordinate system and complete the calibration of the key vertex positions of the component.

[0053] (4): Obtain an image of the component to be located online, establish a component image coordinate system with the center of the component to be located as the origin; perform Gaussian filtering on the image to be located, and perform corner point detection to obtain the corner point position; specifically, the following sub-steps are included:

[0054] (4.1) Collect an image of the component to be positioned, and establish a component image coordinate system with the center of the component to be positioned as the origin; the origin of the component image coordinate system is the center of the component, the X-axis is horizontal to the right, and the Y-axis is vertically upward.

[0055] (4.2) performing Gaussian filtering on the component image selected in step (4.1), specifically: scaling the original image of the selected component to be located by s times, setting the Gaussian filter template size to 5×5, setting the filter kernel height to 1, and setting the filter kernel width to 1 in the embodiment of the present invention, and performing Gaussian filtering on the scaled component image;

[0056] (4.3), extract possible corner points of the image after Gaussian filtering, specifically: grayscale the image after Gaussian filtering in step (4.2), draw the window function, the formula is as follows:

[0057]

[0058] Where w(x,y) is a binary Gaussian function, u means that the window moves horizontally by u pixels each time, and v means that the window moves vertically by v pixels each time. Use this window function to scan the grayscale processed image. When the window function scans to point p(x P ,y p )When E(u,v) is greater than the set threshold, then the point p(x P ,y p ) are possible corner points.

[0059] (4.4), the possible corner points found in step (4.3) are screened. The screening method is as follows: a possible corner point set M is created. The point set M contains valid information of all possible corner points p, including the corner point coordinates p (x P ,y p ) and the window function E(u,v).

[0060] Then calculate the distance between any two possible corner points, the formula is as follows:

[0061]

[0062] in, is the horizontal coordinate of the first corner point, is the horizontal coordinate of the second corner point, is the ordinate of the first corner point, is the ordinate of the second corner point.

[0063] If there is such a distance l P ≤2 times the pin width, then compare the E(u,v) of the two possible corner points, filter out the possible corner points with smaller E(u,v), and record the point set of possible corner points obtained after filtering as the accurate point set H, and get the positions of all corner points.

[0064] (5): Using the corner point positions obtained by screening in step (4), sub-pixel points are extracted to obtain the position of each sub-pixel point.

[0065] Specifically:

[0066] Sub-pixel processing is performed on the corner points in the accurate point set H obtained in step (4): select any point p0 in the accurate point set H and create an n×n window with p0 as the center. Each point in the window is a point around p0 and is denoted as p i ,p iThe gray gradient of the point is denoted by G i [d x ,d y ]; G i Transpose like Figure 3 As shown, in the embodiment of the present invention, a 5×5 window is created with p0 as the center point, p1 and p2 are two surrounding points of p0 in the window, and G1 is the grayscale gradient at point p1. i The sub-pixel corner point is q, the inner product of the direction vector of the sub-pixel corner point with respect to the pixel corner point and the grayscale vector of the pixel corner point = 0,

[0067] in, Since the surrounding points have different influences on p0, the influence weights satisfy the Gaussian distribution. After the Gaussian weight correction is introduced The coordinates of the sub-pixel corner point q can be obtained by using the least squares method. All sub-pixel corner points constitute the point set Q. The sub-pixel corner point extraction result of the grayscale image is as follows: Figure 4 as shown.

[0068] (6): Use the sub-pixel corner point positions obtained in step (5) and the chip key points calibrated in step (3) to align by solving the point set K and point set Q in step 3, that is, find the rotation matrix R and the translation matrix T so that each point in the key point set K and each point in the sub-pixel point set M = RQ + T after the transformation of the rotation matrix R and the translation matrix T can be calculated. When the sum of the minimum distances calculated for each point in the key point set is the minimum, that is, when When the minimum, the point set K is aligned with the point set Q, and the rotation matrix R and translation matrix T at this time are solved to obtain the rotation angle and translation amount of the chip. Specifically, it includes the following sub-steps:

[0069] (6.1) Obtain the center of the key point set K established in step (3) The center of the point set Q formed by the sub-pixel corner points in step (5) The translation matrix is ​​calculated as T, and the formula is as follows:

[0070]

[0071] The translation matrix is

[0072] In the formula, is the center point of the key point set K, is any point in the key point set K, and n is the number of points in the key point set K.

[0073] In the formula, is the center point of the sub-pixel corner point set Q, is any point in the sub-pixel corner point set Q, and m is the number of points in the sub-pixel corner point set Q.

[0074] (6.2) The key point set K and the point set Q consisting of sub-pixel corner points are translated to the center point respectively: Objective Function Can be converted into

[0075] (6.3) The sub-pixel point set Q undergoes a rotation matrix After transformation Calculate this time Solve for the rotation angle θ.

[0076] Corresponding to the aforementioned embodiment of the positioning method based on Gaussian filtering and utilizing the key points of the chip, the present invention also provides an embodiment of the positioning device based on Gaussian filtering and utilizing the key points of the chip.

[0077] See also Figure 5 An embodiment of the present invention provides a positioning device based on Gaussian filtering and utilizing chip key points, comprising one or more processors for implementing the positioning method based on Gaussian filtering and utilizing chip key points in the above embodiment.

[0078] The embodiments of the positioning device for chip key points based on Gaussian filtering of the present invention can be applied to any device with data processing capabilities, which can be devices or apparatuses such as computers. The device embodiments can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From a hardware perspective, if Figure 5 As shown, it is a hardware structure diagram of any device with data processing capability where the positioning device of the present invention based on Gaussian filtering and using the key points of the chip is located. Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0079] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0080] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0081] An embodiment of the present invention further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the positioning method based on Gaussian filtering and using chip key points in the above embodiment is implemented.

[0082] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A positioning method based on Gaussian filtering using chip key points, characterized in that: The following steps are involved: (1) Obtain the design parameters of the chip from the chip design manual and obtain the location information of the key points of the chip; (2) Establishing a template coordinate system with the center of the chip element as the origin of the template coordinate system; (3) according to the position information of the key points of the chip obtained in step (1), calibrating the positions of the key points of the chip in the template coordinate system established in step (2); (4) Collecting an image of the component to be located, taking the center of the component to be located as the origin, and establishing a component image coordinate system; Gaussian filtering is performed on the image of the component to be located, and corner point detection is performed to screen possible corner points, and the point set of possible corner points after screening is recorded as the accurate point set H, and the corner point position is obtained; (5) Using the corner point position obtained in step (4), extract sub-pixel points to obtain the position of each sub-pixel point; specifically, the following steps are performed: for the corner points in the accurate point set H obtained in step (4), sub-pixel processing is performed: any point p0 in the accurate point set H is selected to create an n×n window with p0 as the center, and each point in the window is a surrounding point of p0, denoted as p i ,p i The gray gradient of the point is denoted by G i [d x ,d y ]; G i Transpose Note the point p i The sub-pixel corner point is q, The inner product of the direction vector of the sub-pixel corner point about the pixel corner point and the grayscale vector of the pixel corner point is equal to 0. After the Gaussian weight correction is introduced The coordinates of the sub-pixel corner point q can be obtained by using the least squares method, and all sub-pixel corner points constitute the point set Q; (6) The sub-pixel position obtained in step (5) is aligned with the chip key points calibrated in step (3), the rotation angle and translation amount of the chip are calculated, and the positioning of the chip key points is completed.

2. The positioning method based on Gaussian filtering and using chip key points according to claim 1 is characterized in that: The step (1) is specifically as follows: the design parameters include the size of the chip and the location of the pins; the corner points of the outermost pins on each side of the chip are selected as key points, and according to the chip design manual, the distances between the key points and the central symmetry axes of the chip in the vertical and horizontal directions are obtained, the position information of all key points is obtained, and a key point set K is established.

3. The positioning method based on Gaussian filtering and utilizing chip key points according to claim 2 is characterized in that: First, input the key vertex information of the component pin obtained in step (1); according to the template coordinate system established in step (2), select the origin of the coordinate system as the center of the component, and calculate the distance l between a key point k in the key point set K and the vertical center symmetry axis of the chip. kx As the horizontal coordinate, the distance l between the key point k and the horizontal center symmetry axis of the chip ky As the ordinate, the key point k is marked as k(l kx ,l ky ), k∈{Ⅰ,Ⅱ,Ⅲ…}, draw the key points in the template coordinate system and complete the calibration of the key vertex positions of the component.

4. The positioning method based on Gaussian filtering and utilizing chip key points according to claim 1, characterized in that: The step (4) specifically includes the following sub-steps: (4.1) Collect the image of the component to be located, and establish the component image coordinate system with the center of the component to be located as the origin; (4.2) Scale the component image selected in step (4.1) by s times and then perform Gaussian filtering; (4.3) extracting possible corner points of the image after Gaussian filtering in step (4.2) according to the window function; (4.4) Filter the possible corner points found in step (4.3) to obtain the corner point positions.

5. The positioning method based on Gaussian filtering and utilizing chip key points according to claim 4 is characterized in that: The step (4.3) is specifically as follows: grayscale processing is first performed on the Gaussian filtered image, and a window function is drawn. The formula is as follows: Where w(x,y) is a binary Gaussian function, u means that the window moves horizontally by u pixels each time, and v means that the window moves vertically by v pixels each time. The grayscale processed image is scanned using the window function. When the window function scans to point p(x P ,y p )When E(u,v) is greater than the set threshold, then the point p(x P ,y p ) are possible corner points.

6. The positioning method based on Gaussian filtering and using chip key points according to claim 4 is characterized in that: The screening process in step (4.4) is specifically as follows: creating a possible corner point set M, wherein the point set M contains valid information of all possible corner points p, including the corner point coordinates p(x P ,y p ) and window function E(u,v); Then calculate the distance between any two possible corner points, the formula is as follows: in, is the horizontal coordinate of the first corner point, is the horizontal coordinate of the second corner point, is the ordinate of the first corner point, is the ordinate of the second corner point; If there is such a distance l P ≤2 times the pin width, then compare the E(u,v) of the two possible corner points, filter out the possible corner points with smaller E(u,v), and record the point set of possible corner points after filtering out as the accurate point set H, and get the positions of all corner points.

7. The positioning method based on Gaussian filtering and utilizing chip key points according to claim 2, characterized in that: The step (6) specifically includes the following sub-steps: (6.1) Obtain the center of the key point set K established in step (3) The center of the point set Q formed by the sub-pixel corner points in step (5) The translation matrix is ​​calculated as T, and the formula is as follows: The translation matrix is In the formula, is the center point of the key point set K, is any point in the key point set K, and n is the number of points in the key point set K; In the formula, is the center point of the sub-pixel corner point set Q, is any point in the sub-pixel corner point set Q, and m is the number of points in the sub-pixel corner point set Q; (6.2) The key point set K and the point set Q consisting of sub-pixel corner points are translated to the center point respectively: Objective Function Convert to Euclidean distance (6.3) The sub-pixel point set Q undergoes a rotation matrix After transformation Calculate this time Solve for the rotation angle θ.

8. A positioning device based on Gaussian filtering using chip key points, characterized in that: It comprises one or more processors, used to implement the positioning method based on Gaussian filtering and utilizing chip key points as described in any one of claims 1-7.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, it is used to implement the positioning method based on Gaussian filtering and using chip key points as described in any one of claims 1 to 7.

Images