Chip visual positioning system and method based on image contour feature and grayscale matching

Abstract

The present invention relates to the field of visual detection technology, and in particular to a chip visual positioning system and method based on image contour feature and grayscale matching, wherein an image acquisition module is moved to a chip detection position to acquire an image of a chip to be tested, and an image of the chip to be tested is obtained; an image preprocessing module is used to preprocess the image of the chip to be tested based on mathematical morphological operations, so that the outline of the chip gate bonding area is clear; an image matching positioning module is used to extract the outline of the chip gate bonding area of ​​the preprocessed chip image to be tested, and the center point coordinates and rotation angle of the chip gate bonding area are calculated. Based on the center point coordinates and rotation angle, a local rectangular search area and a matching angle are designed as a search window, and an improved NCC visual positioning method is used to accurately locate the center point of the chip gate bonding area, and a visual positioning detection result is obtained. The present invention can improve the speed, accuracy and robustness of chip visual positioning.

Description

Technical Field

[0001] The present invention belongs to the field of visual detection technology, and in particular relates to a chip visual positioning system and method based on image contour feature and grayscale matching. Background Art

[0002] Power semiconductor devices are the core devices for electric energy conversion. The quality and reliability of power semiconductor devices are closely related to the packaging process. At present, in order to reduce the thermal and electrical impedance and bonding damage of power semiconductor devices to a greater extent and meet the increasing process requirements of high voltage, high current, high reliability, etc., the packaging technology of power semiconductor devices has begun to develop from the traditional full-lead bonding packaging or the hybrid bonding packaging of lead and copper clips to the 2D plane and 3D stacked full copper clip bonding packaging. Full copper clip bonding packaging has begun to become the most advanced bonding packaging process for the manufacture of power semiconductor devices today.

[0003] In the full copper clip bonding packaging process, the bonding part is the main failure part of the power semiconductor device because it is mainly subjected to the repeated opening and closing of the current. In order to ensure the stability and reliability of the power semiconductor device after bonding packaging, it is necessary to ensure the accuracy of the chip dispensing position and the copper clip mounting position on the basis of efficient packaging, that is, the bonding packaging equipment needs to quickly obtain high-precision position information of the chip gate bonding area at the chip dispensing station and the copper clip mounting station respectively. Therefore, designing a high-precision and fast visual positioning system for the tiny gate bonding area on the chip that is correctly bridged with the copper clip is a key link in realizing advanced bonding packaging.

[0004] There are two main chip visual positioning methods in the existing bonding packaging technology. One is a chip visual positioning method based on the Normalized Correlation Coefficient (NCC). By calculating the normalized correlation coefficient between the detection image and the template image obtained by the visual system, the search window position when the normalized correlation coefficient is the maximum is selected as the positioning position to achieve visual positioning of the chip gate bonding area. This method has high positioning accuracy, low sensitivity to image noise, and is less affected by illumination. However, since the matching strategy is ergodic, the large amount of calculation will lead to slow matching and positioning speed. At the same time, the flexibility of angle measurement is poor. Angle matching will further increase the amount of calculation and prolong the matching and positioning time. The other is a chip visual positioning method based on image feature matching, which judges the similarity between the image to be tested and the standard features of the template, analyzes the positional relationship between the image to be tested and the standard features of the template, and thus obtains position and angle information. This method is more flexible in feature selection. The shape parameters, edges, corner points, various moments, etc. of the image can all be used as matching features. In addition, since the significant features of the image are extracted and the effective information of the image is obtained, the amount of calculation is significantly reduced, and the positioning speed is effectively improved. However, precisely because only a small part of the grayscale information is used, this type of method is more sensitive to errors in feature extraction and feature matching, and erroneous features need to be eliminated. Feature extraction and positioning directly affect the accuracy of the matching algorithm, so the stability of this type of method is slightly poor. Summary of the invention

[0005] In view of this, the present invention aims to provide a chip vision positioning system and method based on image contour features and grayscale matching, so as to improve the speed, accuracy and robustness of chip vision positioning.

[0006] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0007] A chip visual positioning system based on image contour feature and grayscale matching includes an image acquisition module, a light source illumination module, an image preprocessing module, an image matching and positioning module, and a network communication module; wherein,

[0008] The light source illumination module is used to provide illumination for the image acquisition module;

[0009] The image acquisition module is used to take pictures of the chip to be tested and transmit the obtained image of the chip to be tested to the image preprocessing module;

[0010] The image preprocessing module is used to preprocess the image of the chip to be tested based on mathematical morphological operations, so that the outline of the chip gate bonding area is clear;

[0011] The image matching and positioning module is used to extract the contour of the chip gate bonding area from the preprocessed chip image to be tested, calculate the center point coordinates and rotation angle of the chip gate bonding area, design a local rectangular search area and matching angle based on the center point coordinates and rotation angle and use it as the search area, use the improved NCC visual positioning method to accurately locate the center point of the chip gate bonding area, obtain the visual positioning detection result, and transmit it to the network communication module;

[0012] The network communication module is used to transmit the visual positioning detection results to the bonding packaging equipment control terminal through the network port.

[0013] Furthermore, the image acquisition module includes a camera and a dual telecentric lens, the camera and the dual telecentric lens are connected and fixed using a C interface or a CS interface, the camera and the dual telecentric lens are integrally fixed on one side of the mounting head, and the camera captures images vertically downward.

[0014] Furthermore, the camera resolution is × , pixel size × The magnification of the bi-telecentric lens is , assuming the size of the target area of ​​the chip to be tested is × , then the parameters of the camera and the bi-telecentric lens satisfy the following formula:

[0015] .

[0016] Furthermore, the light source illumination module includes a point light source and a light source controller. The point light source is connected to the light source controller. The light source controller is connected to the external trigger interface of the camera. The point light source is fixed in the light source hole of the double telecentric lens by a top screw, and the illumination direction is downward.

[0017] Furthermore, the image preprocessing module includes an image grayscale processing unit, an image binarization processing unit, a mathematical morphology closing operation unit, and a mathematical morphology opening operation unit; wherein,

[0018] The image grayscale processing unit is used to perform grayscale processing on the image of the chip to be tested;

[0019] The image binarization processing unit is used to perform binarization processing on the grayscale image of the chip to be tested;

[0020] The mathematical morphology closing operation unit is used to perform a mathematical morphology closing operation on the binary image of the chip to be tested, so as to filter out black spots in the image of the chip to be tested;

[0021] The mathematical morphology opening operation unit is used to perform a mathematical morphology opening operation on the image of the chip to be tested with black spots filtered out, so as to remove noise and obtain the image of the chip to be tested with a clear outline of the gate bonding area.

[0022] Furthermore, the image matching and positioning module includes a contour extraction unit, a contour rough screening unit, a contour fine screening unit, a search area interception unit, a search area correction unit, a template image interception unit, a template image information calculation unit, an image local pixel and calculation unit, an image downsampling unit, an image matching unit and a coordinate transformation unit; wherein,

[0023] The contour extraction unit is used to extract all contours of the preprocessed image of the chip to be tested by using a digital binary image topology method based on boundary tracking;

[0024] The contour rough screening unit is used to perform rough screening on all extracted contours according to the set contour area threshold;

[0025] The contour fine screening unit is used to calculate the minimum circumscribed rectangle of the contour after the rough screening, and calculate the length, width, area, center point coordinates and rotation angle of the minimum circumscribed rectangle , taking the area of ​​the minimum circumscribed rectangle and the length and width difference of the minimum circumscribed rectangle as screening conditions, the rough-screened contour is finely screened to obtain the contour of the chip gate bonding area;

[0026] The search area interception unit is used to intercept a local rectangular search area covering the chip gate bonding area in the image of the chip to be tested, and to connect the center point of the minimum circumscribed rectangle with the rotation angle Set as the center point and rotation angle of the local rectangular search area;

[0027] The search area correction unit is used to rotate the intercepted local rectangular search area , to correct the direction of the local rectangular search area;

[0028] The template image capture unit is used to capture the template image containing the chip gate bonding area in the image of the chip to be tested, and calculate the center point coordinates of the chip gate bonding area in the template image. and width The smaller value of ;

[0029] The template image information calculation unit is used to calculate the template grayscale mean value according to the grayscale information of the template image. , the standard deviation of the biased sample of the chip image to be tested and the biased sample standard deviation of the template image ;

[0030] The image local pixel sum calculation unit is used to calculate the image local pixel sum of the search window by using the image integral map method through the following formula: :

[0031] ;

[0032] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the chip to be tested, The coordinate of the upper left corner of the search window in the integral image of the chip to be tested;

[0033] The image downsampling unit is used to downsample the template image and the local rectangular search area image using the image pyramid method. L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula:

[0034] ;

[0035] In the formula, is the minimum side length of the template image after multiple downsampling;

[0036] The image matching unit is used to match and locate the downsampled template image with the image of the chip to be tested layer by layer in the order from the top layer to the bottom layer of the image pyramid, with the chip gate bonding area as the bottom layer, and calculate the normalized correlation coefficient of each layer by the following formula: , each layer The position of the search window at the maximum value is taken as the optimal matching positioning position, and the positioning position coordinates of the chip gate bonding area are obtained. The normalized correlation coefficients of each layer The calculation formula is:

[0037] ;

[0038] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template image and the chip image to be tested;

[0039] The coordinate transformation unit is used to rotate the positioning coordinates of the chip gate bonding area , and transform to obtain the center point coordinates of the chip gate bonding area.

[0040] A chip visual positioning method implemented by the above-mentioned chip visual positioning system based on image contour features and grayscale matching comprises the following steps:

[0041] S1: Move the image acquisition module to the chip detection position;

[0042] S2: Control the light source illumination module to illuminate the chip under test in a stroboscopic mode, and control the image acquisition module to acquire an image of the chip under test to obtain an image of the chip under test;

[0043] S3: Preprocessing the image of the chip to be tested based on mathematical morphological operations through the image preprocessing module, so that the outline of the chip gate bonding area is clear;

[0044] S4: The preprocessed chip image to be tested is used to extract the outline of the chip gate bonding area through the image matching and positioning module, and the center point coordinates and rotation angle of the chip gate bonding area are calculated. Based on the center point coordinates and rotation angle, a local rectangular search area and matching angle are designed and used as the search area. The center point of the chip gate bonding area is accurately located using the improved NCC visual positioning method to obtain the visual positioning detection result, and the visual positioning detection result is transmitted to the bonding packaging equipment control terminal through the network communication module.

[0045] Furthermore, step S3 specifically includes the following steps:

[0046] S301: grayscale processing is performed on the image of the chip to be tested;

[0047] S302: Binarization processing is performed on the grayscale image of the chip to be tested;

[0048] S303: performing a mathematical morphological closing operation on the binary image of the chip to be tested to filter out black spots in the image of the chip to be tested;

[0049] S304: performing a mathematical morphological opening operation on the image of the chip to be tested with the black spots filtered out to remove noise, thereby obtaining an image of the chip to be tested with a clear outline of the gate bonding area.

[0050] Furthermore, step S4 specifically includes the following steps:

[0051] S401: extracting all contours of the preprocessed image of the chip to be tested by using a digital binary image topology method based on boundary tracking;

[0052] S402: performing a rough screening of all extracted contours according to a set contour area threshold;

[0053] S403: Calculate the minimum circumscribed rectangle of the rough-screened contour, and calculate the length, width, area, center point coordinates and rotation angle of the minimum circumscribed rectangle , taking the area of ​​the minimum circumscribed rectangle and the length and width difference of the minimum circumscribed rectangle as screening conditions, the rough-screened contour is finely screened to obtain the contour of the chip gate bonding area;

[0054] S404: intercepting a local rectangular search area covering the chip gate bonding area in the image of the chip to be tested, and aligning the center point of the minimum circumscribed rectangle with the rotation angle Set as the center point and rotation angle of the local rectangular search area;

[0055] S405: Rotate the intercepted local rectangular search area , to correct the direction of the local rectangular search area;

[0056] S406: intercepting a template image including the chip gate bonding area in the image of the chip to be tested, and calculating the coordinates of the center point of the chip gate bonding area in the template image, and converting the length of the template image and width The smaller value of ;

[0057] S407: Calculate the template grayscale mean value based on the grayscale information of the template image , the standard deviation of the biased sample of the chip image to be tested and the biased sample standard deviation of the template image ;

[0058] S408: Using the image integral graph method, the local pixel sum of the image in the search window is calculated by the following formula: :

[0059] ;

[0060] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the chip to be tested, The coordinate of the upper left corner of the search window in the integral image of the chip to be tested;

[0061] S409: Using the image pyramid method to perform image retrieval on the template image and the local rectangular search area image L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula:

[0062] ;

[0063] In the formula, is the minimum side length of the template image after multiple downsampling;

[0064] S410: Assume the chip gate bonding area as the bottom layer of the image pyramid, and match and locate the downsampled template image and the image of the chip to be tested layer by layer in the order from the top layer to the bottom layer of the image pyramid, and calculate the normalized correlation coefficient of each layer by the following formula , each layer The position of the search window at the maximum value is taken as the optimal matching positioning position, and the positioning position coordinates of the chip gate bonding area are obtained. The normalized correlation coefficients of each layer The calculation formula is:

[0065] ;

[0066] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template image and the chip image to be tested;

[0067] S411: Rotate the positioning coordinates of the chip gate bonding area , and transform to obtain the center point coordinates of the chip gate bonding area.

[0068] Compared with the prior art, the invention can achieve the following beneficial effects:

[0069] (1) The present invention combines the visual positioning method based on image features with the improved NCC visual positioning method. The outline of the chip gate bonding area is first used as the image feature to roughly locate the chip gate bonding area, and the center point coordinates and rotation angle of the chip gate bonding area are calculated. Then, the local rectangular search area and matching angle are designed based on this, and the center point of the chip gate bonding area is accurately located in combination with the improved NCC visual positioning method. While ensuring the positioning accuracy, it not only solves the disadvantage of the traditional NCC-based visual positioning method that the positioning speed is slow, but also solves the problem that the positioning method based on image features is sensitive to erroneous feature information, resulting in mismatching.

[0070] (2) The present invention is less sensitive to image noise and less affected by light. die-Bond ) is poor in levelness and the image is dark, the visual positioning method of the present invention is less affected by the linear change of the image grayscale. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings:

[0072] Figure 1 It is a schematic diagram of the logical structure of the chip visual positioning system based on image contour feature and grayscale matching according to an embodiment of the invention;

[0073] Figure 2 It is a schematic diagram of the installation positions of the image acquisition module and the light source illumination module according to the embodiment of the invention;

[0074] Figure 3 It is a schematic diagram of the process of the chip visual positioning method according to the embodiment of the invention;

[0075] Figure 4 is a schematic diagram of a grayscale image of a chip to be tested according to an embodiment of the invention;

[0076] Figure 5 is a schematic diagram of a binarized image of a chip to be tested according to an embodiment of the invention;

[0077] Figure 6 is a schematic diagram of an image of a chip to be tested after a mathematical closing operation according to an embodiment of the invention;

[0078] Figure 7 is a schematic diagram of an image of a chip to be tested after mathematical opening operation according to an embodiment of the invention;

[0079] Figure 8 is a schematic diagram of the entire outline of the image of the chip to be tested extracted according to the embodiment of the invention;

[0080] Fig. 9 is a schematic diagram of the outline after the coarse screening according to the embodiment of the invention;

[0081] Fig.10 is a schematic diagram of the outline of the chip gate bonding area after fine screening according to an embodiment of the invention;

[0082] Fig.11 It is a schematic diagram of a local rectangular search area and a minimum circumscribed rectangle according to an embodiment of the invention.

[0083] The reference numerals therein include: image acquisition module 1, camera 11, bi-telecentric lens 12, light source illumination module 2, image preprocessing module 3, image matching and positioning module 4, network communication module 5, mounting head 6, suction nozzle 7. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0085] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0086] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0087] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0088] The following will refer to Figure 1-Figure 11 The invention is described in detail with reference to the embodiments.

[0089] like Figure 1 and Figure 2 As shown, the chip visual positioning system based on image contour features and grayscale matching provided by the embodiment of the present invention includes two parts, a front-end and a back-end. The front-end part includes an image acquisition module 1 and a light source illumination module 2, and the back-end part includes an image preprocessing module 3, an image matching and positioning module 4 and a network communication module 5.

[0090] like Figure 2 As shown, the image acquisition module 1 includes a high-resolution camera 11 and a dual telecentric lens 12. The resolution of the camera 11 is × , the pixel size is × (unit: μm); the magnification of the bi-telecentric lens 12 is . Assume the size of the chip to be tested is × , then the parameters of the camera 11 and the bi-telecentric lens 12 satisfy the following formula, so that the field of view of the camera 11 can completely cover the chip to be tested and meet the image acquisition requirements.

[0091] .

[0092] At the same time, the camera 11 and the double telecentric lens 12 are connected through Interface or The interface is connected and fixed, and the connected and fixed camera 11 and the dual telecentric lens 12 are fixed to one side of the mounting head 6 through a metal bracket. A suction nozzle 7 for sucking the copper clip is fixed on the other side of the mounting head 6, so that the camera 11 can vertically downwardly capture the image of the chip to be tested in the chip frame tray. In actual use, the camera 11 transmits the captured image of the chip to be tested to the back-end part through the gigabit network port.

[0093] The light source illumination module 2 includes a point light source with adjustable brightness and a light source controller. The light source voltage of the point light source is 5V. During installation, the point light source is fixed in the light source hole of the double telecentric lens by a top screw, and the illumination direction is downward. The point light source is connected to the light source controller, and the light source controller is connected to the external trigger interface of the camera 11. The illumination mode of the point light source adopts coaxial light illumination to provide a clear and stable light source for the image acquisition module 1. During image acquisition, the point light source is illuminated in stroboscopic mode.

[0094] Point light sources include, but are not limited to, red light sources, green light sources, blue light sources, and white light sources.

[0095] The image preprocessing module 3 is used to preprocess the image of the chip to be tested based on mathematical morphological operations, so that the outline of the chip gate bonding area is clear.

[0096] The image preprocessing module 3 includes an image grayscale processing unit, an image binarization processing unit, a mathematical morphology closing operation unit, and a mathematical morphology opening operation unit; wherein,

[0097] The image grayscale processing unit is used to perform grayscale processing on the image of the chip to be tested;

[0098] The image binarization processing unit is used to perform binarization processing on the grayscale image of the chip to be tested;

[0099] The mathematical morphology closing operation unit is used to perform a mathematical morphology closing operation on the binary image of the chip to be tested, so as to filter out black spots in the image of the chip to be tested;

[0100] The mathematical morphology opening operation unit is used to perform a mathematical morphology opening operation on the image of the chip to be tested with black spots filtered out, so as to remove noise and obtain the image of the chip to be tested with a clear outline of the gate bonding area.

[0101] The image matching and positioning module 4 is used to extract the contour of the chip gate bonding area of ​​the preprocessed chip image to be tested, calculate the center point coordinates and rotation angle of the chip gate bonding area, design a local rectangular search area and matching angle based on the center point coordinates and rotation angle and use it as the search area, use the improved NCC visual positioning method to accurately locate the center point of the chip gate bonding area, obtain the visual positioning detection result, and transmit it to the network communication module.

[0102] The image matching and positioning module includes a contour extraction unit, a contour rough screening unit, a contour fine screening unit, a search area interception unit, a search area correction unit, a template image interception unit, a template image information calculation unit, an image local pixel and calculation unit, an image downsampling unit, an image matching unit and a coordinate transformation unit; wherein,

[0103] The contour extraction unit is used to extract all contours of the preprocessed image of the chip to be tested by using a digital binary image topology method based on boundary tracking;

[0104] The contour rough screening unit is used to perform rough screening on all extracted contours according to the set contour area threshold;

[0105] The contour fine screening unit is used to calculate the minimum circumscribed rectangle of the contour after the rough screening, and calculate the length, width, area, center point coordinates and rotation angle of the minimum circumscribed rectangle , taking the area of ​​the minimum circumscribed rectangle and the length and width difference of the minimum circumscribed rectangle as screening conditions, the rough-screened contour is finely screened to obtain the contour of the chip gate bonding area;

[0106] The search area interception unit is used to intercept a local rectangular search area covering the chip gate bonding area in the image of the chip to be tested, and to connect the center point of the minimum circumscribed rectangle with the rotation angle Set as the center point and rotation angle of the local rectangular search area;

[0107] The search area correction unit is used to rotate the intercepted local rectangular search area , to correct the direction of the local rectangular search area;

[0108] The template image capture unit is used to capture the template image containing the chip gate bonding area in the image of the chip to be tested, and calculate the center point coordinates of the chip gate bonding area in the template image. and width The smaller value of ;

[0109] The template image information calculation unit is used to calculate the template grayscale mean value according to the grayscale information of the template image. , the standard deviation of the biased sample of the chip image to be tested and the standard deviation of the biased samples of the template image ;

[0110] The image local pixel sum calculation unit is used to calculate the image local pixel sum of the search window by using the image integral map method through the following formula: :

[0111] ;

[0112] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the chip to be tested, The coordinate of the upper left corner of the search window in the integral image of the chip to be tested;

[0113] The image downsampling unit is used to downsample the template image and the local rectangular search area image using the image pyramid method. L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula:

[0114] ;

[0115] In the formula, is the minimum side length of the template image after multiple downsampling;

[0116] The image matching unit is used to match and locate the downsampled template image with the image of the chip to be tested layer by layer in the order from the top layer to the bottom layer of the image pyramid, with the chip gate bonding area as the bottom layer, and calculate the normalized correlation coefficient of each layer by the following formula: , each layer The position of the search window at the maximum value is taken as the optimal matching positioning position, and the positioning position coordinates of the chip gate bonding area are obtained. The normalized correlation coefficients of each layer The calculation formula is:

[0117] ;

[0118] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template image and the chip image to be tested;

[0119] The coordinate transformation unit is used to rotate the positioning coordinates of the chip gate bonding area , and transform to obtain the center point coordinates of the chip gate bonding area.

[0120] The network communication module 5 realizes the network communication between the chip visual positioning system and the bonding packaging equipment control terminal based on the socket, and is used to transmit the visual positioning detection result to the bonding packaging equipment control terminal through the network port.

[0121] The hardware of the image preprocessing module 3, the image matching and positioning module 4 and the network communication module 5 are implemented based on an industrial computer, and the software platform is based on Linguistic Software and install the corresponding version Library package.

[0122] The above content describes in detail the structure of the chip vision positioning system based on image contour features and grayscale matching provided by the embodiment of the present invention. Corresponding to the chip vision positioning system, the embodiment of the present invention also provides a chip vision positioning method implemented by the chip vision positioning system.

[0123] like Figure 3 As shown, the chip visual positioning method described in the embodiment of the present invention comprises the following steps:

[0124] S1: Move the image acquisition module to the chip detection position.

[0125] The bonding packaging equipment control terminal controls the camera to move to a position above the chip to be tested.

[0126] S2: Control the light source illumination module to illuminate the chip to be tested in a stroboscopic mode, and control the image acquisition module to acquire an image of the chip to be tested, so as to obtain an image of the chip to be tested.

[0127] The bonding packaging equipment control terminal sends control instructions to the light source illumination module and the image acquisition module. The light source illumination module illuminates the chip to be tested in a stroboscopic mode according to the control instructions, and the image acquisition module acquires the image of the chip to be tested according to the control instructions to obtain the image of the chip to be tested.

[0128] S3: The image of the chip to be tested is preprocessed based on mathematical morphological operations through the image preprocessing module, so that the outline of the chip gate bonding area is clear.

[0129] The method for preprocessing the image of the chip to be tested of the present invention is based on The library is implemented to obtain the outline of the tiny chip gate bonding area, laying the foundation for the next step of coarse positioning.

[0130] The specific steps of the preprocessing method are as follows:

[0131] S301: Call In the library The function performs grayscale processing on the image of the chip to be tested.

[0132] The grayscale image of the chip under test is as follows: Figure 4 shown.

[0133] S302: Call Function, based on Otsu's method, performs binarization processing on the grayscale image of the chip to be tested to obtain a binary image of the chip to be tested.

[0134] The image of the chip to be tested after binarization is as follows Figure 5 shown.

[0135] S303: performing a mathematical morphological closing operation on the binary image of the chip to be tested to filter out black spots in the image of the chip to be tested.

[0136] The black dots in the image of the chip under test are caused by the probe test of the chip under test. Figure 6 shown.

[0137] S304: performing a mathematical morphological opening operation on the image of the chip to be tested with the black spots filtered out to remove noise, thereby obtaining an image of the chip to be tested with a clear outline of the gate bonding area.

[0138] The image of the chip under test after the mathematical morphological closing operation is opened by mathematical morphological opening operation, so that the noise is removed while retaining the original image of the chip under test, and the image of the chip under test with clear boundaries of the chip gate bonding area and less image noise interference is obtained. Figure 7 As shown, after the image of the chip to be tested is preprocessed, the outline of the chip gate bonding area is relatively clear and the image noise is relatively small.

[0139] S4: The preprocessed chip image to be tested is used to extract the outline of the chip gate bonding area through the image matching and positioning module, and the center point coordinates and rotation angle of the chip gate bonding area are calculated. Based on the center point coordinates and rotation angle, a local rectangular search area and matching angle are designed and used as the search area. The center point of the chip gate bonding area is accurately located using the improved NCC visual positioning method to obtain the visual positioning detection result, and the visual positioning detection result is transmitted to the bonding packaging equipment control terminal through the network communication module.

[0140] In order to improve the positioning speed of the chip gate bonding area, the present invention first uses the contour features of the chip gate bonding area to roughly locate the image of the chip to be tested, and obtains the center point coordinates and rotation angle of the chip gate bonding area. In order to subsequently use the improved NCC visual positioning method to accurately locate the center point coordinates of the chip gate bonding area, a large amount of redundant grayscale information in the image is eliminated and the matching angle information constraints are added, thereby effectively improving the image matching positioning efficiency.

[0141] The specific steps for coarse positioning of the chip image to be tested based on the contour features of the chip gate bonding area are as follows:

[0142] S401: Using the digital binary image topology method based on boundary tracking, call The function extracts all the contours of the preprocessed chip image to be tested.

[0143] The entire outline of the image of the chip to be tested is extracted as follows: Figure 8 shown.

[0144] S402: performing a rough screening of all extracted contours according to a set contour area threshold.

[0145] The threshold is set according to the area of ​​the area surrounded by the contour, and all the extracted contours are screened in descending order to select the contour with the largest area. contours, such as Fig. 9 shown.

[0146] S403: By calling The function calculates the minimum bounding rectangle corresponding to the chip gate bonding area contour for the rough-screened contour, and calculates the length, width, area, center point coordinates and rotation angle of the minimum bounding rectangle. , taking the area of ​​the minimum circumscribed rectangle and the length and width difference of the minimum circumscribed rectangle as screening conditions, the contour that has passed the coarse screening is finely screened to obtain the contour of the chip gate bonding area.

[0147] Since the chip gate bonding area can be regarded as a standard square, the area of ​​the minimum enclosing rectangle and the difference between the length and width of the minimum enclosing rectangle are used as the constraints for the second screening of the contour to obtain the outer contour of the chip gate area, such as Fig.10 shown.

[0148] S404: intercepting a local rectangular search area covering the chip gate bonding area in the image of the chip to be tested, and aligning the center point of the minimum circumscribed rectangle with the rotation angle Set as the center point and rotation angle of the local rectangular search area.

[0149] Assume that the side length of the square chip gate bonding area is , the short side length of the local rectangular search area is ,but .like Fig.11 As shown, the yellow local rectangular search area covers the green chip gate bonding area.

[0150] S405: Rotate the intercepted local rectangular search area , to positively orient the local rectangular search area.

[0151] After the direction of the local rectangular search area is corrected, it lays the foundation for subsequent precise matching and positioning.

[0152] The present invention obtains the local rectangular search area of ​​the chip gate bonding area in the image of the chip to be tested through coarse positioning, and obtains the angle information of matching positioning after processing. In order to further improve the positioning accuracy of the chip gate bonding area, reduce the influence of image noise, image illumination and linear change of image grayscale, and improve the positioning speed and stability of the positioning method, the present invention designs an improved NCC visual positioning method to quickly and accurately locate the center point coordinates of the chip gate bonding area in the local rectangular search area.

[0153] The specific steps of using the improved NCC visual positioning method to accurately locate the center point coordinates of the chip gate bonding area are as follows:

[0154] S406: intercepting a template image including the chip gate bonding area in the image of the chip to be tested, and calculating the coordinates of the center point of the chip gate bonding area in the template image, and converting the length of the template image and width The smaller value of .

[0155] S407: Calculate the template grayscale mean value based on the grayscale information of the template image , the standard deviation of the biased sample of the chip image to be tested and the standard deviation of the biased samples of the template image .

[0156] S408: Using the image integral graph method, the local pixel sum of the image in the search window is calculated by the following formula: :

[0157] ;

[0158] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the chip to be tested, It is the coordinate of the upper left corner of the search window in the integral image of the chip to be tested.

[0159] In the process of calculating the normalized mutual correlation coefficient between the chip image to be tested and the template image, the present invention uses an integral graph method to quickly calculate the local pixel sum of the image, which can improve the matching positioning speed of the improved NCC visual positioning method.

[0160] S409: Using the image pyramid method to perform image retrieval on the template image and the local rectangular search area image L-1 Downsampling, Lis the number of image pyramid layers, which is calculated by the following formula:

[0161] ;

[0162] In the formula, is the minimum side length of the template image after multiple downsampling.

[0163] The present invention adopts the image pyramid method to perform downsampling processing on the chip image to be tested and the template image to further improve the matching positioning speed. Suppose the minimum side length of the template image after multiple downsampling is ( ), use the above formula to calculate the number of layers of the image pyramid L , and according to the number of layers L The template image and the chip image to be tested are compared Downsampling.

[0164] S410: Assume the chip gate bonding area as the bottom layer of the image pyramid, and match and locate the downsampled template image and the image of the chip to be tested layer by layer in the order from the top layer to the bottom layer of the image pyramid, and calculate the normalized correlation coefficient of each layer by the following formula , each layer The position of the search window at the maximum value is taken as the optimal matching positioning position, and the positioning position coordinates of the chip gate bonding area are obtained. The normalized correlation coefficients of each layer The calculation formula is:

[0165] ;

[0166] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template image and the chip image to be tested.

[0167] The present invention uses the image of the chip gate bonding area as the bottom layer of the image pyramid, that is, L According to the order of the image pyramid from the top layer to the bottom layer, the downsampled template image is matched with the chip image to be tested layer by layer, and the normalized correlation coefficient of each layer in the image pyramid is calculated. , with each layer When the maximum value is reached, the position of the search window is the optimal matching location.

[0168] When matching and locating the top layer of the image pyramid, select the image of the chip to be tested in this layer for global matching. When matching and locating the next layer, use the positioning position of the previous layer, multiply the coordinates of the matching positioning reference point by 2, and then use the set Interval, small neighborhood matching is performed in the lower layer of the chip image to be tested, and the normalized mutual correlation coefficient is calculated using the above formula when positioning each layer , until the bottom layer of the image pyramid is matched and positioned, and the positioning coordinates of the chip gate bonding area are obtained.

[0169] S411: Rotate the positioning coordinates of the chip gate bonding area , and transform to obtain the center point coordinates of the chip gate bonding area.

[0170] The present invention combines the visual positioning method based on image features with the visual positioning method based on NCC. First, the outline of the chip gate bonding area is used as the image feature to roughly locate the chip gate bonding area, and the center point coordinates and rotation angle of the chip gate bonding area are calculated. Then, the local rectangular search area and matching angle are designed based on this, and the center point of the chip gate bonding area is accurately located in combination with the improved NCC visual positioning method. While ensuring the positioning accuracy, it not only solves the disadvantage of the traditional NCC-based visual positioning method that the positioning speed is slow, but also solves the problem that the positioning method based on image features is sensitive to erroneous feature information, resulting in mismatching.

[0171] The present invention is less sensitive to image noise and less affected by light. die-Bond ) is poor in levelness and the image is dark, the visual positioning method of the present invention is less affected by the linear change of the image grayscale.

[0172] It should be noted that for the visual positioning of the chip to be tested at the dispensing station, since there will be glue spots in the chip gate bonding area after dispensing, if the size and position consistency of the glue spots in the image of the chip to be tested are poor, the chip gate bonding area image in the image of the chip to be tested will be poorly similar to the template image, affecting the matching positioning accuracy. Therefore, other feature areas near the chip gate bonding area can also be selected as template images for matching positioning. Then, the positioning results are combined with the chip design size to recalculate the center point coordinates of the chip gate bonding area.

[0173] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0174] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A chip visual positioning system based on image contour features and grayscale matching, characterized in that: It includes an image acquisition module, a light source illumination module, an image preprocessing module, an image matching and positioning module, and a network communication module; wherein, The light source illumination module is used to provide illumination for the image acquisition module; The image acquisition module is used to take pictures of the chip to be tested and transmit the obtained image of the chip to be tested to the image preprocessing module; The image preprocessing module is used to preprocess the image of the chip to be tested based on mathematical morphological operations, so that the outline of the chip gate bonding area is clear; The image matching and positioning module is used to extract the contour of the chip gate bonding area from the preprocessed chip image to be tested, calculate the center point coordinates and rotation angle of the chip gate bonding area, design a local rectangular search area and matching angle based on the center point coordinates and rotation angle and use it as the search area, use the improved NCC visual positioning method to accurately locate the center point of the chip gate bonding area, obtain the visual positioning detection result, and transmit it to the network communication module; The image matching and positioning module includes a contour extraction unit, a contour rough screening unit, a contour fine screening unit, a search area interception unit, a search area correction unit, a template image interception unit, a template image information calculation unit, an image local pixel and calculation unit, an image downsampling unit, an image matching unit and a coordinate transformation unit; wherein, The contour extraction unit is used to extract all contours of the preprocessed image of the chip to be tested by using a digital binary image topology method based on boundary tracking; The contour rough screening unit is used to perform rough screening on all extracted contours according to the set contour area threshold; The contour fine screening unit is used to calculate the minimum circumscribed rectangle of the contour after the rough screening, and calculate the length, width, area, center point coordinates and rotation angle of the minimum circumscribed rectangle. , taking the area of ​​the minimum circumscribed rectangle and the length and width difference of the minimum circumscribed rectangle as screening conditions, the rough-screened contour is finely screened to obtain the contour of the chip gate bonding area; The search area interception unit is used to intercept a local rectangular search area covering the chip gate bonding area in the image of the chip to be tested, and to connect the center point of the minimum circumscribed rectangle with the rotation angle Set as the center point and rotation angle of the local rectangular search area; The search area correction unit is used to rotate the intercepted local rectangular search area , to correct the direction of the local rectangular search area; The template image capture unit is used to capture the template image containing the chip gate bonding area in the image of the chip to be tested, and calculate the center point coordinates of the chip gate bonding area in the template image. and width The smaller value of ; The template image information calculation unit is used to calculate the template grayscale mean value according to the grayscale information of the template image. , the standard deviation of the biased sample of the chip image to be tested and the biased sample standard deviation of the template image ; The image local pixel sum calculation unit is used to calculate the image local pixel sum of the search window by using the image integral map method through the following formula: : ； In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the local rectangular search area of ​​the chip image to be tested, The coordinate of the upper left corner of the search window in the integral image of the local rectangular search area of ​​the image of the chip to be tested; The image downsampling unit is used to downsample the template image and the local rectangular search area image using the image pyramid method. L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula: ； In the formula, is the minimum side length of the template image after multiple downsampling; The image matching unit matches and locates the downsampled template image with the downsampled local rectangular search area image layer by layer in the order from the top layer to the bottom layer of the image pyramid, and calculates the normalized correlation coefficient of each layer by the following formula: , each layer The position of the search window at the maximum value is taken as the optimal matching positioning position, and the positioning position coordinates of the chip gate bonding area are obtained. The normalized correlation coefficients of each layer The calculation formula is: ； In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template image and the search window image within the local rectangular search area; The coordinate transformation unit is used to rotate the positioning coordinates of the chip gate bonding area , transform to obtain the coordinates of the center point of the chip gate bonding area; The network communication module is used to transmit the visual positioning detection results to the bonding packaging equipment control terminal through the network port.

2. The chip visual positioning system based on image contour feature and grayscale matching according to claim 1, characterized in that: The image acquisition module includes a camera and a dual telecentric lens. The camera and the dual telecentric lens are connected and fixed using a C interface or a CS interface. The camera and the dual telecentric lens are fixed as a whole on one side of the placement head, and the camera collects images vertically downward.

3. The chip visual positioning system based on image contour feature and grayscale matching according to claim 2, characterized in that: The camera resolution is × , pixel size × The magnification of the bi-telecentric lens is , assuming the size of the target area of ​​the chip to be tested is × , then the parameters of the camera and the bi-telecentric lens satisfy the following formula: 。 4. The chip visual positioning system based on image contour feature and grayscale matching according to claim 2 or 3, characterized in that: The light source illumination module includes a point light source and a light source controller. The point light source is connected to the light source controller. The light source controller is connected to the external trigger interface of the camera. The point light source is fixed in the light source hole of the double telecentric lens through a top screw, and the illumination direction is downward.

5. The chip visual positioning system based on image contour feature and grayscale matching according to claim 1, characterized in that: The image preprocessing module includes an image grayscale processing unit, an image binarization processing unit, a mathematical morphology closing operation unit, and a mathematical morphology opening operation unit; wherein, The image grayscale processing unit is used for grayscale processing of the image of the chip to be tested; The image binarization processing unit is used to perform binarization processing on the grayscale image of the chip to be tested; The mathematical morphology closing operation unit is used to perform a mathematical morphology closing operation on the binary image of the chip to be tested, so as to filter out black spots in the image of the chip to be tested; The mathematical morphology opening operation unit is used to perform a mathematical morphology opening operation on the image of the chip to be tested with black spots filtered out, so as to remove noise and obtain the image of the chip to be tested with a clear outline of the gate bonding area.

6. A chip visual positioning method implemented by the chip visual positioning system based on image contour features and grayscale matching according to any one of claims 1 to 5, characterized in that: The steps include: S1: Move the image acquisition module to the chip detection position; S2: Control the light source illumination module to illuminate the chip under test in a stroboscopic mode, and control the image acquisition module to acquire an image of the chip under test to obtain an image of the chip under test; S3: Preprocessing the image of the chip to be tested based on mathematical morphological operations through the image preprocessing module, so that the outline of the chip gate bonding area is clear; S4: extract the outline of the chip gate bonding area from the preprocessed chip image through the image matching and positioning module, calculate the center point coordinates and rotation angle of the chip gate bonding area, design a local rectangular search area and matching angle based on the center point coordinates and rotation angle and use it as the search area, use the improved NCC visual positioning method to accurately locate the center point of the chip gate bonding area, obtain the visual positioning detection result, and transmit the visual positioning detection result to the bonding packaging equipment control terminal through the network communication module; Step S4 specifically includes the following steps: S401: extracting all contours of the preprocessed image of the chip to be tested by using a digital binary image topology method based on boundary tracking; S402: performing a rough screening of all extracted contours according to a set contour area threshold; S403: Calculate the minimum circumscribed rectangle of the rough-screened contour, and calculate the length, width, area, center point coordinates and rotation angle of the minimum circumscribed rectangle , taking the area of ​​the minimum circumscribed rectangle and the length and width difference of the minimum circumscribed rectangle as screening conditions, the rough-screened contour is finely screened to obtain the contour of the chip gate bonding area; S404: intercepting a local rectangular search area covering the chip gate bonding area in the image of the chip to be tested, and aligning the center point of the minimum circumscribed rectangle with the rotation angle Set as the center point and rotation angle of the local rectangular search area; S405: Rotate the intercepted local rectangular search area , to correct the direction of the local rectangular search area; S406: intercepting a template image including the chip gate bonding area in the image of the chip to be tested, and calculating the coordinates of the center point of the chip gate bonding area in the template image, and converting the length of the template image and width The smaller value of ; S407: Calculate the template grayscale mean value based on the grayscale information of the template image , the standard deviation of the biased sample of the chip image to be tested and the biased sample standard deviation of the template image ; S408: Using the image integral graph method, the local pixel sum of the image in the search window is calculated by the following formula: : ； In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the local rectangular search area of ​​the chip image to be tested, The coordinate of the upper left corner of the search window in the integral image of the local rectangular search area of ​​the image of the chip to be tested; S409: Using the image pyramid method to perform image retrieval on the template image and the local rectangular search area image L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula: ； In the formula, is the minimum side length of the template image after multiple downsampling; S410: According to the order from the top layer to the bottom layer of the image pyramid, the downsampled template image and the downsampled local rectangular search area image are matched and located layer by layer, and the normalized correlation coefficient of each layer is calculated by the following formula , each layer The position of the search window at the maximum value is taken as the optimal matching positioning position, and the positioning position coordinates of the chip gate bonding area are obtained. The normalized correlation coefficients of each layer The calculation formula is: ； In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template image and the search window image within the local rectangular search area; S411: Rotate the positioning coordinates of the chip gate bonding area , and transform to obtain the center point coordinates of the chip gate bonding area.

7. The chip visual positioning method according to claim 6, characterized in that: Step S3 specifically includes the following steps: S301: grayscale the image of the chip to be tested; S302: Binarization processing is performed on the grayscale image of the chip to be tested; S303: performing a mathematical morphological closing operation on the binary image of the chip to be tested to filter out black spots in the image of the chip to be tested; S304: performing a mathematical morphological opening operation on the image of the chip to be tested with the black spots filtered out to remove noise, thereby obtaining an image of the chip to be tested with a clear outline of the gate bonding area.

Images