Visual recognition positioning system and method based on preamble guidance and image grayscale matching

Abstract

The present invention relates to the field of visual inspection technology, and in particular to a visual recognition positioning system and method based on pre-order guidance and image grayscale matching. When inspecting an image of a test piece, it is first determined whether the system has a pre-order inspection result. If a pre-order inspection result exists, the position of the pre-order inspection result is used as the reference of a local matching area, and a local matching area is obtained based on the pre-order guidance and image grayscale matching. NCC image matching visual positioning; if there is no previous detection result, the image of the tested object is globally matched based on NCC The image matching visual positioning is performed, and the visual positioning detection result is used as the previous detection result to provide the previous guidance information in the next detection. The present invention can use the previous detection result to exclude a large amount of invalid image grayscale information, while reducing the amount of calculation and improving the positioning speed, ensuring high positioning accuracy and good algorithm stability.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual detection, and in particular relates to a visual recognition and positioning system and method based on preamble guidance and image grayscale matching. Background Art

[0002] Power semiconductor devices are the core devices for electric energy conversion. The quality and reliability of power semiconductor devices are closely related to the packaging process. At present, in order to reduce the thermal and electrical impedance and bonding damage of power semiconductor devices to a greater extent and meet the increasing process requirements of high voltage, high current, high reliability, etc., the packaging technology of power semiconductor devices has begun to develop from the traditional full-lead bonding packaging or the hybrid bonding packaging of lead and copper clips to the 2D plane and 3D stacked full copper clip bonding packaging. Full copper clip bonding packaging has begun to become the most advanced bonding packaging process for the manufacture of power semiconductor devices today.

[0003] In the full copper clip bonding packaging process, the bonding part is the main failure part of the power semiconductor device because it is mainly subjected to the repeated opening and closing of the current. In order to ensure the stability and reliability of the power semiconductor device after bonding packaging, it is necessary to ensure the accuracy of the copper clip mounting position on the basis of efficient packaging, that is, the bonding packaging equipment needs to quickly obtain high-precision position information of the copper clip bonding area at the copper clip mounting station. Therefore, designing a high-precision and fast visual recognition and positioning system for the copper clip bonding area that is correctly bridged with the chip gate is a key link in realizing advanced bonding packaging.

[0004] There are two main types of copper clip visual recognition and positioning methods in the existing bonding packaging technology. One type is a visual recognition and positioning method based on the Normalized Correlation Coefficient (NCC). By calculating the normalized correlation coefficient between the detection image and the template image obtained by the visual system, the search window position when the normalized correlation coefficient is the maximum is selected as the positioning position, so as to realize the visual recognition and positioning of the bonding area at the upper end of the copper clip. This method has high positioning accuracy, low sensitivity to image noise, and little influence by illumination. However, since the matching strategy is ergodic, the large amount of calculation will lead to slow matching and positioning speed. At the same time, the flexibility of angle measurement is poor. Angle matching will further increase the amount of calculation and prolong the matching and positioning time. The other type is a matching method based on image features, which judges the similarity between the image to be tested and the standard features of the template, analyzes the positional relationship between the image to be tested and the standard features of the template, and thus obtains the position and angle information. This method is more flexible in feature selection. The shape parameters, edges, corner points, various moments, etc. of the image can all be used as matching features. In addition, since the significant features of the image are extracted and the effective information of the image is obtained, the amount of calculation is significantly reduced, and the positioning speed is effectively improved. However, precisely because only a small part of the grayscale information is used, this type of method is more sensitive to errors in feature extraction and feature matching, and erroneous features need to be eliminated. Feature extraction and positioning directly affect the accuracy of the matching algorithm, so the stability of this type of method is slightly poor. Summary of the invention

[0005] In view of this, the present invention aims to provide a visual recognition positioning system and method based on pre-order guidance and image grayscale matching, so as to improve the positioning speed, positioning accuracy and robustness of the visual recognition positioning of the copper clip.

[0006] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0007] A visual recognition and positioning system based on pre-order guidance and image grayscale matching includes an image acquisition module, a light source illumination module, a detection and guidance module, an image matching and positioning module, and a network communication module; wherein:

[0008] The light source illumination module is used to illuminate the workpiece to be tested;

[0009] The image acquisition module is used to take pictures of the test piece and transmit the obtained image of the test piece to the detection guidance module;

[0010] The detection guidance module is used to detect whether the visual recognition positioning system has a previous detection result. If so, the image of the test piece is roughly positioned according to the previous detection result to obtain a local matching area; if not, the image of the test piece is used as a global matching area;

[0011] The image matching and positioning module is used to use the improved NCC algorithm to accurately locate the target area in the local matching area or the global matching area, obtain the visual positioning detection result, and transmit it to the network communication module; wherein, when accurately locating the target area in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection;

[0012] The network communication module is used to transmit the visual positioning detection results to the bonding packaging equipment control terminal through the network port.

[0013] Furthermore, the image acquisition module includes a camera, a double telecentric lens and a steering prism. The camera and the double telecentric lens are connected using a C interface or a CS interface. The double telecentric lens is installed on the steering prism. The camera, the double telecentric lens and the steering prism are installed as a whole on a lifting and translation platform through a lens base.

[0014] Furthermore, the camera resolution is × , pixel size × The magnification of the bi-telecentric lens is , assuming the size of the target area of ​​the test piece is × , then the parameters of the camera and the bi-telecentric lens satisfy the following formula:

[0015] .

[0016] Furthermore, the light source illumination module includes a ring light source and a light source controller. The ring light source is connected to the light source controller. The light source controller is connected to the external trigger interface of the camera. The ring light source is fixed above the lens port of the steering prism through a bracket. The illumination direction of the ring light source is upward.

[0017] Furthermore, the detection guidance module includes a coordinate point calculation unit, which calculates the coordinates of the reference points in the local matching area by the following formula: :

[0018] ;

[0019] In the formula, is the coordinate of the visual positioning reference point in the previous detection result, × is the size of the template image, × is the size of the local matching area, , .

[0020] Furthermore, the image matching and positioning module includes a template image interception unit, a template image information calculation unit, an image local pixel calculation unit, an image downsampling unit, an image matching unit and a target area center point coordinate calculation unit; wherein,

[0021] The template image capture unit is used to capture the template image containing the target area in the image of the test piece and obtain the center point coordinates of the target area in the template image. , the length of the template image and width The smaller value of min_wh ;

[0022] The template image information calculation unit is used to calculate the grayscale mean of the template image according to the grayscale information of the template image. , the standard deviation of the biased sample of the test piece image and the standard deviation of the biased samples of the template image ;

[0023] The image local pixel sum calculation unit is used to calculate the image local pixel sum of the search window by using the image integral map method through the following formula: :

[0024] ;

[0025] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the DUT image, The coordinates of the upper left corner of the search window in the integral image of the device under test;

[0026] The image downsampling unit is used to downsample the template image and the image of the object to be tested using the image pyramid method. L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula:

[0027] ;

[0028] In the formula, is the minimum side length of the template image after multiple downsampling;

[0029] The image matching unit is used to match and locate the downsampled template image and the image of the object to be tested layer by layer, and calculate the normalized correlation coefficient of each layer by the following formula: , each layer The position of the search window at the maximum value is taken as the optimal matching location:

[0030] ;

[0031] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template and the image of the object to be tested;

[0032] The center point coordinate calculation unit of the target area is used to obtain the positioning reference point coordinates of the local matching area. , combined with the center point coordinates of the target area in the template image Coordinates of the reference points in the local matching area , calculate the center point coordinates of the target area in the image of the test piece:

[0033] .

[0034] A visual recognition positioning method implemented by the above-mentioned visual recognition positioning system based on preamble guidance and image grayscale matching comprises the following steps:

[0035] S1: Move the DUT to the testing position through the bonding and packaging equipment;

[0036] S2: The bonding and packaging equipment control terminal sends a positioning instruction to the light source illumination module and the image acquisition module. The light source illumination module illuminates the DUT in a stroboscopic mode, and the image acquisition module acquires an image of the DUT to obtain an image of the DUT.

[0037] S3: Detecting whether there is a previous detection result in the visual recognition positioning system through the detection guidance module. If yes, roughly positioning the image of the object to be tested according to the previous detection result to obtain a local matching area; if no, taking the image of the object to be tested as a global matching area;

[0038] S4: Using the improved NCC algorithm through the image matching positioning module to accurately locate the target area in the local matching area or the global matching area, obtain the visual positioning detection result, and transmit the visual positioning detection result to the bonding packaging equipment control terminal through the network communication module; wherein, when accurately locating the target area in the global matching area, the visual positioning detection result is used as the previous detection result of the next detection.

[0039] Further, in step S3, the coordinates of the visual positioning reference point in the previous detection result are set to , the size of the template image is × , the size of the local matching area is × , , , then the coordinates of the reference points in the local matching area are calculated by the following formula :

[0040] .

[0041] Furthermore, step S4 specifically includes the following steps:

[0042] S41: intercepting a template image containing a target area in the image of the object to be tested, and obtaining the coordinates of the center point of the target area in the template image , the length of the template image and width The smaller value of min_wh ;

[0043] S42: Calculate the grayscale mean of the template image according to the grayscale information of the template image , the standard deviation of the biased sample of the test piece image and the standard deviation of the biased samples of the template image ;

[0044] S43: Using the image integral graph method, the local pixel sum of the image in the search window is calculated by the following formula :

[0045] ;

[0046] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the DUT image, The coordinates of the upper left corner of the search window in the integral image of the device under test;

[0047] S44: Use the image pyramid method to compare the template image and the image of the test piece L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula:

[0048] ;

[0049] In the formula, is the minimum side length of the template image after multiple downsampling;

[0050] S45: Assume that the local matching area is the bottom layer of the image pyramid, and match and locate the downsampled template image and the image of the object to be tested layer by layer in the order from the top layer to the bottom layer of the image pyramid, and calculate the normalized correlation coefficient of each layer by the following formula , each layer The position of the search window at the maximum value is taken as the optimal matching location:

[0051] ;

[0052] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template and the image of the object to be tested;

[0053] S46: Obtaining the coordinates of the positioning reference points in the local matching area , combined with the center point coordinates of the target area in the template image Coordinates of the reference points in the local matching area , calculate the center point coordinates of the target area in the image of the test piece:

[0054] .

[0055] Compared with the prior art, the invention can achieve the following beneficial effects:

[0056] The present invention is in the traditional NCC Based on the algorithm, the previous detection results are combined for rapid positioning and auxiliary guidance. When the image of the test piece is detected, first determine whether the system has a previous detection result. If so, the position of the previous detection result is used as the reference of the local matching area, and the local matching area is based on NCC image matching visual positioning; if there is no previous inspection result, the image of the tested object is globally matched based on NCC The image matching visual positioning is performed, and the visual positioning detection result is used as the previous detection result to provide the previous guidance information in the next detection. The present invention can use the previous detection result to exclude a large amount of invalid image grayscale information, while reducing the amount of calculation and improving the positioning speed, ensuring high positioning accuracy and good algorithm stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings:

[0058] Figure 1 It is a logical structure diagram of a visual recognition and positioning system based on preamble guidance and image grayscale matching according to an embodiment of the invention;

[0059] Figure 2 is a schematic diagram of the installation position of the image acquisition module according to an embodiment of the invention;

[0060] Figure 3 It is a schematic diagram of the installation position of the light source lighting module according to the embodiment of the invention;

[0061] Figure 4It is a flowchart of the visual recognition positioning method according to the embodiment of the invention;

[0062] Figure 5 is a schematic diagram of preamble positioning information of a preamble detection result according to an embodiment of the present invention;

[0063] Figure 6 It is a schematic diagram of local visual positioning according to an embodiment of the invention.

[0064] The reference numerals therein include: image acquisition module 1, camera 11, bi-telecentric lens 12, steering prism 13, lens base 14, lifting and translation stage 15, light source illumination module 2, annular light source 21, bracket 22, detection and guidance module 3, image matching and positioning module 4 and network communication module 5. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0066] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0067] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0068] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0069] The following will refer to Figure 1-Figure 6 The invention is described in detail with reference to the embodiments.

[0070] The embodiments of the present invention provide a visual recognition and positioning system and method based on preceding guidance and image grayscale matching. The objects of visual recognition and positioning include but are not limited to copper clips. The copper clips are used as an example below to illustrate the visual recognition and positioning system and method based on preceding guidance and image grayscale matching provided by the embodiments of the present invention. The same can be said for other visual recognition and positioning objects.

[0071] like Figure 1 As shown, the visual recognition and positioning system based on pre-order guidance and image grayscale matching provided by the embodiment of the present invention includes two parts, a front-end and a back-end. The front-end part includes an image acquisition module 1 and a light source illumination module 2, and the back-end part includes a detection and guidance module 3, an image matching and positioning module 4 and a network communication module 5.

[0072] like Figure 2 As shown, the image acquisition module 1 includes a high-resolution camera 11 and a dual telecentric lens 12. The resolution of the camera 11 is × , the pixel size is × (unit: μm); the magnification of the bi-telecentric lens 12 is . Assume that the size of the upper bonding area of ​​the copper clip is × , then the parameters of the camera 11 and the bi-telecentric lens 12 satisfy the following formula, so that the field of view of the camera 11 can completely cover the upper bonding area of ​​the copper clip, meeting the image acquisition requirements.

[0073] .

[0074] At the same time, the camera 11 and the double telecentric lens 12 are connected through Interface or Interface connection, dual telecentric lens 12 is installed on The image acquisition module 1 is formed on a steering prism 13 of a type, and then the image acquisition module 1 is fixed on the lifting and translation stage 15 by using the lens base 14, and the image acquisition module 1 is moved horizontally and vertically by the lifting and translation stage. The steering prism 13 allows the camera 11 to look up vertically and shoot the copper clips sucked by the mounting head of the bonding packaging equipment. In actual use, the camera 11 transmits the collected copper clip images to the back-end part through the gigabit network port.

[0075] like Figure 3 As shown, the light source illumination module 2 includes a ring light source 21 with adjustable brightness and a light source controller. The light source voltage of the ring light source 21 is 24V. During installation, the ring light source 21 is fixed to the The annular light source 21 is connected to the light source controller, and the light source controller is connected to the external trigger interface of the camera 11. The annular light source 21 is illuminated by bright field illumination to provide a clear and stable light source for the image acquisition module 1. During image acquisition, the annular light source 21 is adjusted to the stroboscopic mode to illuminate the copper clip.

[0076] The annular light source 21 includes, but is not limited to, a red light source, a green light source, a blue light source, and a white light source.

[0077] The detection guidance module 3 is used to detect whether there is a previous detection result in the visual recognition positioning system. If so, the image of the test piece is roughly positioned according to the previous detection result to obtain a local matching area; if not, the image of the test piece is used as the global matching area. The previous detection result is used to perform rough positioning in the copper clip image, aiming to obtain a relatively small local matching area, laying the foundation for the next step of local precise positioning.

[0078] The detection guidance module 3 includes a coordinate point calculation unit, which calculates the coordinates of the reference points in the local matching area by the following formula: :

[0079] ;

[0080] In the formula, is the coordinate of the visual positioning reference point in the previous detection result, × is the size of the template image, × is the size of the local matching area, , .

[0081] The image matching and positioning module 4 is used to use the improved NCC algorithm to accurately locate the target area in the local matching area or the global matching area, obtain the visual positioning detection result, and transmit it to the network communication module 5. When accurately locating the target area in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection, providing pre-order guidance information for the next detection.

[0082] The image matching and positioning module 4 includes a template image interception unit, a template image information calculation unit, an image local pixel calculation unit, an image downsampling unit, an image matching unit and a target area center point coordinate calculation unit; wherein,

[0083] The template image capture unit is used to capture the template image containing the target area in the image of the test piece and calculate the center point coordinates of the target area in the template image. , the length of the template image and width The smaller value of min_wh ;

[0084] The template image information calculation unit is used to calculate the template grayscale mean value according to the grayscale information of the template image. , the standard deviation of the biased sample of the test piece image and the biased sample standard deviation of the template image ;

[0085] The image local pixel sum calculation unit is used to calculate the image local pixel sum of the search window by using the image integral map method through the following formula: :

[0086] ;

[0087] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the DUT image, is the coordinate of the upper left corner of the search window in the integral image of the DUT image;

[0088] The image downsampling unit is used to downsample the template image and the image of the object to be tested using the image pyramid method. L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula:

[0089] ;

[0090] In the formula, is the minimum side length of the template image after multiple downsampling;

[0091] The image matching unit is used to match and locate the downsampled template image and the image of the object to be tested layer by layer, and calculate the normalized correlation coefficient of each layer by the following formula: , each layer The location of the search window at the maximum value is used as the optimal matching location to obtain the coordinates of the positioning reference point of the local matching area. , the normalized correlation coefficient of each layer The calculation formula is:

[0092] ;

[0093] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template and the image of the object to be tested;

[0094] The center point coordinate calculation unit of the target area is used to calculate the coordinates of the positioning reference point of the local matching area. , the center point coordinates of the target area in the template image Coordinates of the reference points in the local matching area , calculate the center point coordinates of the target area in the image of the test piece :

[0095] .

[0096] The network communication module 5 realizes the network communication between the visual recognition and positioning system and the bonding and packaging equipment control terminal based on the socket, and is used to transmit the visual positioning detection result to the bonding and packaging equipment control terminal through the network port.

[0097] The hardware of the detection and guidance module 3, the image matching and positioning module 4 and the network communication module 5 are realized based on an industrial computer, and the software platform is based on Linguistic Software and install the corresponding version Library package.

[0098] The above content describes in detail the structure of the visual recognition and positioning system based on pre-order guidance and image grayscale matching provided by the embodiment of the present invention. Corresponding to the visual recognition and positioning system, the embodiment of the present invention also provides a visual recognition and positioning method implemented by the visual recognition and positioning system.

[0099] like Figure 4 As shown, the visual recognition positioning method described in the embodiment of the present invention comprises the following steps:

[0100] S1: Move the copper clip to the detection position through the bonding packaging equipment.

[0101] The copper clip is picked up by the placement head of the bonding packaging equipment and moved to the detection position of the camera.

[0102] S2: The bonding packaging equipment control terminal sends a positioning instruction to the light source illumination module and the image acquisition module. The light source illumination module illuminates the copper clip in a stroboscopic mode, and the image acquisition module acquires an image of the copper clip to obtain an image of the copper clip.

[0103] S3: Detect whether there is a previous detection result in the visual recognition positioning system through the detection guidance module. If so, roughly locate the copper clip image according to the previous detection result to obtain a local matching area; if not, use the copper clip image as the global matching area.

[0104] When the visual recognition and positioning system has previous detection results, such as Figure 5 As shown, let the coordinates of the visual positioning reference point in the previous detection result be , the size of the template image is × , the size of the local matching area is × , , , then the reference point of the local matching area is calculated by the following formula Coordinates :

[0105] .

[0106] When there is no previous detection result in the visual recognition and positioning system, the copper clip image is globally used as the matching area.

[0107] S4: Using the improved NCC algorithm through the image matching positioning module to accurately locate the target area in the local matching area or the global matching area, obtain the visual positioning detection result, and transmit the visual positioning detection result to the bonding packaging equipment control terminal through the network communication module; wherein, when accurately locating the target area in the global matching area, the visual positioning detection result is used as the previous detection result of the next detection.

[0108] The present invention realizes the rough positioning of the copper clip buckle image through the preamble guidance information provided by the preamble detection result, and obtains the local matching area of ​​the copper clip buckle upper end bonding area in the copper clip buckle image. In order to further improve the positioning accuracy of the copper clip buckle upper end bonding area, reduce the influence of image noise, image illumination and linear change of image grayscale, and improve the positioning speed and the stability of the positioning method, the present invention designs a method based on the improved NCC algorithm to quickly and accurately locate the target area in the local matching area to obtain the visual positioning detection result. The present invention uses the calculated center point coordinates of the copper clip buckle upper end bonding area as the visual positioning detection result.

[0109] Step S4 specifically includes the following steps:

[0110] S41: intercepting a template image including the upper end of the copper clip buckle in the copper clip buckle image, and obtaining the center point of the target area in the template image Coordinates , the length of the template image and width The smaller value of min_wh .

[0111] The present invention takes the upper end area of ​​the copper clip as the target area.

[0112] In the present invention, the center point of the target area in the template image Coordinates When constructing a template image, the operator selects or directly sets the actual target center point on the template image based on the GUI interface.

[0113] S42: Calculate the grayscale mean of the template image according to the grayscale information of the template image , the standard deviation of the biased sample of the test piece image and the biased sample standard deviation of the template image .

[0114] Grayscale mean calculation formula of template image:

[0115] ;

[0116] in, is the position in the grayscale image The pixel value of is the width of the grayscale image, h_ gray is the height of the grayscale image.

[0117] The formula for calculating the biased sample standard deviation of the template image is:

[0118] .

[0119] The grayscale mean calculation formula of the image to be tested is:

[0120] .

[0121] The calculation formula of the biased sample standard deviation of the image of the test piece is:

[0122] .

[0123] in, is the position in the image to be tested The pixel value of is the width of the image of the test piece, h_test is the height of the image of the object under test.

[0124] In actual calculation, the standard library numpy in Python language is used to calculate the grayscale mean of the template image by calling the formula in the library , the standard deviation of the biased sample of the test piece image , the grayscale mean of the image to be tested and the biased sample standard deviation of the template image .

[0125] S43: Using the image integral graph method, the local pixel sum of the image in the search window is calculated by the following formula :

[0126] ;

[0127] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the DUT image, It is the coordinate of the upper left corner of the search window in the integral image of the DUT image.

[0128] To improve NCC The matching positioning speed of the algorithm is improved. In the process of calculating the normalized mutual correlation coefficient between the copper clip image and the template image, the present invention uses the integral graph method to quickly calculate the local pixel sum of the image.

[0129] S44: Use the image pyramid method to compare the template image and the image of the test piece L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula:

[0130] ;

[0131] In the formula, is the minimum side length of the template image after multiple downsampling.

[0132] The present invention adopts the image pyramid method to downsample the copper clip image and the template image to further improve the matching positioning speed. Suppose the minimum side length of the template image after multiple downsampling is ( ), use the above formula to calculate the number of layers of the image pyramid L , and according to the number of layers L The template image and the copper clip image are Downsampling.

[0133] S45: Assume that the local matching area is the bottom layer of the image pyramid, and match and locate the downsampled template image and the copper clip image layer by layer in the order from the top layer to the bottom layer of the image pyramid, and calculate the normalized correlation coefficient of each layer by the following formula , each layer The position of the search window at the maximum value is taken as the optimal matching location:

[0134] ;

[0135] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template and the copper clip image.

[0136] The present invention uses the image where the local matching area is located as the bottom layer of the image pyramid, that is, L According to the order of the image pyramid from the top layer to the bottom layer, the downsampled template image is matched with the copper clip image layer by layer, and the normalized correlation coefficient of each layer in the image pyramid is calculated. , with each layer When the maximum value is reached, the search window position is the optimal matching position. When matching and positioning the first layer, select the copper clip image of this layer for global matching. When matching and positioning other layers, use the positioning position of the previous layer, multiply the matching positioning reference coordinates by 2, and then use the set In the interval, a small neighborhood matching is performed in the lower layer copper clip image, and the normalized mutual correlation coefficient is calculated using the above formula when positioning each layer. .

[0137] S46: Obtaining the positioning reference points of the local matching area Coordinates , combined with the center point of the target area in the template image Coordinates The reference points of the local matching area Coordinates , calculate the coordinates of the center point of the bonding area at the upper end of the copper clip in the copper clip image .

[0138] Coordinates of the center point of the bonding area at the upper end of the copper clip in the copper clip image The calculation formula is:

[0139] .

[0140] The present invention is based on the previous guidance and improved NCCThe algorithm realizes the visual positioning of the bonding area at the upper end of the copper clip. First, the copper clip image is roughly positioned through the previous detection results to obtain the local matching area, and then precise positioning is performed in the local matching area to obtain the center coordinates of the bonding area at the upper end of the copper clip. While ensuring the positioning accuracy, it solves the traditional NCC The algorithm solves the problem of slow positioning speed during matching and positioning, thus improving the overall detection efficiency of the system.

[0141] The present invention is less sensitive to image noise and less affected by light. When the copper clip is affected by the adsorption of the mounting head, resulting in poor flatness of the copper clip and darker image, the present invention is less affected by linear changes in image grayscale.

[0142] It should be noted that, depending on the chips to be packaged and bonded, it is necessary to design corresponding special-shaped copper clips of different shapes. The embodiment of the present invention is a special-shaped copper clip with a protruding upper end stamped and bent, and its protruding part bridges the chip gate. Therefore, if other parts of the designed special-shaped copper clip have similar obvious features, other feature areas can also be selected as template images for matching and positioning, and then the positioning results are combined with the copper clip design size to recalculate the coordinates of the center point of the copper clip bonding area.

[0143] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0144] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A visual recognition and positioning system based on preamble guidance and image grayscale matching, characterized in that: It includes an image acquisition module, a light source illumination module, a detection and guidance module, an image matching and positioning module, and a network communication module; wherein, The light source illumination module is used to illuminate the piece to be tested; The image acquisition module is used to take pictures of the test piece and transmit the obtained image of the test piece to the detection guidance module; The detection guidance module is used to detect whether the visual recognition positioning system has a previous detection result. If so, the image of the test piece is roughly positioned according to the previous detection result to obtain a local matching area; if not, the image of the test piece is used as a global matching area; The detection guidance module includes a coordinate point calculation unit, which calculates the coordinates of the reference points in the local matching area by the following formula: : ； In the formula, is the coordinate of the visual positioning reference point in the previous detection result, × is the size of the template image, × is the size of the local matching area, , ; The image matching and positioning module is used to use the improved NCC algorithm to accurately locate the center point of the target area in the local matching area or the global matching area, obtain the visual positioning detection result, and transmit it to the network communication module; wherein, when accurately locating the center point of the target area in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection; the image matching and positioning module includes a template image capture unit, a template image information calculation unit, an image local pixel and calculation unit, an image downsampling unit, an image matching unit and a target area center point coordinate calculation unit; wherein, The template image capture unit is used to capture the template image containing the target area in the image of the test piece and obtain the center point coordinates of the target area in the template image. , the length of the template image and width The smaller value of min_ wh ; The template image information calculation unit is used to calculate the grayscale mean of the template image according to the grayscale information of the template image. , the standard deviation of the biased sample of the test piece image and the biased sample standard deviation of the template image ; The image local pixel sum calculation unit is used to calculate the image local pixel sum of the search window by using the image integral map method through the following formula: : ； In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the DUT image, is the coordinate of the upper left corner of the search window in the integral image of the DUT image; The image downsampling unit is used to downsample the template image and the image of the object to be tested using the image pyramid method. L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula: ； In the formula, is the minimum side length of the template image after multiple downsampling; The image matching unit is used to match and locate the downsampled template image and the image of the object to be tested layer by layer, and calculate the normalized correlation coefficient of each layer by the following formula: , each layer The position of the search window at the maximum value is taken as the optimal matching location: ； In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template and the image of the object to be tested; The center point coordinate calculation unit of the target area is used to obtain the positioning reference point coordinates of the local matching area. , combined with the center point coordinates of the target area in the template image Coordinates of the reference points in the local matching area , calculate the center point coordinates of the target area in the image of the test piece: ； The network communication module is used to transmit the visual positioning detection results to the bonding packaging equipment control terminal through the network port.

2. The visual recognition and positioning system based on preamble guidance and image grayscale matching according to claim 1, characterized in that: The image acquisition module includes a camera, a double telecentric lens and a steering prism. The camera and the double telecentric lens are connected using a C interface or a CS interface. The double telecentric lens is installed on the steering prism. The camera, the double telecentric lens and the steering prism are installed as a whole on a lifting and translation platform through a lens base.

3. The visual recognition and positioning system based on preamble guidance and image grayscale matching according to claim 2, characterized in that: The camera resolution is × , pixel size × The magnification of the bi-telecentric lens is , assuming the size of the target area of ​​the test piece is × , then the parameters of the camera and the bi-telecentric lens satisfy the following formula: 。 4. The visual recognition and positioning system based on preamble guidance and image grayscale matching according to claim 2 or 3, characterized in that: The light source illumination module includes a ring light source and a light source controller. The ring light source is connected to the light source controller. The light source controller is connected to the external trigger interface of the camera. The ring light source is fixed above the lens opening of the steering prism through a bracket. The illumination direction of the ring light source is upward.

5. A visual recognition and positioning method implemented by the visual recognition and positioning system based on preamble guidance and image grayscale matching according to any one of claims 1 to 4, characterized in that: The steps include: S1: Move the DUT to the testing position through the bonding and packaging equipment; S2: The bonding and packaging equipment control terminal sends a positioning instruction to the light source illumination module and the image acquisition module. The light source illumination module illuminates the DUT in a stroboscopic mode, and the image acquisition module acquires an image of the DUT to obtain an image of the DUT. S3: Detecting whether there is a previous detection result in the visual recognition positioning system through the detection guidance module, and if so, roughly positioning the image of the object to be tested according to the previous detection result to obtain a local matching area; If it does not exist, the image of the object to be tested is used as the global matching area; where the coordinates of the visual positioning reference point in the previous detection result are set to , the size of the template image is × , the size of the local matching area is × , , , then the coordinates of the reference points in the local matching area are calculated by the following formula : ； S4: using the improved NCC algorithm to accurately locate the center point of the target area in the local matching area or the global matching area through the image matching positioning module, obtaining the visual positioning detection result, and transmitting the visual positioning detection result to the bonding packaging equipment control terminal through the network communication module; wherein, when accurately locating the center point of the target area in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection; Step S4 specifically includes the following steps: S41: intercepting a template image containing a target area in the image of the object to be tested, and obtaining the coordinates of the center point of the target area in the template image , the length of the template image and width The smaller value of min_wh ; S42: Calculate the grayscale mean of the template image according to the grayscale information of the template image , the standard deviation of the biased sample of the test piece image and the biased sample standard deviation of the template image ; S43: Using the image integral graph method, the local pixel sum of the image in the search window is calculated by the following formula : ； In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the DUT image, is the coordinate of the upper left corner of the search window in the integral image of the DUT image; S44: Use the image pyramid method to compare the template image and the image of the test piece L-1 Downsampling, L is the number of image pyramid layers, which is calculated by the following formula: ； In the formula, is the minimum side length of the template image after multiple downsampling; S45: Assume that the local matching area is the bottom layer of the image pyramid, and match and locate the downsampled template image and the image of the object to be tested layer by layer in the order from the top layer to the bottom layer of the image pyramid, and calculate the normalized correlation coefficient of each layer by the following formula , each layer The position of the search window at the maximum value is taken as the optimal matching location: ； In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template and the image of the object to be tested; S46: Obtaining the coordinates of the positioning reference points in the local matching area , combined with the center point coordinates of the target area in the template image Coordinates of the reference points in the local matching area , calculate the center point coordinates of the target area in the image of the test piece: 。

Images