Pitch dual-camera visual calibration system and calibration method based on optical center alignment

Abstract

The present invention relates to the field of visual inspection technology, and in particular to a pitch dual-camera visual calibration system and calibration method based on optical center alignment. First, a pitch camera is used to collect and acquire calibration plate images by looking down and looking up at a preset calibration position; then, the optical center offset of the pitch camera is calculated by calculating the center of the calibration plate to align the optical centers of the dual cameras; then, the placement head is moved to obtain the center position of the chip bonding area and the center position of the copper clip bonding area and the distance of the two placement head movements; then, a pitch camera calibration model based on optical center alignment is constructed to establish the connection between the chip bonding area position information and the copper clip bonding area position information. The present invention performs calibration in a non-contact manner, which can effectively improve the calibration accuracy and eliminate the various error effects caused by the copper clip suction nozzle in the traditional dot calibration process. Moreover, the present invention only needs to calibrate the optical center of the pitch camera once, which can improve the calibration efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual detection, and in particular relates to a pitch dual-camera visual calibration system and a calibration method based on optical center alignment. Background Art

[0002] Power semiconductor devices are the core devices for electric energy conversion. The quality and reliability of power semiconductor devices are closely related to the packaging process. At present, in order to reduce the thermal and electrical impedance and bonding damage of power semiconductor devices to a greater extent and meet the increasing process requirements of high voltage, high current, high reliability, etc., the packaging technology of power semiconductor devices has begun to develop from the traditional full-lead bonding packaging or the hybrid bonding packaging of lead and copper clips to the 2D plane and 3D stacked full copper clip bonding packaging. Full copper clip bonding packaging has begun to become the most advanced bonding packaging process for the manufacture of power semiconductor devices today.

[0003] In the full copper clip bonding packaging process, the bonding part is the main failure part of the power semiconductor device because it is mainly subjected to the repeated opening and closing of the current. In order to ensure the stability and reliability of the power semiconductor device after bonding packaging, it is necessary to use the downward camera and the upward camera to respectively obtain the accurate position of the chip bonding area and the copper clip bonding area, and at the same time establish the core correlation model of the two positions, so as to provide the packaging bonding equipment with the mounting movement guidance data. Therefore, designing an efficient and high-precision pitch dual-camera visual correlation calibration method is a key link in realizing advanced bonding packaging.

[0004] At present, the traditional pitch dual camera visual calibration method mainly adopts the dot calibration method, the core of which is to calculate the distance from the optical center of the top-down camera on the placement head of the bonding packaging equipment to the center point of the copper clip nozzle. The traditional dot method first controls the copper clip nozzle to move downward to dot, and then the placement head moves a predetermined distance along the guide rail to make the top-down camera on the placement head recognize the dot center point of the copper clip nozzle, and then calculates the distance between the optical center of the top-down camera and the center point of the copper clip nozzle. Finally, this distance is used as the correlation condition for the pitch dual camera visual positioning, combined with the chip bonding area position information obtained by the top-down camera and the distance information from the copper clip to the copper clip nozzle center point obtained by the top-down camera, and the accurate distance required to move during the placement process is calculated jointly. The calibration process of this dot calibration method is relatively simple, but because it is a contact calibration, the calibration accuracy is relatively low. During calibration, the system will be affected by factors such as the offset error caused by the copper clip nozzle's collision, the center point offset error caused by the copper clip nozzle sucking the copper clip, and the verticality error caused by the copper clip nozzle moving down, resulting in reduced calibration accuracy. At the same time, if multiple copper clip nozzles are installed on the placement head or new copper clip nozzles are replaced, it is necessary to calibrate the distance between the center point of different copper clip nozzles and the optical center of the overhead camera, and the calibration efficiency is relatively low. Summary of the invention

[0005] In view of this, the present invention aims to provide a pitch dual-camera visual calibration system and calibration method based on optical center alignment, so as to solve the problems of low calibration accuracy and low calibration efficiency of the traditional dot calibration method.

[0006] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0007] A pitch dual-camera visual calibration system based on optical center alignment includes a pitch visual image acquisition module, a pitch visual light source illumination module, a pitch camera optical center alignment module, a pitch visual image matching and positioning module, and a network communication module; wherein the pitch visual image acquisition module includes a top view image acquisition submodule and a bottom view image acquisition submodule, the top view image acquisition submodule includes a top view camera, the top view camera is used to respectively shoot a chip to be tested and a calibration board to obtain an image of the chip to be tested and a top view image of the calibration board; the bottom view image acquisition submodule includes an top view camera, the top view camera is used to respectively shoot a copper clip buckle to be tested and a calibration board to obtain an image of the copper clip buckle to be tested and a bottom view image of the calibration board; wherein The calibration plate is set up between the down-looking camera and the upward-looking camera; the pitch vision light source lighting module includes a down-looking light source lighting submodule and an upward-looking light source lighting submodule, the down-looking light source lighting submodule is used to provide lighting for the down-looking camera, and the upward-looking light source lighting submodule is used to provide lighting for the upward-looking camera; the pitch camera optical center alignment module is used to calculate the offset between the optical center of the down-looking camera and the optical center of the upward-looking camera according to the down-looking image of the calibration plate and the upward-looking image of the calibration plate, using the center point of the calibration plate, to obtain the optical center alignment position of the down-looking camera and the upward-looking camera; and record the distance moved from the initial position of the down-looking camera to the optical center alignment position; the pitch vision image matching and positioning module includes a chip image recognition submodule, a copper clip buckle image The chip image recognition submodule is used to identify and locate the center point of the chip bonding area in the image of the chip to be tested, and obtain the offset of the center point of the chip bonding area relative to the optical center of the downward-looking camera; the copper clip image recognition submodule is used to identify and locate the center point of the copper clip end face bonding area in the image of the copper clip to be tested, and obtain the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward-looking camera; the distance recording submodule is used to record the distance moved from the downward-looking camera to the top of the chip to be tested and the distance moved from the copper clip to the top of the upward-looking camera; the pitch camera calibration model construction submodule is used to calculate the distance between the initial position of the downward-looking camera and the distance between the initial position of the downward-looking camera and the distance between the copper clip to be tested and the upward-looking camera; the pitch camera calibration model construction submodule is used to calculate ... The offset between the optical center of the downward camera and the optical center of the upward camera, the distance the downward camera moves from the initial position to the optical center alignment position, the offset of the center point of the chip bonding area relative to the optical center of the downward camera, the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera, the distance the downward camera moves from the initial position to above the chip to be tested and the distance the copper clip to be tested moves to above the upward camera are used to construct a pitch camera calibration model based on optical center alignment, and establish the connection between the chip bonding area position information and the copper clip bonding area position information; the network communication module is used to transmit the chip bonding area position information and the copper clip bonding area position information to the bonding packaging equipment control terminal through the network port.

[0008] Furthermore, the top view image acquisition submodule also includes a first double telecentric lens, which is connected and fixed to the top view camera using a C interface or a CS interface, and the first double telecentric lens and the top view camera are fixed as a whole to one side of the mounting head of the packaging bonding device through a lens bracket, and the top view camera captures images vertically downward; the upward view image acquisition submodule also includes a second double telecentric lens and a steering prism, the second double telecentric lens is connected to the top view camera using a C interface or a CS interface, the second double telecentric lens is installed on the steering prism, and the top view camera, the second double telecentric lens and the steering prism are installed as a whole on a lifting and translation table through a lens base.

[0009] Furthermore, the resolution of the top-down camera is × , pixel size × The first pair of telecentric lenses has a magnification of , the upward camera resolution is × , pixel size × , the second telecentric lens has a magnification of , the size of the calibration area at the center of the calibration plate is × , then the parameters of the downward-looking camera and the first double telecentric lens, and the parameters of the upward-looking camera and the second double telecentric lens satisfy the following formulas:

[0010] ;

[0011] .

[0012] Furthermore, the downward-looking light source lighting submodule includes a point light source and a first light source controller, the point light source is connected to the first light source controller, the first light source controller is connected to the external trigger interface of the downward-looking camera, the point light source is fixed in the light source hole of the first double telecentric lens by a top screw, and the lighting direction is downward; the upward-looking light source lighting submodule includes a ring light source and a second light source controller, the ring light source is connected to the second light source controller, the second light source controller is connected to the external trigger interface of the upward-looking camera, the ring light source is fixed above the lens port of the steering prism, and the lighting direction is upward.

[0013] Furthermore, the calibration plate is a transparent cross calibration plate, a black and white checkerboard calibration plate, or a black and white concentric circle calibration plate.

[0014] Furthermore, the pitch camera optical center alignment module includes a calibration plate top view image coordinate extraction submodule, a calibration plate bottom view image coordinate extraction submodule and an offset calculation submodule; wherein the calibration plate top view image coordinate extraction submodule is used to extract the coordinates of the center point of the calibration plate in the calibration plate top view image. ; The calibration plate bottom view image coordinate extraction submodule is used to extract the center point coordinates of the calibration plate in the calibration plate bottom view image The offset calculation submodule is used to calculate the offset in the X direction between the optical center of the downward-looking camera and the optical center of the upward-looking camera by the following formula: and the offset in the Y direction :

[0015] ;

[0016] Wherein, the X direction is defined as the moving direction of the chip frame tray carrying the chip to be tested, and the Y direction is defined as the moving direction of the placement head, and the X direction is perpendicular to the Y direction.

[0017] Furthermore, the chip image recognition submodule includes a chip detection guidance unit, a chip image matching and positioning unit, and a chip offset calculation unit. The chip detection guidance unit is used to detect whether there is a chip pre-order detection result in the visual calibration system. If so, the chip image to be tested is roughly positioned according to the chip pre-order detection result to obtain a local matching area of ​​the chip to be tested; if not, the chip image to be tested is used as a global matching area; the chip image matching and positioning unit is used to use an improved NCC algorithm to accurately locate the center point of the chip bonding area in the local matching area or the global matching area to obtain a visual positioning detection result; wherein, when the center point of the chip bonding area is accurately located in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection; the chip offset calculation unit is used to calculate the offset of the center point of the chip bonding area relative to the optical center of the overhead camera in the X direction and the offset in the Y direction; the copper clip image recognition submodule includes a copper clip detection guidance unit , a copper clip image matching and positioning unit and a copper clip offset calculation unit, the copper clip detection guidance unit is used to detect whether there is a copper clip pre-order detection result in the visual calibration system, if so, the copper clip image to be tested is roughly positioned according to the copper clip pre-order detection result to obtain the local matching area of ​​the copper clip to be tested; if not, the copper clip image to be tested is used as the global matching area; the copper clip image matching and positioning unit is used to use the improved NCC algorithm to accurately locate the center point of the copper clip end face bonding area in the local matching area or the global matching area to obtain the visual positioning detection result; wherein, when the center point of the copper clip end face bonding area is accurately located in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection; the copper clip offset calculation unit is used to calculate the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera in the X direction and the offset in the Y direction; the pitch camera calibration model constructed by the pitch camera calibration model construction submodule is:

[0018] ;

[0019] In the formula, and are the motion compensation coefficients of the visual calibration system in the X and Y directions, and are the offsets of the center point of the chip bonding area relative to the optical center of the top-view camera in the X direction and the Y direction respectively; and are the offsets of the center point of the bonding area on the copper clip end face relative to the optical center of the upward-looking camera in the X direction and the Y direction respectively; For the top-down camera in visual recognition positioning The distance from the initial position to the top of the chip to be tested when there is a chip to be tested. For the upward-looking camera in visual recognition positioning The distance from the initial position to the top of the copper clip to be tested when the copper clip is tested, S is the distance from the initial position to the optical center alignment position of the top-view camera, is the distance the chip frame tray needs to move, The distance the placement head needs to move.

[0020] A pitch dual-camera visual calibration method implemented by using the above-mentioned pitch dual-camera visual calibration system based on optical center alignment comprises the following steps:

[0021] S1: The placement head drives the downward-looking camera to move above the upward-looking camera;

[0022] S2: Use a lifting bracket to horizontally set up the calibration plate between the downward-looking camera and the upward-looking camera, and adjust the height of the lifting bracket and the height of the upward-looking camera so that the downward-looking camera and the upward-looking camera can capture a clear image of the center point of the calibration plate;

[0023] S3: photographing the calibration plate with a downward-looking camera and an upward-looking camera respectively, obtaining an upward-looking image and an upward-looking image of the calibration plate, extracting the coordinates of the center point of the calibration plate in the downward-looking image and the coordinates of the center point of the calibration plate in the upward-looking image respectively, so as to calculate the offset between the optical center of the downward-looking camera and the optical center of the upward-looking camera, and obtain the optical center alignment position of the downward-looking camera and the upward-looking camera; and recording the distance moved by the downward-looking camera from the initial position to the optical center alignment position;

[0024] S4: The placement head drives the top-view camera to move from the initial position to the top of the chip to be tested, records the moving distance of the top-view camera, uses the improved NCC algorithm to accurately locate the center point of the chip bonding area, obtains the offset of the center point of the chip bonding area relative to the optical center of the top-view camera, and uses the copper clip suction nozzle on the placement head to suck the copper clip to be tested, and drives the copper clip to be tested to move to the top of the upward-view camera, records the moving distance of the upward-view camera, uses the improved NCC algorithm to accurately locate the center point of the copper clip end face bonding area, and obtains the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward-view camera;

[0025] S5: According to the offset between the optical center of the downward camera and the optical center of the upward camera, the distance moved from the initial position of the downward camera to the optical center alignment position, the offset of the center point of the chip bonding area relative to the optical center of the downward camera, the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera, the distance moved from the initial position of the downward camera to above the chip to be tested, and the distance moved from the copper clip to be tested to above the upward camera, a pitch camera calibration model based on optical center alignment is constructed, the connection between the chip bonding area position information and the copper clip bonding area position information is established, and the chip bonding area position information and the copper clip bonding area position information are transmitted to the bonding packaging equipment control terminal.

[0026] Further, in step S3, the coordinates of the center point of the calibration plate extracted from the top view image of the calibration plate are , the coordinates of the center point of the calibration plate extracted from the bottom-up image of the calibration plate are , then the offset in the X direction between the optical center of the downward-looking camera and the optical center of the upward-looking camera is calculated by the following formula and the offset in the Y direction :

[0027] ;

[0028] Wherein, the X direction is defined as the moving direction of the chip frame tray carrying the chip to be tested, and the Y direction is defined as the moving direction of the placement head, and the X direction is perpendicular to the Y direction.

[0029] Furthermore, in step S5, the pitch camera calibration model constructed is:

[0030] ;

[0031] In the formula, and are the motion compensation coefficients of the visual calibration system in the X and Y directions, and are the offsets of the center point of the chip bonding area relative to the optical center of the top-view camera in the X direction and the Y direction respectively; and are the offsets of the center point of the bonding area on the copper clip end face relative to the optical center of the upward-looking camera in the X direction and the Y direction respectively; For the top-down camera in visual recognition positioning The distance from the initial position to the top of the chip to be tested when there is a chip to be tested. For the upward-looking camera in visual recognition positioning The distance from the initial position to the top of the copper clip to be tested, The distance that the top-view camera moves from its initial position to the optical center alignment position.

[0032] Furthermore, in step S4, the center point of the chip bonding area is accurately located using the improved NCC algorithm, and the process of obtaining the offset of the center point of the chip bonding area relative to the optical center of the top-view camera is as follows:

[0033] The first step is to detect whether there is a chip pre-order detection result in the visual calibration system. If so, the image of the chip to be tested is roughly positioned according to the chip pre-order detection result to obtain the local matching area of ​​the chip to be tested; if not, the image of the chip to be tested is used as the global matching area;

[0034] In the second step, the improved NCC algorithm is used to accurately locate the center point of the chip bonding area in the local matching area or the global matching area to obtain the visual positioning detection result; wherein, when accurately locating the center point of the chip bonding area in the global matching area, the visual positioning detection result is used as the previous detection result of the next detection;

[0035] The third step is to calculate the offset of the center point of the chip bonding area relative to the optical center of the top-view camera in the X direction and the Y direction;

[0036] The improved NCC algorithm is used to accurately locate the center point of the copper clip end face bonding area. The process of obtaining the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera is as follows:

[0037] The first step is to detect whether there is a copper clip buckle pre-order detection result in the visual calibration system. If so, the image of the chip to be tested is roughly positioned according to the copper clip buckle pre-order detection result to obtain the local matching area of ​​the copper clip buckle to be tested; if not, the copper clip buckle image to be tested is used as the global matching area;

[0038] In the second step, the improved NCC algorithm is used to accurately locate the center point of the copper clip end face bonding area in the local matching area or the global matching area to obtain the visual positioning detection result; wherein, when accurately locating the center point of the copper clip end face bonding area in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection;

[0039] The third step is to calculate the offset of the center point of the bonding area of ​​the copper clip end face relative to the optical center of the upward-looking camera in the X direction and the Y direction.

[0040] Compared with the prior art, the invention can achieve the following beneficial effects:

[0041] 1. The present invention indirectly obtains the distance between the optical centers of the two cameras when the mounting head is at any position by calibrating the distance between the optical centers of the pitch cameras, and establishes a connection between the chip bonding area position information obtained by the pitch camera visual positioning and the copper clip bonding area position information. At the same time, the present invention performs calibration in a non-contact manner, which can effectively improve the calibration accuracy and eliminate the various errors caused by the copper clip nozzle in the traditional dot calibration process. In addition, the present invention only needs to calibrate the optical center of the pitch camera once, which can improve the calibration efficiency.

[0042] 2. The present invention is in the traditional NCC Based on the algorithm, the previous detection results are combined for quick positioning and auxiliary guidance. When the image of the chip to be tested and the image of the copper clip to be tested are detected, first determine whether the system has a previous detection result. If so, the position of the previous detection result is used as the reference of the local matching area, and the local matching area is based on NCC image matching visual positioning; if there is no previous detection result, the image of the chip to be tested and the image of the copper clip to be tested are globally matched based on NCC The image matching visual positioning is performed, and the visual positioning detection result is used as the previous detection result to provide the previous guidance information in the next detection. The present invention can use the previous detection result to exclude a large amount of invalid image grayscale information, while reducing the amount of calculation and improving the positioning speed, ensuring high positioning accuracy and good algorithm stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings:

[0044] Figure 1 It is a schematic diagram of the logical structure of a pitch dual-camera visual calibration system based on optical center alignment according to an embodiment of the invention;

[0045] Figure 2 It is a schematic diagram of the installation position of the pitch visual image acquisition module according to the embodiment of the invention;

[0046] Figure 3 It is a schematic diagram of the installation position of the pitch vision light source illumination module according to the embodiment of the invention;

[0047] Figure 4 is a schematic diagram of the structure of a transparent cross-marking plate according to an embodiment of the present invention;

[0048] Figure 5 It is a structural schematic diagram of a black and white chessboard calibration plate according to an embodiment of the invention;

[0049] Figure 6 is a schematic diagram of the optical center alignment position of the pitch camera according to an embodiment of the invention;

[0050] Figure 7 It is a flowchart of a pitch dual-camera visual calibration method according to an embodiment of the present invention;

[0051] Figure 8 It is a schematic diagram of the pre-sequence positioning information of the pre-sequence detection result of the copper clip buckle according to the embodiment of the invention;

[0052] Fig. 9 It is a schematic diagram of the local visual positioning of the copper clip buckle image described in the embodiment of the invention.

[0053] The accompanying drawings include: a pitch vision image acquisition module 1, a top view image acquisition submodule 11, an upward view image acquisition submodule 12, a lens bracket 13, a pitch vision light source illumination module 2, a top view light source illumination submodule 21, an upward view light source illumination submodule 22, a lens base 23, a pitch camera optical center alignment module 3, a pitch vision image matching and positioning module 4, a network communication module 5, a mounting head 6, a guide rail 7, a chip frame tray 8, a copper clip nozzle 9, and a calibration plate 10. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0055] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0056] In the description of the present invention, it should be understood that the terms "center point", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0057] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0058] The following will refer to Figure 1-Figure 9 The invention is described in detail with reference to the embodiments.

[0059] like Figure 1-Figure 6 As shown, the pitch dual-camera visual calibration system based on optical center alignment provided by the embodiment of the present invention includes two parts, a front end and a back end. The front end part includes a pitch visual image acquisition module 1 and a pitch visual light source illumination module 2, and the back end part includes a pitch camera optical center alignment module 3, a pitch visual image matching and positioning module 4 and a network communication module 5.

[0060] The pitch vision image acquisition module 1 includes a top view image acquisition submodule 11 and a top view image acquisition submodule 12. The top view image acquisition submodule 11 includes a high-resolution top view camera and a first double telecentric lens. The top view camera and the first double telecentric lens are connected and fixed via a C interface or a CS interface. During installation, the top view image acquisition submodule 11 is fixed to one side of the mounting head 6 via a lens bracket 13, so that the top view camera can vertically downwardly acquire images of the chip to be tested and the calibration board 10, and obtain the image of the chip to be tested and the top view image of the calibration board. The resolution of the top view camera is × , the pixel size is × (μm), the magnification of the first double telecentric lens is .

[0061] The upward image acquisition submodule 12 includes a high-resolution upward camera, a steering prism, and a second dual telecentric lens. The upward camera and the second dual telecentric lens are connected and fixed via a C interface or a CS interface, and the second dual telecentric lens is connected and fixed to the lens hole of the steering prism. During installation, the upward image acquisition submodule 12 is fixed to a small lifting and translation platform via a lens base 23, so that the upward camera can vertically upwardly acquire images of the copper clip to be tested and the calibration plate 10, and obtain the image of the copper clip to be tested and the upward image of the calibration plate. The resolution of the upward camera is × , the pixel size is × (μm), the second telecentric lens has a magnification of .

[0062] Assume that the size of the calibration area at the center of the calibration plate 10 is × , then the parameters of the downward-looking camera and the first double telecentric lens, and the parameters of the upward-looking camera and the second double telecentric lens satisfy the following formulas respectively, so that the fields of view of the downward-looking camera and the upward-looking camera can cover the calibration area at the center of the calibration plate 10, meeting the calibration image acquisition requirements:

[0063]

[0064] The calibration plate 10 required for calibration in the present invention can be a transparent cross calibration plate (such as Figure 4 As shown in Figure 2), you can also use a similar calibration plate with transparent material and obvious feature points, such as a 2×2 black and white checkerboard calibration plate (such as Figure 5 As shown), positioning chessboard corner points, black and white concentric circle calibration plates, etc.

[0065] The mounting head 6 is fixed on the slider of the guide rail 7 and can move horizontally along the guide rail 7. The copper clip suction nozzle 9 is fixed on the mounting head 6 and is used to suck the copper clip to be tested. The chip to be tested is placed on the chip frame tray 8. The moving direction of the chip frame tray 8 is horizontally perpendicular to the moving direction of the mounting head 6 in space. The present invention defines the moving direction of the chip frame tray 8 as the X direction, and the moving direction of the mounting head 6 as the Y direction.

[0066] The pitch vision light source illumination module 2 includes a downward light source illumination submodule 21 and an upward light source illumination submodule 22. The downward light source illumination submodule 21 uses a 5V point light source with adjustable brightness and a first light source controller. The first light source controller is connected to the external trigger interface of the downward camera, and the illumination method uses coaxial light illumination. During installation, the point light source is fixed in the light source hole of the first double telecentric lens by a top screw, and the illumination direction is downward.

[0067] The upward light source illumination submodule 22 uses a 24V ring light source with adjustable brightness and a second light source controller, the ring light source is connected to the second light source controller, the second light source controller is connected to the external trigger interface of the upward camera, and the illumination method adopts bright field illumination. When installed, the ring light source is fixed above the mirror port of the steering prism, and the illumination direction is upward.

[0068] When in use, the downward-looking light source illumination submodule 21 and the upward-looking light source illumination submodule 22 perform illumination in a strobe mode.

[0069] The ring light source and the point light source include but are not limited to red light source, green light source, blue light source and white light source.

[0070] The pitch camera optical center alignment module 3 is used to calculate the offset between the optical center of the downward camera and the optical center of the upward camera according to the downward image and the upward image of the calibration plate, using the center point of the calibration plate, to obtain the optical center alignment position of the downward camera and the upward camera; and when the downward camera and the upward camera are in the following Figure 6 When the optical center is aligned to the position shown, record the distance the top-view camera moves from the initial position to the optical center alignment position.

[0071] The pitch camera optical center alignment module 3 includes a calibration plate top view image coordinate extraction submodule, a calibration plate bottom view image coordinate extraction submodule and an offset calculation submodule; wherein the calibration plate top view image coordinate extraction submodule is used to extract the center point coordinate of the calibration plate in the calibration plate top view image. ; The calibration plate bottom view image coordinate extraction submodule is used to extract the center point coordinates of the calibration plate in the calibration plate bottom view image The offset calculation submodule is used to calculate the offset in the X direction between the optical center of the downward-looking camera and the optical center of the upward-looking camera by the following formula: and the offset in the Y direction :

[0072] .

[0073] The pitch visual image matching and positioning module 4 includes a chip image recognition submodule, a copper clip image recognition submodule, a distance recording submodule and a pitch camera calibration model construction submodule; wherein the chip image recognition submodule is used to identify and locate the center point of the chip bonding area in the image of the chip to be tested, and obtain the offset of the center point of the chip bonding area relative to the optical center of the downward camera; the copper clip image recognition submodule is used to identify and locate the center point of the copper clip end face bonding area in the image of the copper clip to be tested, and obtain the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera; the distance recording submodule is used to record the distance from the initial position of the downward camera to the top of the chip to be tested. The pitch camera calibration model construction submodule is used to construct a pitch camera calibration model based on optical center alignment according to the offset between the optical center of the downward camera and the optical center of the upward camera, the distance the downward camera moves from the initial position to the optical center alignment position, the offset of the center point of the chip bonding area relative to the optical center of the downward camera, the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera, the distance the downward camera moves from the initial position to above the chip to be tested and the distance the copper clip to be tested moves to above the upward camera, so as to establish the connection between the chip bonding area position information and the copper clip bonding area position information.

[0074] The chip image recognition submodule includes a chip detection guidance unit, a chip image matching and positioning unit, and a chip offset calculation unit. The chip detection guidance unit is used to detect whether there is a chip pre-order detection result in the visual calibration system. If so, the chip image to be tested is roughly positioned according to the chip pre-order detection result to obtain a local matching area of ​​the chip to be tested; if not, the chip image to be tested is used as a global matching area; the chip image matching and positioning unit is used to use an improved NCC algorithm to accurately locate the center point of the chip bonding area in the local matching area or the global matching area to obtain a visual positioning detection result; wherein, when the center point of the chip bonding area is accurately located in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection; the chip offset calculation unit is used to calculate the offset of the center point of the chip bonding area relative to the optical center of the overhead camera in the X direction and the offset in the Y direction.

[0075] The copper clip image recognition submodule includes a copper clip detection guidance unit, a copper clip image matching and positioning unit and a copper clip offset calculation unit. The copper clip detection guidance unit is used to detect whether there is a copper clip pre-order detection result in the visual calibration system. If so, the copper clip image to be tested is roughly positioned according to the copper clip pre-order detection result to obtain a local matching area of ​​the copper clip to be tested; if not, the copper clip image to be tested is used as the global matching area; the copper clip image matching and positioning unit is used to use an improved NCC algorithm to accurately locate the center point of the copper clip end face bonding area in the local matching area or the global matching area to obtain a visual positioning detection result; wherein, when the center point of the copper clip end face bonding area is accurately located in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection; the copper clip offset calculation unit is used to calculate the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward-looking camera in the X direction and the offset in the Y direction.

[0076] The copper clip detection guide unit includes a coordinate point calculation unit, which calculates the reference point coordinates of the local matching area of ​​the copper clip to be tested by the following formula: :

[0077] ;

[0078] In the formula, is the coordinate of the visual positioning reference point in the copper clip pre-inspection result, × is the size of the template image, × is the size of the local matching area, , .

[0079] The copper clip image matching and positioning unit includes a template image interception subunit, a template image information calculation subunit, an image local pixel calculation subunit, an image downsampling subunit, an image matching subunit and a target area center point coordinate calculation subunit.

[0080] The template image capture subunit is used to capture the template image containing the target area in the copper clip image to be tested, and obtain the center point coordinates of the target area in the template image. , the length of the template image and width The smaller value of .

[0081] The template image information calculation subunit is used to calculate the grayscale mean of the template image according to the grayscale information of the template image. , the standard deviation of the biased sample of the copper clip image to be tested and the standard deviation of the biased samples of the template image .

[0082] Grayscale mean calculation formula of template image:

[0083] ;

[0084] in, is the position in the grayscale image The pixel value of is the width of the grayscale image, is the height of the grayscale image.

[0085] The calculation formula of the biased sample standard deviation of the template image is:

[0086] .

[0087] Grayscale mean calculation formula of the image to be tested:

[0088] .

[0089] The calculation formula of the biased sample standard deviation of the image of the test piece is:

[0090] ;

[0091] in, is the position in the image to be tested The pixel value of is the width of the image of the test piece, is the height of the image of the object under test.

[0092] In actual calculation, the standard library numpy in Python language is used to calculate the grayscale mean of the template image by calling the formula in the library , the standard deviation of the biased sample of the test piece image , the grayscale mean of the image to be tested and the biased sample standard deviation of the template image .

[0093] The image local pixel sum calculation subunit is used to calculate the image local pixel sum of the search window by using the image integral map method through the following formula: :

[0094] ;

[0095] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the copper clip to be tested, It is the coordinate of the upper left corner of the search window in the integral image of the copper clip image to be tested.

[0096] The image downsampling unit is used to downsample the template image and the copper clip image to be tested using the image pyramid method. L-1 Downsampling, L is the number of layers of the image pyramid, which is calculated by the following formula:

[0097] ;

[0098] In the formula, is the minimum side length of the template image after multiple downsampling.

[0099] The image matching subunit is used to match and locate the downsampled template image with the copper clip image to be tested layer by layer, and calculate the normalized correlation coefficient of each layer by the following formula: , each layer The position of the search window at the maximum value is taken as the optimal matching location:

[0100] ;

[0101] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template and the copper clip image to be tested.

[0102] The target area center point coordinate calculation subunit is used to obtain the positioning reference point coordinates of the local matching area , combined with the center point coordinates of the target area in the template image Coordinates of the reference points in the local matching area , calculate the center point coordinates of the target area in the copper clip image to be tested .

[0103] Coordinates of the center point of the bonding area on the upper end face of the copper clip in the copper clip image The calculation formula is:

[0104] .

[0105] The internal structure of the chip image matching and positioning unit refers to the copper clip image matching and positioning unit. The principles of the two are the same, so they will not be repeated here.

[0106] The pitch camera calibration model constructed by the pitch camera calibration model construction submodule is:

[0107] ;

[0108] In the formula, and are the motion compensation coefficients of the visual calibration system in the X and Y directions, and are the offsets of the center point of the chip bonding area relative to the optical center of the top-view camera in the X direction and the Y direction respectively; and are the offsets of the center point of the bonding area on the copper clip end face relative to the optical center of the upward-looking camera in the X direction and the Y direction respectively; For the top-down camera in visual recognition positioning The distance from the initial position to the top of the chip to be tested when there is a chip to be tested. For the upward-looking camera in visual recognition positioning The distance from the initial position to the top of the copper clip to be tested when the copper clip is tested, S is the distance from the initial position to the optical center alignment position of the top-view camera, is the distance the chip frame tray needs to move, The distance the placement head needs to move.

[0109] The network communication module 5 realizes the network communication between the visual recognition and positioning system and the bonding and packaging equipment control terminal based on the socket, and is used to transmit the chip bonding area position information and the copper clip bonding area position information to the bonding and packaging equipment control terminal through the network port.

[0110] The hardware of the pitch camera optical center alignment module 3, the pitch visual image matching and positioning module 4, and the network communication module 5 are implemented based on an industrial computer, and the software platform is based on Linguistic Software and install the corresponding version Library package.

[0111] The above content describes in detail the structure of the pitch dual-camera visual calibration system based on optical center alignment provided by the embodiment of the present invention. Corresponding to the visual calibration system, the embodiment of the present invention also provides a pitch dual-camera visual calibration method implemented using the visual calibration system.

[0112] like Figure 7 As shown, the pitch dual-camera visual calibration method described in the embodiment of the present invention includes the following steps:

[0113] S1: The placement head drives the downward-looking camera to move above the upward-looking camera.

[0114] Drive the placement head to move along the guide rail so that the downward-looking camera is above the upward-looking camera.

[0115] S2: Use a lifting bracket to horizontally set up the calibration plate between the downward-looking camera and the upward-looking camera. By adjusting the height of the lifting bracket and the height of the upward-looking camera through the lifting and translation stage, the downward-looking camera and the upward-looking camera can capture a clear image of the center point of the calibration plate.

[0116] S3: photograph the calibration plate with a downward-looking camera and an upward-looking camera respectively to obtain an upward-looking image and an upward-looking image of the calibration plate, extract the coordinates of the center point of the calibration plate in the downward-looking image and the coordinates of the center point of the calibration plate in the upward-looking image respectively, so as to calculate the offset between the optical center of the downward-looking camera and the optical center of the upward-looking camera, obtain the optical center alignment position of the downward-looking camera and the upward-looking camera; and record the distance that the downward-looking camera moves from the initial position to the optical center alignment position.

[0117] In step S3, the coordinates of the center point of the calibration plate extracted from the top view image of the calibration plate are , the coordinates of the center point of the calibration plate extracted from the bottom-up image of the calibration plate are , then the offset in the X direction between the optical center of the downward-looking camera and the optical center of the upward-looking camera is calculated by the following formula and the offset in the Y direction :

[0118] .

[0119] When the calibration plate adopts a transparent cross calibration plate, the straight line detection algorithm is used to extract the center point coordinates of the calibration plate extracted from the top view image of the calibration plate and the center point coordinates of the calibration plate extracted from the bottom view image of the calibration plate; when the calibration plate adopts a 2×2 black and white chessboard calibration plate, the corner detection algorithm is used to extract the center point coordinates of the calibration plate extracted from the top view image of the calibration plate and the center point coordinates of the calibration plate extracted from the bottom view image of the calibration plate; when the calibration plate adopts a black and white concentric circle calibration plate, the center point coordinates of the calibration plate extracted from the top view image of the calibration plate and the center point coordinates of the calibration plate extracted from the bottom view image of the calibration plate are extracted by locating the center of the concentric circles.

[0120] S4: The placement head drives the top-view camera to move from the initial position to the top of the chip to be tested, records the moving distance of the top-view camera, uses the improved NCC algorithm to accurately locate the center point of the chip bonding area, obtains the offset of the center point of the chip bonding area relative to the optical center of the top-view camera, and uses the copper clip suction nozzle on the placement head to suck the copper clip to be tested, and drives the copper clip to be tested to move to the top of the upward-view camera, records the moving distance of the upward-view camera, uses the improved NCC algorithm to accurately locate the center point of the copper clip end face bonding area, and obtains the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward-view camera.

[0121] The improved NCC algorithm is used to accurately locate the center point of the copper clip end face bonding area. The process of obtaining the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera is as follows:

[0122] The first step is to detect whether there is a copper clip pre-detection result in the visual calibration system. If so, the image of the chip to be tested is roughly positioned according to the copper clip pre-detection result to obtain the local matching area of ​​the copper clip to be tested; if not, the image of the copper clip to be tested is used as the global matching area.

[0123] When the visual calibration system has a copper clip pre-order detection result, such as Figure 8 As shown, the coordinates of the visual positioning reference point in the copper clip pre-inspection result are , the size of the template image is × , the size of the local matching area is × , , , then the reference point of the local matching area is calculated by the following formula Coordinates :

[0124] .

[0125] When the visual calibration system does not have a previous detection result of the copper clip buckle, the global copper clip buckle image is used as the matching area.

[0126] In the second step, the improved NCC algorithm is used to accurately locate the center point of the copper clip end face bonding area in the local matching area or the global matching area to obtain the visual positioning detection result; among which, when accurately locating the center point of the copper clip end face bonding area in the global matching area, the visual positioning detection result is used as the predecessor detection result of the next detection.

[0127] The present invention realizes the rough positioning of the copper clip buckle image to be tested through the preamble guidance information provided by the preamble detection result of the copper clip buckle, and obtains the local matching area of ​​the bonding area of ​​the upper end face of the copper clip buckle in the copper clip buckle image to be tested. In order to further improve the positioning accuracy of the bonding area of ​​the upper end face of the copper clip buckle, reduce the influence of image noise, image illumination and linear change of image grayscale, and improve the positioning speed and stability of the positioning method, the present invention designs a method based on an improved NCC algorithm to quickly and accurately locate the target area in the local matching area to obtain a visual positioning detection result. The present invention uses the calculated center point coordinates of the bonding area of ​​the upper end face of the copper clip buckle as the visual positioning detection result.

[0128] like Fig. 9 As shown, the specific steps of quickly and accurately locating the target area in the local matching area based on the improved NCC algorithm are as follows:

[0129] S41: intercepting a template image including the upper end of the copper clip buckle in the copper clip buckle image, and calculating the center point of the target area in the template image Coordinates , the length of the template image and width The smaller value of .

[0130] In the present invention, the center point of the target area in the template image Coordinates When constructing a template image, the operator selects or directly sets the actual target center point on the template image based on the GUI interface.

[0131] S42: Calculate the grayscale mean of the template image according to the grayscale information of the template image , the standard deviation of the biased sample of the copper clip image to be tested and the standard deviation of the biased samples of the template image .

[0132] The grayscale mean calculation formula of the template image is:

[0133] ;

[0134] in, is the position in the grayscale image The pixel value of is the width of the grayscale image, is the height of the grayscale image.

[0135] The formula for calculating the biased sample standard deviation of the template image is:

[0136] .

[0137] The grayscale mean calculation formula of the image to be tested is:

[0138] .

[0139] The formula for calculating the biased sample standard deviation of the image of the test piece is:

[0140] .

[0141] in, is the position in the image to be tested The pixel value of is the width of the image of the test piece, is the height of the image of the object under test.

[0142] In actual calculation, the standard library numpy in Python language is used to calculate the grayscale mean of the template image by calling the formula in the library , the standard deviation of the biased sample of the test piece image , the grayscale mean of the image to be tested and the standard deviation of the biased samples of the template image .

[0143] S43: Using the image integral graph method, the local pixel sum of the image in the search window is calculated by the following formula :

[0144] ;

[0145] In the formula, S is a search window of the same size as the template image, is the coordinate of the lower right corner of the search window in the integral image of the copper clip to be tested, It is the coordinate of the upper left corner of the search window in the integral image of the copper clip image to be tested.

[0146] To improve NCC The matching positioning speed of the algorithm is improved. In the process of calculating the normalized mutual correlation coefficient between the copper clip image and the template image, the present invention uses the integral graph method to quickly calculate the local pixel sum of the image.

[0147] S44: Using the image pyramid method to compare the template image and the copper clip image to be tested L-1 Downsampling, L is the number of layers of the image pyramid, which is calculated by the following formula:

[0148] ;

[0149] In the formula, is the minimum side length of the template image after multiple downsampling.

[0150] The present invention adopts the image pyramid method to downsample the copper clip image and the template image to further improve the matching and positioning speed. Suppose the minimum side length of the template image after multiple downsampling is ( ), use the above formula to calculate the number of layers of the image pyramid L , and according to the number of layers L The template image and the copper clip image are Downsampling.

[0151] S45: Assume that the local matching area is the bottom layer of the image pyramid, and match and locate the downsampled template image and the copper clip image layer by layer in the order from the top layer to the bottom layer of the image pyramid, and calculate the normalized correlation coefficient of each layer by the following formula , each layer The position of the search window at the maximum value is taken as the optimal matching location:

[0152] ;

[0153] In the formula, i is the number of matching layers, For the i Grayscale convolution of the layer template and the copper clip image.

[0154] The present invention uses the image where the local matching area is located as the bottom layer of the image pyramid, that is, L Layer. According to the order of the image pyramid from the top layer to the bottom layer, the downsampled template image is matched with the copper clip image layer by layer, and the normalized correlation coefficient of each layer in the image pyramid is calculated. , with each layer When the maximum value is reached, the search window position is the optimal matching position. When matching and positioning the first layer, select the copper clip image of this layer for global matching. When matching and positioning other layers, use the positioning position of the previous layer, multiply the matching positioning reference coordinates by 2, and then use the set In the interval, a small neighborhood matching is performed in the lower layer copper clip image, and the normalized mutual correlation coefficient is calculated using the above formula when positioning each layer. .

[0155] S46: Obtaining the positioning reference points of the local matching area Coordinates , combined with the center point of the target area in the template image Coordinates The reference points of the local matching area Coordinates , calculate the center point of the bonding area of ​​the upper end face of the copper clip in the copper clip image Coordinates .

[0156] The center point of the bonding area on the upper end face of the copper clip in the copper clip image Coordinates The calculation formula is:

[0157] .

[0158] The present invention is based on the previous guidance and improved NCC The algorithm realizes the visual positioning of the bonding area of ​​the upper end face of the copper clip. First, the copper clip image is roughly positioned through the previous detection results to obtain the local matching area, and then precise positioning is performed in the local matching area to obtain the coordinates of the center point of the bonding area of ​​the upper end face of the copper clip. While ensuring the positioning accuracy, it solves the traditional NCC The algorithm solves the problem of slow positioning speed during matching and positioning, thus improving the overall detection efficiency of the system.

[0159] The present invention is less sensitive to image noise and less affected by light. When the copper clip is affected by the adsorption of the mounting head, resulting in poor flatness of the copper clip and darker image, the present invention is less affected by linear changes in image grayscale.

[0160] It should be noted that, depending on the chips to be packaged and bonded, it is necessary to design corresponding special-shaped copper clips of different shapes. The embodiment of the present invention is a special-shaped copper clip with a protruding upper end stamped and bent, and its protruding part bridges the chip gate. Therefore, if other parts of the designed special-shaped copper clip have similar obvious features, other feature areas can also be selected as template images for matching and positioning, and then the positioning results are combined with the copper clip design size to recalculate the coordinates of the center point of the copper clip bonding area.

[0161] The third step is to calculate the offset of the center point of the bonding area of ​​the copper clip end face relative to the optical center of the upward-looking camera in the X direction and the Y direction.

[0162] The process of accurately locating the center point of the chip bonding area using the improved NCC algorithm refers to the process of accurately locating the center point of the copper clip end face bonding area using the improved NCC algorithm. The principles of the two are the same, so they will not be repeated here.

[0163] S5: According to the offset between the optical center of the downward camera and the optical center of the upward camera, the distance moved from the initial position of the downward camera to the optical center alignment position, the offset of the center point of the chip bonding area relative to the optical center of the downward camera, the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera, the distance moved from the initial position of the downward camera to above the chip to be tested, and the distance moved from the copper clip to be tested to above the upward camera, a pitch camera calibration model based on optical center alignment is constructed, the connection between the chip bonding area position information and the copper clip bonding area position information is established, and the chip bonding area position information and the copper clip bonding area position information are transmitted to the bonding packaging equipment control terminal.

[0164] The constructed pitch camera calibration model is:

[0165] ;

[0166] In the formula, and are the motion compensation coefficients of the visual calibration system in the X and Y directions, and are the offsets of the center point of the chip bonding area relative to the optical center of the top-view camera in the X direction and the Y direction respectively; and are the offsets of the center point of the bonding area on the copper clip end face relative to the optical center of the upward-looking camera in the X direction and the Y direction respectively; For the top-down camera in visual recognition positioning The distance from the initial position to the top of the chip to be tested when there is a chip to be tested. For the upward-looking camera in visual recognition positioning The distance from the initial position to the top of the copper clip to be tested, The distance that the top-view camera moves from its initial position to the optical center alignment position.

[0167] During the mounting process, after the downward-looking camera visually locates and identifies the chip to be tested and the upward-looking camera visually locates and identifies the copper clip end face to be tested, in order to achieve bonding packaging, the chip frame tray needs to move a distance of , in the Y direction, the distance the placement head needs to move along the guide rail is .

[0168] The calibration method of the present invention is a non-contact calibration method. Compared with the traditional contact-type dot calibration method, since the present invention eliminates the various error effects caused by the contact of the copper clip nozzle during calibration, it can effectively improve the calibration accuracy. In addition, the core of the traditional contact-type dot calibration method is to establish the connection between the visual detection information of the overhead camera and the upward camera based on the position of the center of the copper clip nozzle. Therefore, it is necessary to calibrate each copper clip nozzle once. If the copper clip nozzle is replaced due to wear or collision, the equipment needs to be recalibrated, and the calibration efficiency is low. The present invention establishes the connection between the visual detection information of the overhead camera and the upward camera through the distance between the optical center of the overhead camera and the optical center of the upward camera. The calibration is only related to the pitch camera, so it only needs to be calibrated once, and the calibration efficiency is high.

[0169] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0170] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A pitch dual-camera visual calibration system based on optical center alignment, characterized in that: It includes a pitch vision image acquisition module, a pitch vision light source illumination module, a pitch camera optical center alignment module, a pitch vision image matching and positioning module and a network communication module; wherein, The pitch visual image acquisition module includes a top view image acquisition submodule and a bottom view image acquisition submodule. The top view image acquisition submodule includes a top view camera, which is used to photograph the chip to be tested and the calibration board respectively, to obtain the image of the chip to be tested and the top view image of the calibration board; the bottom view image acquisition submodule includes an top view camera, which is used to photograph the copper clip to be tested and the calibration board respectively, to obtain the image of the copper clip to be tested and the bottom view image of the calibration board; wherein the calibration board is set between the top view camera and the bottom view camera; The pitch vision light source lighting module includes a downward light source lighting submodule and an upward light source lighting submodule. The downward light source lighting submodule is used to provide lighting for the downward camera, and the upward light source lighting submodule is used to provide lighting for the upward camera. The pitch camera optical center alignment module is used to calculate the offset between the optical center of the downward camera and the optical center of the upward camera according to the downward image and the upward image of the calibration plate, using the center point of the calibration plate, to obtain the optical center alignment position of the downward camera and the upward camera; and to record the distance moved by the downward camera from the initial position to the optical center alignment position; The pitch vision image matching and positioning module includes a chip image recognition submodule, a copper clip image recognition submodule, a distance recording submodule and a pitch camera calibration model construction submodule; wherein the chip image recognition submodule is used to identify and locate the center point of the chip bonding area in the image of the chip to be tested, and obtain the offset of the center point of the chip bonding area relative to the optical center of the downward camera; the copper clip image recognition submodule is used to identify and locate the center point of the copper clip end face bonding area in the image of the copper clip to be tested, and obtain the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera; the distance recording submodule is used to record the distance from the initial position of the downward camera to the top of the chip to be tested. The pitch camera calibration model construction submodule is used to construct a pitch camera calibration model based on optical center alignment according to the offset between the optical center of the downward camera and the optical center of the upward camera, the distance from the downward camera to the optical center alignment position, the offset of the center point of the chip bonding area relative to the optical center of the downward camera, the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera, the distance from the downward camera to the chip to be tested and the distance from the copper clip to the upward camera, and establish the connection between the chip bonding area position information and the copper clip bonding area position information; The network communication module is used to transmit the chip bonding area position information and the copper clip bonding area position information to the bonding packaging equipment control terminal through the network port.

2. The pitch dual-camera visual calibration system based on optical center alignment according to claim 1, characterized in that: The top-view image acquisition submodule also includes a first double telecentric lens, which is connected and fixed to the top-view camera using a C interface or a CS interface. The first double telecentric lens and the top-view camera are fixed as a whole to one side of the mounting head of the packaging bonding device through a lens bracket, and the top-view camera vertically captures images downward; The upward-looking image acquisition submodule also includes a second double telecentric lens and a steering prism. The second double telecentric lens is connected to the upward-looking camera using a C interface or a CS interface. The second double telecentric lens is installed on the steering prism. The upward-looking camera, the second double telecentric lens and the steering prism are installed as a whole on a lifting and translation platform through a lens base.

3. The pitch dual-camera visual calibration system based on optical center alignment according to claim 2, characterized in that: The resolution of the top-down camera is × , pixel size × The first pair of telecentric lenses has a magnification of , the upward camera resolution is × , pixel size × , the second telecentric lens has a magnification of , the size of the calibration area at the center of the calibration plate is × , then the parameters of the downward-looking camera and the first double telecentric lens, and the parameters of the upward-looking camera and the second double telecentric lens satisfy the following formulas: ； 。 4. The pitch dual-camera visual calibration system based on optical center alignment according to claim 2 or 3, characterized in that: The overhead light source illumination submodule includes a point light source and a first light source controller, the point light source is connected to the first light source controller, the first light source controller is connected to the external trigger interface of the overhead camera, the point light source is fixed in the light source hole of the first double telecentric lens by a top screw, and the illumination direction is downward; The upward-looking light source illumination submodule includes an annular light source and a second light source controller. The annular light source is connected to the second light source controller. The second light source controller is connected to the external trigger interface of the upward-looking camera. The annular light source is fixed above the lens opening of the steering prism, and the illumination direction is upward.

5. The pitch dual-camera visual calibration system based on optical center alignment according to claim 1, characterized in that: The calibration plate is a transparent cross calibration plate, a black and white checkerboard calibration plate, or a black and white concentric circle calibration plate.

6. The pitch dual-camera visual calibration system based on optical center alignment according to claim 2, characterized in that: The pitch camera optical center alignment module includes a calibration plate top view image coordinate extraction submodule, a calibration plate bottom view image coordinate extraction submodule and an offset calculation submodule; wherein, The calibration plate top view image coordinate extraction submodule is used to extract the center point coordinates of the calibration plate in the calibration plate top view image. ; The calibration plate bottom view image coordinate extraction submodule is used to extract the coordinates of the center point of the calibration plate in the calibration plate bottom view image. ; The offset calculation submodule is used to calculate the offset in the X direction between the optical center of the downward-looking camera and the optical center of the upward-looking camera by the following formula: and the offset in the Y direction : ； Wherein, the X direction is defined as the moving direction of the chip frame tray carrying the chip to be tested, and the Y direction is defined as the moving direction of the placement head, and the X direction is perpendicular to the Y direction.

7. The pitch dual-camera visual calibration system based on optical center alignment according to claim 6, characterized in that: The chip image recognition submodule includes a chip detection guidance unit, a chip image matching and positioning unit, and a chip offset calculation unit. The chip detection guidance unit is used to detect whether there is a chip pre-order detection result in the visual calibration system. If so, the image of the chip to be tested is roughly positioned according to the chip pre-order detection result to obtain a local matching area of ​​the chip to be tested. If it does not exist, the image of the chip to be tested is used as the global matching area; The chip image matching and positioning unit is used to use the improved NCC algorithm to accurately locate the center point of the chip bonding area in the local matching area or the global matching area to obtain the visual positioning detection result; wherein, when accurately locating the center point of the chip bonding area in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection; the chip offset calculation unit is used to calculate the offset of the center point of the chip bonding area relative to the optical center of the overhead camera in the X direction and the offset in the Y direction; The copper clip image recognition submodule includes a copper clip detection guidance unit, a copper clip image matching and positioning unit, and a copper clip offset calculation unit. The copper clip detection guidance unit is used to detect whether there is a copper clip pre-order detection result in the visual calibration system. If so, the copper clip image to be tested is roughly positioned according to the copper clip pre-order detection result to obtain a local matching area of ​​the copper clip to be tested; if not, the copper clip image to be tested is used as a global matching area; the copper clip image matching and positioning unit is used to use an improved NCC algorithm to accurately locate the center point of the copper clip end face bonding area in the local matching area or the global matching area to obtain a visual positioning detection result; wherein, when the center point of the copper clip end face bonding area is accurately located in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection; the copper clip offset calculation unit is used to calculate the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward-looking camera in the X direction and the offset in the Y direction; The pitch camera calibration model constructed by the pitch camera calibration model construction submodule is: ； In the formula, and are the motion compensation coefficients of the visual calibration system in the X and Y directions, and are the offsets of the center point of the chip bonding area relative to the optical center of the top-view camera in the X direction and the Y direction respectively; and are the offsets of the center point of the bonding area on the copper clip end face relative to the optical center of the upward-looking camera in the X direction and the Y direction respectively; For the top-down camera in visual recognition positioning The distance from the initial position to the top of the chip to be tested when there is a chip to be tested. For the upward-looking camera in visual recognition positioning The distance from the initial position to the top of the copper clip to be tested when the copper clip is tested, S is the distance from the initial position to the optical center alignment position of the top-view camera, is the distance the chip frame tray needs to move, The distance the placement head needs to move.

8. A pitch dual-camera visual calibration method implemented by using the pitch dual-camera visual calibration system based on optical center alignment according to any one of claims 1 to 7, characterized in that: The steps include: S1: The placement head drives the downward-looking camera to move above the upward-looking camera; S2: Use a lifting bracket to horizontally set up the calibration plate between the downward-looking camera and the upward-looking camera, and adjust the height of the lifting bracket and the height of the upward-looking camera so that the downward-looking camera and the upward-looking camera can capture a clear image of the center point of the calibration plate; S3: photographing the calibration plate with a downward-looking camera and an upward-looking camera respectively, obtaining an upward-looking image and an upward-looking image of the calibration plate, extracting the coordinates of the center point of the calibration plate in the downward-looking image and the coordinates of the center point of the calibration plate in the upward-looking image respectively, so as to calculate the offset between the optical center of the downward-looking camera and the optical center of the upward-looking camera, and obtain the optical center alignment position of the downward-looking camera and the upward-looking camera; and recording the distance moved by the downward-looking camera from the initial position to the optical center alignment position; S4: The placement head drives the top-view camera to move from the initial position to the top of the chip to be tested, records the moving distance of the top-view camera, uses the improved NCC algorithm to accurately locate the center point of the chip bonding area, obtains the offset of the center point of the chip bonding area relative to the optical center of the top-view camera, and uses the copper clip suction nozzle on the placement head to suck the copper clip to be tested, and drives the copper clip to be tested to move to the top of the upward-view camera, records the moving distance of the upward-view camera, uses the improved NCC algorithm to accurately locate the center point of the copper clip end face bonding area, and obtains the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward-view camera; S5: According to the offset between the optical center of the downward camera and the optical center of the upward camera, the distance moved from the initial position of the downward camera to the optical center alignment position, the offset of the center point of the chip bonding area relative to the optical center of the downward camera, the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera, the distance moved from the initial position of the downward camera to above the chip to be tested, and the distance moved from the copper clip to be tested to above the upward camera, a pitch camera calibration model based on optical center alignment is constructed, the connection between the chip bonding area position information and the copper clip bonding area position information is established, and the chip bonding area position information and the copper clip bonding area position information are transmitted to the bonding packaging equipment control terminal.

9. The pitch dual camera visual calibration method according to claim 8, characterized in that: In step S3, the coordinates of the center point of the calibration plate extracted from the top view image of the calibration plate are , the coordinates of the center point of the calibration plate extracted from the bottom-up image of the calibration plate are , then the offset in the X direction between the optical center of the downward-looking camera and the optical center of the upward-looking camera is calculated by the following formula and the offset in the Y direction : ； Wherein, the X direction is defined as the moving direction of the chip frame tray carrying the chip to be tested, and the Y direction is defined as the moving direction of the placement head, and the X direction is perpendicular to the Y direction.

10. The pitch dual camera visual calibration method according to claim 9, characterized in that: In step S5, the pitch camera calibration model constructed is: ； In the formula, and are the motion compensation coefficients of the visual calibration system in the X and Y directions, and are the offsets of the center point of the chip bonding area relative to the optical center of the top-view camera in the X direction and the Y direction respectively; and are the offsets of the center point of the bonding area on the copper clip end face relative to the optical center of the upward-looking camera in the X direction and the Y direction respectively; For the top-down camera in visual recognition positioning The distance from the initial position to the top of the chip to be tested when there is a chip to be tested. For the upward-looking camera in visual recognition positioning The distance from the initial position to the top of the copper clip to be tested, The distance that the top-view camera moves from its initial position to the optical center alignment position.

11. The pitch dual camera visual calibration method according to claim 9, characterized in that: In step S4, the improved NCC algorithm is used to accurately locate the center point of the chip bonding area, and the process of obtaining the offset of the center point of the chip bonding area relative to the optical center of the top-view camera is as follows: The first step is to detect whether there is a chip pre-order detection result in the visual calibration system. If so, the image of the chip to be tested is roughly located according to the chip pre-order detection result to obtain a local matching area of ​​the chip to be tested; If it does not exist, the image of the chip to be tested is used as the global matching area; In the second step, the improved NCC algorithm is used to accurately locate the center point of the chip bonding area in the local matching area or the global matching area to obtain the visual positioning detection result; wherein, when accurately locating the center point of the chip bonding area in the global matching area, the visual positioning detection result is used as the previous detection result of the next detection; The third step is to calculate the offset of the center point of the chip bonding area relative to the optical center of the top-view camera in the X direction and the Y direction; The improved NCC algorithm is used to accurately locate the center point of the copper clip end face bonding area. The process of obtaining the offset of the center point of the copper clip end face bonding area relative to the optical center of the upward camera is as follows: The first step is to detect whether there is a copper clip buckle pre-order detection result in the visual calibration system. If so, the image of the chip to be tested is roughly positioned according to the copper clip buckle pre-order detection result to obtain the local matching area of ​​the copper clip buckle to be tested; if not, the copper clip buckle image to be tested is used as the global matching area; In the second step, the improved NCC algorithm is used to accurately locate the center point of the copper clip end face bonding area in the local matching area or the global matching area to obtain the visual positioning detection result; wherein, when accurately locating the center point of the copper clip end face bonding area in the global matching area, the visual positioning detection result is used as the pre-order detection result of the next detection; The third step is to calculate the offset of the center point of the bonding area of ​​the copper clip end face relative to the optical center of the upward-looking camera in the X direction and the Y direction.

Images