Lead bonding machine welding area Z-direction positioning system and force position switching method

Abstract

The invention discloses a Z-direction positioning system for a welding area of a lead bonding machine and a force position switching method. The system comprises a point cloud processing module and a visual module, the visual module is used for positioning and extracting welding spots in a welding area through image shooting; and the point cloud processing module obtains point cloud data of the welding area through visual identification and carries out fusion processing on the obtained point cloud data and the welding spot position data extracted by the visual module so as to calculate Z-direction position information of each welding spot in the welding area. The method performs force position switching control according to a calculation result of the system. The beneficial effects are that compared with traditional two-dimensional positioning data, the three-dimensional point cloud processing module is combined with the traditional two-dimensional positioning data to realize target identification and positioning of the chip and the substrate, and the precision and accuracy are higher. By measuring the heights of the chip and the substrate and using the position deviation as a force position switching condition, more stable force position switching operation can be realized, and the precision and consistency of lead bonding are improved.

Description

Technical Field

[0001] The present application relates to the technical field of bonding machines, and in particular to a Z-axis positioning system for a welding area of ​​a wire bonding machine and a force position switching method. Background Art

[0002] During the bonding process of the wire bonder, it is critical to ensure the correct positioning of the solder joints of the chip and the substrate. Current wire bonders usually use 2D vision cameras for target recognition and positioning.

[0003] The traditional method of Z-axis control of wire bonders mainly includes two stages: the high-speed point control stage and the slower, controlled speed search stage when approaching the work surface. During Z-axis control, since there is no force sensor, the bonding device usually switches from position control to force control through switching strategies such as speed change and position change. The position and speed information are obtained through the grating ruler, both of which are closely related to the position information of the solder joints of the chip and substrate in the Z direction.

[0004] Normally, due to differences in processing and assembly processes, the bonding points of the chip and substrate to be welded will have micron-level height changes, so it is not possible to use unified height information to form position errors for force-position switching. Therefore, in order to avoid excessive impact and damage to the chip, or insufficient impact leading to poor welding quality, the wire bonding machine must be able to detect Z-direction information at each bonding point. However, existing wire bonding machines are unable to obtain the positioning and height information of the chip and substrate solder joints in the Z direction, and therefore cannot use the Z-direction position deviation for switching in force-position switching control. Summary of the invention

[0005] One of the objectives of the present application is to provide a wire bonding machine welding area Z-axis positioning system that can solve at least one of the defects in the above-mentioned background technology.

[0006] Another object of the present application is to provide a wire bonding machine force position switching method that can solve at least one of the defects in the above-mentioned background technology.

[0007] In order to achieve at least one of the above-mentioned purposes, the technical solution adopted in the present application is: a Z-axis positioning system for a welding area of ​​a wire bonding machine, comprising a point cloud processing module and a vision module; the vision module locates and extracts the solder joints in the welding area by image capture; the point cloud processing module obtains the point cloud data of the welding area by visual recognition, and fuses the obtained point cloud data with the solder joint position data extracted by the vision module to calculate the Z-axis position information of each solder joint in the welding area.

[0008] Preferably, the point cloud processing module includes a point cloud acquisition unit and a data processing unit; the point cloud acquisition unit is arranged above the substrate for placing the chip, and the point cloud acquisition unit performs image acquisition on the welding area based on 3D visual recognition to obtain the point cloud data; the data processing unit is suitable for receiving the point cloud data and fusing it with the solder point position data to obtain fused point cloud data corresponding only to each solder point in the welding area; the fused point cloud data is processed by an algorithm to calculate the Z-axis position information of the chip and each solder point on the substrate.

[0009] Preferably, the point cloud acquisition unit adopts an RGB-D camera, and the RGB-D camera includes a depth module and an RGB module; the depth module is used to obtain a point cloud with depth information, and the RGB module is used to obtain an original color image; and then the data of the depth module and the RGB module are matched to obtain the point cloud data with color.

[0010] Preferably, the point cloud acquisition unit adopts a three-dimensional scanner, and the process of the three-dimensional scanner acquiring the point cloud data is as follows: the three-dimensional scanner emits pulses to the area where the chip and the substrate are located, and calculates the distance from the substrate and the chip to the three-dimensional scanner based on the reflection time difference of the pulse; a coordinate point set is obtained after coordinate calculation based on the distance measured by the three-dimensional scanner at multiple deflection angles; according to the platform angle for installing the three-dimensional scanner, the coordinate point set is converted into a polar coordinate point set, and then the point cloud data is obtained through coordinate conversion; or, the three-dimensional scanner acquires the point cloud data by surface scanning.

[0011] Preferably, the visual module includes a camera unit, and the point cloud acquisition unit is installed side by side with the camera unit used to acquire images of the chip and the substrate; the point cloud acquisition unit is suitable for calibration after installation, and the camera unit adopts a CCD camera or a CMOS camera.

[0012] Preferably, the point cloud data and the weld point position data are both denoised after being acquired; based on the denoised point cloud data and the weld point position data, data fusion is performed after coordinates are uniformly transformed into a world coordinate system to obtain the fused point cloud data of each weld point in the welding area determined by the weld point position data with the corresponding point cloud.

[0013] Preferably, the processing process of the fused point cloud data by the data processing unit includes point cloud segmentation and point cloud fitting; key features are extracted from the filtered fused point cloud data through the point cloud segmentation to obtain a point cloud data set in which the point cloud is clustered; the point cloud data set is fitted through the point cloud fitting to obtain two fitting planes corresponding to the chip and the substrate respectively; based on the distance from any fitting point on one of the fitting planes to the other fitting plane, the Z-direction position information of the solder joint corresponding to the fitting point is obtained.

[0014] Preferably, the point cloud segmentation adopts Euclidean clustering segmentation, including the following specific segmentation process: randomly select a point P in the fused point cloud data; search for k points closest to point P, and cluster the points whose distance is less than a set threshold into a point set Q; if the number of elements in the point set Q no longer increases, the clustering process ends, otherwise it is necessary to select points other than point P in the point set Q to continue searching and clustering until the number of elements in the point set Q no longer increases; the obtained point set Q is the point cloud data set.

[0015] Preferably, the point cloud fitting adopts the RANSAC plane fitting algorithm, including the following specific fitting process: randomly extracting three points in the point cloud data set to verify whether the three points are collinear; determining a reference plane based on the three non-collinear points, and then calculating the distance from the remaining points in the point cloud data set to the reference plane; based on the comparison result of the calculated distance of the remaining points with the set threshold, adding the points with errors less than the set value to the inner point set of the reference plane and iterating; repeating the above process within a specified number of iterations to obtain multiple reference planes; selecting the reference plane with the largest number of inner points as the fitting plane.

[0016] A wire bonding machine force position switching method comprises the following steps:

[0017] S100: The Z-direction average height x of the solder joints on the chip and the substrate is obtained by the Z-direction positioning system of the wire bonding machine soldering area. d ;

[0018] S200: obtaining the current wedge displacement x during the bonding process;

[0019] S300: Setting a switching threshold Δ; when x d -x>Δ, the force control mode is adopted; when x d -x≤Δ, position control mode is adopted.

[0020] Preferably, in step S100, the wire bonding machine welding area Z-direction positioning system uses transfer learning to quickly adapt to chips of different welding sizes. The specific transfer learning process is as follows: design a point cloud annotation method to provide annotation data for the semantic segmentation task; determine the source domain that meets the current project, and establish a corresponding data set to train the source domain model, and then extract effective feature representations; use the extracted feature representations to improve the semantic segmentation algorithm to improve the performance of the algorithm in the target domain; when chips of different sizes are used as target domains, reduce the model training time by migrating from the source domain to the target domain.

[0021] Compared with the prior art, the beneficial effects of this application are:

[0022] Compared with traditional two-dimensional positioning data, this application uses a three-dimensional point cloud processing module combined with traditional two-dimensional positioning data to achieve target recognition and positioning of chips and substrates, with higher precision and accuracy. By measuring the height of the chip and substrate and using the position deviation as the force position switching condition, a more stable force position switching operation can be achieved, improving the accuracy and consistency of wire bonding. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the structure of the wire bonding machine in this application.

[0024] Figure 2 Schematic diagram of the working process of the wire bonding machine in this application.

[0025] Figure 3 This is a schematic diagram of the workflow of the welding zone Z-axis positioning system in this application.

[0026] Figure 4 This is a schematic diagram of the workflow when the point cloud acquisition unit in this application uses an RGB-D camera.

[0027] Figure 5 This is a schematic diagram of the workflow when the point cloud acquisition unit in this application uses a laser scanner.

[0028] Figure 6 Schematic diagram of the bonding workflow of a wire bonding machine when the laser scanner in this application adopts a line scanner.

[0029] Figure 7 Schematic diagram of the bonding workflow of a wire bonding machine when the laser scanner in this application adopts a grating scanner.

[0030] Figure 8 This is a schematic diagram of the specific installation structure of the point cloud acquisition unit in this application.

[0031] Fig. 9 Schematic diagram of the use of radius filtering for point cloud data in this application.

[0032] Fig.10 Schematic diagram of the workflow of point cloud segmentation using Euclidean clustering segmentation in this application.

[0033] Fig.11 Schematic diagram of the workflow of the RANSAC fitting algorithm used for point cloud fitting in this application.

[0034] Fig.12 This is a schematic diagram of the workflow for multi-plane fitting using the RANSAC fitting algorithm in this application.

[0035] In the figure: control module 1, control unit 11, display unit 12, I / O device 13, bonding module 2, bonding head 21, ultrasonic unit 22, heating unit 23, workbench 24, EFO unit 25, wire feeding module 3, tension control unit 31, wire feeding unit 32, material module 4, material speed unit 41, positioning unit 42, loading and unloading unit 43, vision module 5, camera unit 51, lighting unit 52, image processing unit 53, point cloud processing module 6, point cloud acquisition unit 61, data processing unit 62. DETAILED DESCRIPTION

[0036] Below, the present application is further described in conjunction with specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0037] In the description of the present application, it should be noted that directional words, such as the terms "center", "lateral", "longitudinal", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating directions and positional relationships are based on the directions or positional relationships shown in the accompanying drawings, which are only for the convenience of narrating the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and cannot be understood as limiting the specific scope of protection of the present application.

[0038] It should be noted that the terms "first", "second", etc. in the description and claims of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0039] The terms "including" and "having" and any variations thereof in the specification and claims of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0040] In order to facilitate the subsequent understanding of the technical solution of the present application, a brief description of the specific structure and working process of a traditional wire bonding machine may be given first.

[0041] like Figure 1 As shown, the conventional wire bonding machine includes a control module 1 , a welding module 2 , a wire feeding module 3 , a material module 4 and a vision module 5 .

[0042] Among them, the control module 1 is mainly responsible for controlling the movement of the components corresponding to each module of the wire bonding machine. The control module 1 mainly includes a control unit 1, a display unit 12 and an I / O device 13. The control unit 1 is used to store the control software and to make signal connections with the control elements of the other modules. The control unit 1 can control the movement of the components of the wire bonding machine according to the received command signals; the display unit 12 is used to display the control status of the control unit 1 and the working status of the wire bonding machine; the I / O device 13 is used for data communication.

[0043] The welding module 2 is mainly responsible for the ball welding of the wire bonding machine. The welding module 2 mainly includes a bonding head 21, an ultrasonic unit 22, a heating unit 23, a workbench 24 and an EFO unit 25. The workbench 24 is used to place materials that need to be welded, such as chips and substrates, and the workbench 24 can only move in the horizontal XY direction; the bonding head 21 is arranged directly above the workbench 24 for bonding and welding the chips and substrates placed on the workbench 24; the ultrasonic unit 22 is used to collect the ultrasonic impedance value of the solder joint during the bonding process, so as to facilitate the detection of the welding quality of the solder joint; the heating unit 23 can be used to provide a high temperature environment for the bonding, so that the lead frame is heated, thereby promoting the bonding process; the EFO unit 25 can be installed on the bonding head 21, and the EFO unit 25 can improve the quality of the bonding process through the ignition ball technology.

[0044] The wire feeding module 3 is mainly responsible for feeding the welding wire from the wire coil, completing the feeding of the welding wire, and ensuring the smoothness of the bonding process. The wire feeding module 3 mainly includes a tension control unit 31 and a wire feeding unit 32; the wire feeding unit 32 is used to feed the welding wire from the wire coil, and the tension control unit 31 is used to tension the fed welding wire.

[0045] The material module 4 is mainly responsible for completing tasks such as loading, transporting, positioning and unloading of welding wires. The material module 4 mainly includes a material transfer unit 41, a positioning unit 42 and a loading and unloading unit 43. The material transfer unit 41 is mainly used to transfer welding wires, chips and substrates to specific workstations; the loading and unloading unit 43 is mainly used to unload the welded chips; the positioning unit 42 is used to press and position the chips and substrates to be bonded.

[0046] The visual module 5 can obtain images of the chip and substrate in real time during the bonding process, and locate the solder joints in the soldering area through data analysis. The visual module 5 mainly includes a camera unit 51, a lighting unit 52, and an image processing unit 53; the camera unit 51 is used to capture images of the soldering area; the lighting unit 52 is used to illuminate the soldering area to ensure that the camera unit 51 can capture sufficiently clear images; the image processing unit 53 can analyze and process the image data captured by the camera unit 51 to obtain the positioning data of each solder joint in the soldering area.

[0047] From the basic structure of the above-mentioned conventional wire bonding machine, it can be known that the positioning system of the welding area of ​​the conventional wire bonding machine mainly includes the visual module 5. However, the welding point positioning data collected by the visual module 5 is two-dimensional data, and it is impossible to accurately obtain the positioning data of each welding point in the welding area in the Z direction; in order to accurately obtain the Z-direction positioning information of each welding point in the welding area, the present application has made improvements on the basis of the positioning system of the conventional wire bonding machine, and the specific scheme is as follows.

[0048] One aspect of the present application provides a wire bonding machine pad Z-positioning system, such as Figure 1 As shown, one of the preferred embodiments adds a point cloud processing module 6 on the basis of the traditional vision module 5. During the bonding process, the vision module 5 can locate and extract the solder joints in the welding area by image capture to obtain two-dimensional solder joint position data. The point cloud processing module 6 can obtain three-dimensional point cloud data of the welding area through visual recognition; and can fuse the obtained point cloud data with the solder joint position data extracted by the vision module 5 to calculate the Z-direction position information of each solder joint in the welding area. Then the control module 1 of the wire bonding machine can receive the Z-direction position information of the solder joints finally obtained by the Z-direction positioning system of the welding area to control the welding module 2 to perform precise force and position control switching to ensure that the bonding failure caused by too small a feed amount or the chip damage caused by too large a feed amount will not occur during bonding.

[0049] It can be understood that the image data of the welding area captured by the conventional visual module 5 through the camera unit 51 is two-dimensional data; while the visual recognition of the welding area by the point cloud processing module 6 can obtain three-dimensional point cloud data. Since the point cloud data obtained in the point cloud processing module 6 not only includes the point cloud formed by the welding point, but also includes the point cloud formed by other impurities, by fusing the welding point image data collected by the conventional visual module 5 with the point cloud data obtained by the point cloud processing module 6, the point cloud that does not meet the requirements can be eliminated, leaving only the three-dimensional point cloud data corresponding to each welding point. Therefore, the Z-direction position information corresponding to the welding points on the chip and the substrate can be calculated based on the three-dimensional information of the point cloud data, and the obtained Z-direction position information of the welding points has higher precision and accuracy. According to the obtained Z-direction position information of the welding points, the height of the chip and the substrate can be measured, and then the position deviation can be used as the force position switching condition to achieve a more stable force position switching operation and improve the accuracy and consistency of wire bonding.

[0050] It should be known that the bonding process of the wire bonder is as follows Figure 2 As shown, it mainly includes the following steps: the first step is to perform preparatory work before bonding and pre-treat the chip that needs to be bonded; the second step is to place the chip and the corresponding substrate on the workbench 24 and position the welding area of ​​the chip and the substrate through the positioning unit 42; the third step is to bond the positioned chip and the substrate through the bonding head 21, and the bonding process includes the steps of moving the bonding head-burning the ball-first welding-arc initiation-arc pulling-second welding-cutting the tail wire; the fourth step is to check the quality of the welding after completing the bonding process. After the inspection is completed, the current bonding process is terminated and ready to enter the next bonding process. In this embodiment, the process of collecting point cloud data of the welding area through the point cloud processing module 6 is mainly executed between the second step and the third step; during the period when the original camera unit 51 of the wire bonding machine shoots and locates the welding area, the point cloud processing module 6 can collect point cloud data of the welding area, and then calculate the Z-direction height information of the chip and the solder joints on the substrate after pre-processing the point cloud through the algorithm, and then pass it to the control module 1 to perform force position switching through the position error to ensure that the bonding failure caused by too small a feed amount or the chip is damaged by too large a feed amount during bonding will not occur.

[0051] It should be noted that the spatial position of the solder joint can be represented by the spatial coordinates XYZ, that is, the position information of the solder joint in the horizontal plane is represented by the horizontal coordinates XY, and the height information of the solder joint in space is represented by the coordinates Z. At the same time, the point cloud processing module 6 of the present application and the original module of the wire bonding machine are parallel module systems; that is, the point cloud processing module 6 is an additional part of the module system based on the structure of the traditional wire bonding machine, and its communication control mechanism is consistent with other module systems of the wire bonding machine.

[0052] In this embodiment, Figure 1 and Figure 3 As shown, the point cloud processing module 6 includes a point cloud acquisition unit 61 and a data processing unit 62. The point cloud acquisition unit 61 is arranged above the substrate for placing the chip, and the point cloud acquisition unit 61 can perform image acquisition of the welding area by means of 3D visual recognition to obtain corresponding point cloud data; the data processing unit 62 can receive the point cloud data collected by the point cloud acquisition unit 61 and fuse it with the welding point position data collected by the visual module 5 to obtain fused point cloud data corresponding only to each welding point in the welding area; the fused point cloud data is processed by an algorithm to calculate the Z-direction position information of each welding point on the chip and the substrate.

[0053] It should be known that for the data collection of solder joint positioning in the soldering area, in a conventional wire bonding machine, the camera unit 51 generally uses a CCD camera; the CCD camera has the characteristics of high sensitivity, small size, long life and high reliability, and is the most frequently used visual sensor in industrial visual inspection systems. According to the scanning dimension and range, CCD cameras can also be divided into area array cameras and line array cameras; area array cameras can shoot objects on a surface, and line array cameras can collect image information on a line, and then shoot the entire surface through the movement of the camera.

[0054] Of course, CMOS cameras are also used for image acquisition and play an important role in the field of optics. Unlike CCD cameras, CMOS cameras have a higher sensitivity than CCD cameras, especially in dark working environments, and have a stronger ability to suppress noise. With the development of technology, the performance of CMOS cameras has been further enhanced, even surpassing CCD cameras, so there is a broad development prospect. Therefore, in the selection of the camera unit 51 of the wire bonding machine, CMOS cameras are also of reference value. Therefore, in this embodiment, the camera unit 51 is preferably selected using a CCD camera or a CMOS camera; accordingly, in this embodiment, image processing and some other algorithms also need to be suitable for CCD cameras or CMOS cameras. In order to facilitate the description of the subsequent content, the subsequent content will be described by taking the camera unit 51 using a CCD camera as an example.

[0055] It should also be known that there are many specific options for the point cloud acquisition unit 61. Common point cloud acquisition units 61 include RGB-D cameras and three-dimensional scanners. For ease of understanding, the following will use RGB-D cameras and three-dimensional scanners as examples to describe in detail how the point cloud acquisition unit 61 acquires point cloud data.

[0056] Example 1: The point cloud acquisition unit 61 uses an RGB-D camera.

[0057] like Figure 4As shown, the RGB-D camera mainly includes a depth module and an RGB module. The depth module can obtain a depth image of the welding area by shooting images, so as to obtain a point cloud with depth information; the RGB module can obtain an original color image by shooting images. After obtaining a point cloud with depth information and an original color image, the obtained point cloud can be preprocessed to eliminate the point cloud that obviously does not belong to the welding point; then the coordinates of the point cloud are translated and rotated to coincide with the coordinates of the original color image; finally, the point cloud with coincident coordinates is matched with the original color image, and finally the point cloud data with color information can be output. It should be noted that by coloring the point cloud with an RGB-D camera, the amount of calculation of the Z-direction positioning information of the welding point calculated by the point cloud and the complexity of data processing can be reduced.

[0058] Example 2: The point cloud acquisition unit 61 uses a three-dimensional scanner.

[0059] It is important to know that 3D scanners can generally be divided into area scanners and line scanners; common line scanners include laser scanners, etc., and common area scanners include raster scanners, etc. For ease of understanding, the following will take laser scanners and raster scanners as examples for detailed description.

[0060] Laser scanners are monochromatic and linear, so there is no need to perform special surface treatment on the target object when scanning, and the point cloud obtained has small errors and noise, which makes the scanning efficiency, scanning accuracy and scanning authenticity of the point cloud data high. Raster scanners can obtain corresponding point cloud data by shooting and scanning the opposite side; raster scanners are small in size, and their image acquisition method is similar to that of RGB-D cameras, so they will not be elaborated in detail here. The following is a detailed description of the process of laser scanners obtaining point cloud data.

[0061] like Figure 5 As shown, the laser scanner emits laser pulses to the area where the chip and substrate, which are the target objects, are located; then the reflected laser pulses are received by the built-in sensor of the laser scanner to calculate the actual reflection time difference of the pulses; the distance from the substrate and the chip to the three-dimensional scanner can be calculated through the calculated time difference; the distance of the laser point of the laser scanner at each deflection angle is measured, and the coordinate point set can be obtained after coordinate solution based on the distance measured by the laser scanner at multiple deflection angles; according to the platform angle used to install the laser scanner, that is, the platform angle when the laser pulse is emitted, the coordinate point set is converted into a polar coordinate point set, and then the required point cloud data can be obtained after coordinate conversion.

[0062] It should be known that due to the different scanning types of laser scanners and grating scanners, there will be certain differences in the working order of the two during the bonding process; for ease of understanding, the working order of laser scanners and grating scanners during the bonding process will be described in detail below.

[0063] (1) For laser scanners, such as Figure 6 As shown, the laser scanner needs to be calibrated before performing the bonding work. After the calibration is completed, the laser scanner can be turned on and the workbench 24 can be moved slightly in the XY direction, so that the laser scanner can first scan the chip on the workbench 24 to obtain its three-dimensional point cloud data, that is, the laser scanner needs to be pre-collected before bonding; then the bonding work starts, and the CCD camera takes a picture of the welding area; the data processing unit 62 can send the Z-axis positioning information of the welding point to the control module 1 according to the fused data, so that the control module 1 controls the bonding head 21 to perform corresponding feeding according to the received data information, and at the same time, the control module 1 can control the motion of the drive motor for moving the workbench 24, so that the workbench 24 can cooperate with the bonding head 21 to move to complete the bonding work.

[0064] (2) For raster scanners, such as Figure 7 As shown, the laser scanner needs to be calibrated before the bonding work is carried out, and the bonding work starts after the calibration is completed; at this time, the raster scanner and the CCD camera can both perform surface scanning on the welding area to obtain point cloud data and original color images respectively; then the point cloud data and the original color image data are processed, and the processing results are sent to the control module 1; the control module 1 controls the bonding head 21 to perform corresponding feeding according to the received data information, and at the same time, the control module 1 can control the motion of the driving motor for moving the workbench 24, so that the workbench 24 can cooperate with the bonding head 21 to move to complete the bonding work.

[0065] It should be noted that after the point cloud acquisition unit 61 is installed, the calibration can be performed through a calibration plate or other reference objects to perform image calibration, and the internal and external parameters of the point cloud acquisition unit 61 can be determined through calibration to obtain more accurate image measurement results. That is, based on the point cloud information of the chip and the substrate, combined with the known structural parameters of the point cloud acquisition unit 61 and the calibration results, the height information of the solder joints on the chip and the substrate is calculated.

[0066] It is also necessary to know that the RGB-D camera and the raster scanner will be affected by external light to a certain extent. Therefore, when the point cloud acquisition unit 61 uses an RGB-D camera or a raster scanner, the light intensity should be reasonably controlled under the light conditions required for the CCD camera to prevent the acquisition of point cloud data from being affected by excessive or dark light.

[0067] In this embodiment, it can be seen from the above content that the point cloud acquisition unit 61 needs to be installed directly above the workbench 24 to ensure the accuracy of image acquisition; there are many specific installation methods for the point cloud acquisition unit 61, and the following will be described in detail through one of the installation structures. Figure 8 As shown, the point cloud acquisition unit 61 can be installed side by side with the camera unit 51 of the visual module 5 and surround the bonding head 21, so that the point cloud acquisition unit 61 and the camera unit 51 are basically located directly above the working area of ​​the workbench 24; therefore, when performing image processing at a later stage, it is only necessary to bring the transfer matrix of the offset error and the parameters of the lens distortion into the calculation.

[0068] It should be noted that when selecting the point cloud acquisition unit 61, the main settings on the market should be compared, and the point cloud acquisition unit 61 suitable for collecting chip point cloud information should be selected in combination with the working environment of the wire bonding machine. Specifically, the selection of the point cloud acquisition unit 61 is mainly considered from the following two aspects:

[0069] (1) Based on the working environment of the wire bonding machine, the selected point cloud acquisition unit 61 needs to have the characteristics of high accuracy, fast speed, complete data and stable operation to ensure that the wire bonding machine can work stably and quickly.

[0070] (2) Based on the installation position of the point cloud acquisition unit 61, the volume of the point cloud acquisition unit 61 should be small to ensure that it does not affect the normal operation of other components after installation.

[0071] In this embodiment, the point cloud data and the weld point position data are both denoised after being acquired; based on the denoised point cloud data and weld point position data, data fusion is performed after the coordinates are uniformly transformed into a world coordinate system to obtain fused point cloud data of each weld point in the welding area determined by the weld point position data with the corresponding point cloud.

[0072] It should be known that the denoising process for solder joint position data is a well-known technology for those skilled in the art, that is, the image denoising processing method commonly used in the current conventional wire bonding machine vision module 5 can be adopted, so it will not be elaborated in detail here. The denoising of point cloud data is mainly point cloud filtering, which requires that the filtered point cloud should ensure that details are not missing, and there is no under-filtering and filtering. There are many specific point cloud filtering methods, such as bilateral filtering, radius filtering, Gaussian filtering, pass-through filtering and conditional filtering, etc. One or more filtering methods can be used for collaborative filtering; for ease of understanding, this embodiment takes pass-through filtering and radius filtering as examples for detailed description.

[0073] If the point cloud filtering adopts the straight-through filtering, after the point cloud acquisition unit 61 is calibrated, the scanned chip and substrate parts for welding can be roughly extracted by the straight-through filtering algorithm to remove the rest of the environmental point cloud. Specifically, a dimension can be formulated and the value range of the dimension can be set first, and then all the point clouds can be traversed to determine whether the traversed points are within the value range. If not, the point is deleted, and the points left after the traversal are the point cloud data after the straight-through filtering.

[0074] If the point cloud filtering adopts radius filtering, the radius filtering has the characteristics of fast processing speed, good effect and easy modification and adjustment; its main working process is: first traverse all points of the point cloud data, if the number of adjacent points of a point within a given search radius is less than a given threshold number, then the point is determined to be an outlier and deleted; until all points are searched, the remaining points are the point cloud data after radius filtering; this filtering method can further remove noise far away from the scanned surface. One specific example is Fig. 9 As shown in , if the threshold number within the specified search radius is two neighboring points, then Fig. 9 The black points in will be deleted, and the blue points can be retained; if the threshold number within the specified search radius is three neighboring points, then Fig. 9 The black and blue points in the image are deleted.

[0075] It should be noted that the above two point cloud filtering methods can be implemented separately, or through-filtering can be implemented first and then radius filtering. Since the chip is small and thin, when performing point cloud filtering, the accuracy of the corresponding filtering program must meet the requirements and a reasonable threshold must be set. When encountering a large number of difficult point clouds, downsampling can also be used to streamline the point cloud data according to the actual scanning situation to save the point cloud processing calculation speed. Requirements for downsampling filtering: key information should not be over-processed, and key information on the solder joints of the chip and substrate should be retained while reducing the number of point clouds, without affecting subsequent operations.

[0076] In this embodiment, in order to facilitate the fusion of point cloud data and solder joint position data, after completing the filtering and denoising of the point cloud data, the coordinate system of the point cloud data and the coordinate system of the solder joint position data need to be unified into a world coordinate system. The unification of coordinates can be achieved through the following conversion formula.

[0077]

[0078] Among them, Z u represents the z coordinate of the imaging point, K represents the intrinsic parameter matrix of the imaging unit (such as RGB-D camera and CCD camera), R represents the rotation matrix, t represents the translation vector, (u, v) represents the coordinates of the point cloud in the pixel plane coordinate system, (u0, v0) represents the coordinates of the center point of the pixel plane, (XW , Y W , Z W ) represents the coordinates of the imaging point in the world coordinate system, f represents the focal length, s x and y They represent the physical sizes of the imaging point in the u-axis and v-axis directions respectively.

[0079] After completing the coordinate transformation, the solder joint positioning data captured by the CCD camera can be linearly transformed through the collinear equation. The specific transformation is achieved through the collinear equation, and the formula of the collinear equation is as follows:

[0080]

[0081] Among them, (x, y) represents the coordinates of the image point on the image plane, x0, y0 and f0 represent the internal parameters of the influence, (X S , Y S , Z S ) represents the object space coordinates of the CCD camera photography center, (X, Y, Z) is the object space coordinates of the imaging point; a i , b i , c i (i=1, 2, 3) are the 9 corresponding direction cosines composed of the 3 exterior azimuth elements of the image.

[0082] In this embodiment, after completing the above-mentioned linear transformation, the RGB information of the circular positioning key points in the solder joint positioning data can be mapped to the corresponding point cloud in the point cloud data to obtain the required fused point cloud data. Specifically, the fusion process of the point cloud data and the solder joint position data is: the chip and substrate image data taken by the CCD camera are subjected to denoising to extract the key points, and then the key points extracted from the image are fused with the point cloud data so that the point cloud RGB information of the corresponding key points appears on the point cloud to obtain the corresponding fused point cloud data. It should be noted that when performing data fusion, only the RGB information on the circular point located by the solder joint position data collected by the CCD camera is fused with the point cloud, and then the circular point cloud corresponding to the solder joints on the chip and substrate can be displayed by color to obtain the fused point cloud data.

[0083] In this embodiment, after obtaining the fused point cloud data, the circular point clouds of the chip and substrate solder joints can be segmented and extracted by color, and then the average Z value of the points can be calculated; then the distance between the chip solder joints and the substrate solder joints can be estimated, so as to control the z-axis feed of the bonding head 21 and perform 3D detection on the bonding action of the bonding head 21. Specifically, Figure 3As shown, the processing process of the fused point cloud data by the data processing unit 62 includes point cloud segmentation and point cloud fitting. Through point cloud segmentation, the key features of the filtered fused point cloud data are extracted to obtain a point cloud data set in which the point cloud is clustered. Through point cloud fitting, the point cloud data set is fitted to obtain two fitting planes corresponding to the chip and the substrate respectively. Based on the distance from any fitting point on one of the fitting planes to the other fitting plane, the Z-direction position information of the solder joint corresponding to the fitting point is obtained, so that the position error is used as the condition for subsequent force position switching control.

[0084] It should be noted that due to the small size and thinness of the chip, a reasonable threshold needs to be set when performing point cloud segmentation; and the segmentation range setting of the segmentation algorithm should be set as far as possible in the order from large to small and from coarse to fine, so as to ensure that the corresponding key information required can be fully extracted. For point cloud fitting, the error of the fitting surface is required to be as small as possible to facilitate the subsequent distance calculation of the chip and the substrate. There are many specific implementation methods for point cloud segmentation and point cloud fitting. For point cloud segmentation, there are mainly Euclidean clustering segmentation, regional growing algorithm and segmentation based on minimum graph cut. For point cloud fitting, there are mainly RANSAC plane fitting algorithm and least squares plane fitting algorithm. For the convenience of understanding, the following will use Euclidean clustering algorithm and RANSAC plane fitting algorithm as examples for point cloud segmentation and point cloud fitting.

[0085] Specifically, the point cloud segmentation adopts Euclidean clustering segmentation, which can extract key parts such as chip and substrate solder joints. After the substrate is filtered through the straight-through method, the solder joints are also retained, and the shape is regular. Therefore, during clustering, the point cloud data set corresponding to the solder joints on the chip and the point cloud data set corresponding to the solder joints on the substrate can be extracted separately. The basic principle of Euclidean clustering segmentation is: its distance judgment criterion is Euclidean distance, and the calculation formula of Euclidean distance d in three-dimensional space is:

[0086]

[0087] Among them, (x1, y1, z1) and (x2, y2, z2) represent the three-dimensional space coordinates of two points respectively.

[0088] It should be noted that when performing Euclidean clustering segmentation, a range search algorithm needs to be used. This embodiment will take the kd tree nearest neighbor search algorithm as an example to describe in detail the specific segmentation process of Euclidean clustering segmentation.

[0089] like Fig.10As shown, the fused point cloud data is first introduced, and the parameters are initialized and the thresholds are set. Then a point P is randomly selected in the fused point cloud data; the k points closest to point P are searched, and the distances from these k closest points to point P are calculated in turn by the above-mentioned Euclidean distance calculation formula. The points whose distances are less than the set threshold are clustered in the point set Q; if the number of elements in the point set Q no longer increases, the clustering process ends, otherwise it is necessary to select points other than point P in the point set Q to continue searching and clustering until the number of elements in the point set Q no longer increases; the final clustered point set Q is saved to obtain the required point cloud data set. It should be known that the specific value of the threshold set in the point cloud segmentation algorithm can be selected according to the actual needs of technicians in this field.

[0090] Specifically, the point cloud fitting adopts the RANSAC plane fitting algorithm, such as Fig.11 As shown, the specific fitting process includes the following: first, the number of iterations is initialized, the threshold is set, and various variables are assigned values; at the same time, the number of internal points and plane parameters also need to be initialized.

[0091] After initialization, the elements in the point cloud data set are randomly rearranged and three points are randomly extracted to verify whether the three points are collinear. The collinearity of the three points can be verified by using the rule of the slope of the linear vector. After deduction, the collinearity formula of the three points can be obtained as follows: (x 02 -x 01 )(y 03 -y 01 )=(x 03 -x 01 )(y 02 -y 01 ); where (x 01 ,y 01 ), (x 02 ,y 02 ), (x 03 ,y 03 ) are the plane coordinates of three random points respectively.

[0092] After obtaining three non-collinear points, a reference plane can be determined based on these three non-collinear points. At this time, these three points constitute the internal point set of the reference plane; then the distances from the remaining points in the point cloud data set to the reference plane are calculated, and according to the result of comparing the calculated distances of the remaining points with the set threshold, the points with errors less than the set value are added to the internal point set of the reference plane and iterated. Repeating the above process within the specified number of iterations can obtain multiple reference planes, and the reference plane with the largest number of internal points is selected as the fitting plane.

[0093] It should be known that there are many ways to determine the fitting of the reference plane. A common way is to use the least squares method for fitting. The fitting process is a well-known technology for those skilled in the art, so it will not be elaborated in detail here.

[0094] It is important to know that when calculating the Z-direction positioning information of the solder joints on the chip and the substrate, two fitting planes corresponding to the solder joints on the chip and the substrate are obtained at the same time. However, point cloud fitting can only fit one plane from the point cloud data set at a time, so multiple fittings are required to obtain two fitting planes belonging to the solder joints on the chip and the substrate. For ease of understanding, the following is a detailed description of the process of obtaining the two required fitting planes by multi-plane fitting segmentation based on the RANSAC plane fitting algorithm.

[0095] like Fig.12 As shown, first load the original point cloud file, and define the original point cloud data, point cloud data set, and the index object of the plane coefficient storage inner point; after completing the loading definition, you can set the number of detection surfaces, and set the loop variable i = 0. Based on the loaded point cloud data set, the fitting plane can be obtained by the RANSAC plane fitting algorithm; for the first fitting of the RANSAC plane fitting algorithm, the plane with the most points is fitted first. After the plane fitting is completed, the obtained fitting plane is extracted, and after the extraction is completed, it is saved separately and removed from the point cloud data set, so that the points of the retained fitting plane in the point cloud data set are deleted. Then continue to fit from the remaining points in the point cloud data set, and still choose to fit the plane with the most points first, so that the second fitting plane can be obtained, and the variable i is added by 1 at this time. Then judge whether the value of the variable i at this time corresponds to the set number of detection surfaces; if the variable i is less than the number of detection surfaces, continue to remove the obtained fitting plane and continue fitting; if the variable i is greater than or equal to the number of detection surfaces, output the retained fitting plane, and the required number of fitting planes can be obtained.

[0096] Specifically, for this embodiment, the number of detection surfaces required should be two, so it is necessary to loop twice to obtain three fitting planes to make the variable i = 2; but at this time, only the planes of the first two fittings are saved, so the number of fitting planes output is two, and they correspond to the solder joints on the chip and the substrate respectively. That is, in the point cloud data set obtained by point cloud segmentation, the point cloud can be clustered in two areas to correspond to the solder joints on the chip and the substrate respectively; so in the point cloud fitting, only the plane accuracy obtained by the first two fittings can meet the requirements.

[0097] Another aspect of the present application provides a wire bonding machine force position switching method, wherein a preferred embodiment includes the following steps:

[0098] S100: The Z-direction average height x of the solder joints on the chip and the substrate is obtained by the Z-direction positioning system of the wire bonding machine soldering area. d .

[0099] S200: Acquire the current cleaver displacement x during the bonding process.

[0100] S300: Setting a switching threshold Δ; when x d -x>Δ, the force control mode is adopted; when x d -x≤Δ, position control mode is adopted.

[0101] It should be known that the Z-direction feed distance of the bonding head 21 depends on the Z-direction height of the solder joint. Since there is a slight difference in the Z-direction height of each solder joint, it can be replaced by an average value. The bonding of the bonding head 21 can be a chip or a substrate; for ease of understanding, the above-mentioned force position switching process will be described in detail below while focusing on the welding target.

[0102] Specifically, based on the different welding targets, the average Z-axis height x d Including the average Z-axis height x of the solder joint on the corresponding chip dchip , and the average Z-axis height x of the corresponding solder joint on the substrate dbase ; What you need to know is that x dchip and x dbase The precision needs to be accurate to the micro-nanometer level. Then, in the experimental environment, the grating sensor is used to capture the displacement x of the current splitter for chip bonding and substrate bonding during the bonding process. chopper-chip and x chopper-base The switching threshold Δ can be obtained based on the experience of technicians in the field of welding equipment debugging. The switching threshold values ​​in the chip welding and substrate welding processes can be consistent or inconsistent, and can be set according to actual needs.

[0103] When performing chip soldering, if x dchip -x chopper-chip >Δ, the force position is switched to enter the force control mode; if x dchip -x chopper-chip ≤Δ, the force position is switched to enter the position control mode.

[0104] When soldering the substrate, if x dbase -x chopper-base >Δ, the force position is switched to enter the force control mode; if x dbase -x chopper-base ≤Δ, the force position is switched to enter the position control mode.

[0105] In this embodiment, a wire bonder often needs to weld multiple chips of different sizes. If the above algorithm process needs to be re-recorded and calculated every time a chip of a different size is replaced, a large amount of data processing may cause a certain delay in the wire bonder and reduce production efficiency. Since the structural parameters of the wire bonder itself will basically not change, when welding chips of different sizes, the data processing amount can be reduced through transfer learning to improve production efficiency.

[0106] Specifically, in this embodiment, since there may be large domain differences between the source domain data and the target domain data, the semantic segmentation method of the improved three-dimensional point cloud can be introduced into the transfer learning to achieve rapid transfer learning from the source domain to the target domain. The specific transfer learning process is as follows: design an effective point cloud annotation method to provide annotation data for the semantic segmentation task; determine the source domain that meets the current project, and establish a corresponding data set to train the source domain model, and then extract effective feature representations; use the extracted feature representations to improve the semantic segmentation algorithm to improve the performance of the algorithm in the target domain; when chips of different sizes are used as the target domain, quickly migrate from the source domain to the target domain to reduce the model training time to improve production efficiency.

[0107] The above describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited by the above embodiments, and the above embodiments and the specification only describe the principles of the present application. The present application may have various changes and improvements without departing from the spirit and scope of the present application, and these changes and improvements fall within the scope of the present application for which protection is sought. The scope of protection claimed by the present application is defined by the attached claims and their equivalents.

Claims

1. A wire bonding machine welding zone Z-direction positioning system, characterized in that: include: Vision module; The visual module locates and extracts the welding points in the welding area by image capture; as well as Point cloud processing module; The point cloud processing module obtains point cloud data of the welding area through visual recognition, and fuses the obtained point cloud data with the welding point position data extracted by the visual module to calculate the Z-direction position information of each welding point in the welding area.

2. The wire bonding machine welding zone Z-direction positioning system according to claim 1, characterized in that: The point cloud processing module includes: A point cloud acquisition unit; the point cloud acquisition unit is arranged above a substrate for placing a chip, and the point cloud acquisition unit acquires an image of the welding area based on 3D visual recognition to obtain the point cloud data; and A data processing unit; the data processing unit is suitable for receiving the point cloud data and fusing it with the solder point position data to obtain fused point cloud data corresponding only to each solder point in the soldering area; the fused point cloud data is processed by an algorithm to calculate the Z-direction position information of each solder point on the chip and the substrate.

3. The wire bonding machine welding zone Z-direction positioning system according to claim 2, characterized in that: The point cloud acquisition unit adopts an RGB-D camera, and the RGB-D camera includes a depth module and an RGB module; The depth module is used to obtain a point cloud with depth information, and the RGB module is used to obtain an original color image; then the data of the depth module and the RGB module are matched to obtain the point cloud data with color.

4. The wire bonding machine welding zone Z-direction positioning system according to claim 2, characterized in that: The point cloud acquisition unit uses a three-dimensional scanner, and the process of the three-dimensional scanner acquiring the point cloud data is as follows: The three-dimensional scanner emits pulses to the area where the chip and the substrate are located, and calculates the distance from the substrate and the chip to the three-dimensional scanner based on the reflection time difference of the pulses; A set of coordinate points is obtained based on distances measured by the three-dimensional scanner at multiple deflection angles after coordinate calculation; According to the angle of the platform for installing the three-dimensional scanner, the coordinate point set is converted into a polar coordinate point set, and then the point cloud data is obtained through coordinate conversion; Alternatively, the three-dimensional scanner acquires the point cloud data by surface scanning.

5. The wire bonding machine welding zone Z-direction positioning system according to claim 2, characterized in that: The point cloud data and the weld point position data are both denoised after acquisition; based on the denoised point cloud data and the weld point position data, data fusion is performed after the coordinates are uniformly transformed into a world coordinate system to obtain the fused point cloud data of each weld point in the welding area determined by the weld point position data with the corresponding point cloud.

6. The wire bonding machine welding zone Z-direction positioning system according to any one of claims 2 to 5, characterized in that: The processing process of the fused point cloud data by the data processing unit includes point cloud segmentation and point cloud fitting; Extract key features of the fused point cloud data by segmenting the point cloud to obtain a point cloud data set in which the point clouds are clustered; Fitting the point cloud data set by the point cloud fitting to obtain two fitting planes corresponding to the chip and the substrate respectively; Based on the distance from any fitting point on one of the fitting planes to another fitting plane, the Z-direction position information of the welding point corresponding to the fitting point is obtained.

7. The wire bonding machine welding zone Z-direction positioning system according to claim 6, characterized in that: The point cloud segmentation adopts Euclidean clustering segmentation, including the following specific segmentation process: Randomly select a point P in the fused point cloud data; Search for the k points closest to point P, and cluster the points whose distance is less than the set threshold into the point set Q; If the number of elements in the point set Q no longer increases, the clustering process ends. Otherwise, it is necessary to select points other than point P in the point set Q to continue searching and clustering until the number of elements in the point set Q no longer increases. The obtained point set Q is the point cloud data set.

8. The wire bonding machine welding zone Z-direction positioning system according to claim 6, characterized in that: The point cloud fitting adopts the RANSAC plane fitting algorithm, including the following specific fitting process: Randomly extract three points from the point cloud data set to verify whether the three points are collinear; Determine a reference plane based on three non-collinear points, and then calculate the distance between the remaining points in the point cloud data set and the reference plane; According to the comparison result between the calculated spacing of the remaining points and the set threshold, the points whose errors are less than the set value are added to the inner point set of the reference plane and iterated; The above process is repeatedly implemented within a specified number of iterations to obtain multiple reference planes; the reference plane with the largest number of internal points is selected as the fitting plane.

9. A wire bonding machine power position switching method, characterized in that: The following steps are included: S100: Obtain the Z-direction average height x of the solder joints on the chip and the substrate respectively by using the Z-direction positioning system of the solder joint of the wire bonder according to any one of claims 1 to 8. d ; S200: obtaining the current wedge displacement x during the bonding process; S300: Setting a switching threshold Δ; when x d -x>Δ, the force control mode is adopted; when x d -x≤Δ, position control mode is adopted.

10. The wire bonding machine power position switching method according to claim 9, characterized in that: In step S100, the Z-direction positioning system of the wire bonding machine pad is adapted to chips of different bonding sizes through transfer learning. The specific transfer learning process is as follows: Design a point cloud annotation method to provide annotation data for semantic segmentation tasks; Determine the source domain that matches the current project and establish the corresponding data set to train the source domain model, thereby extracting effective feature representations; The extracted feature representation is used to improve the semantic segmentation algorithm to improve the performance of the algorithm in the target domain; When chips of different sizes are used as target domains, model training time can be reduced by migrating from the source domain to the target domain.

Images