High-precision visual alignment and bonding method

Abstract

The application discloses a high-precision visual alignment and bonding method, belongs to the industrial visual field, and solves the technical problem of inaccurate visual positioning and bonding in microelectronic product processing and comprises the following steps: S1, regional calibration; S2, nine-point repeated calibration combined with precision verification; S3, performing sub-pixel interpolation processing on upper and lower layer images of a bonded workpiece; S4, creating a template image and matching the template image with upper and lower layer interpolation images; and S5, bonding point alignment. The high-precision alignment and bonding in the microelectronic product processing process can improve the production efficiency and quality of the microelectronic product.

Description

Technical Field

[0001] This invention belongs to the field of industrial vision technology, specifically relating to a high-precision visual alignment and bonding method. Background Technology

[0002] Visual alignment technology is a technique that matches a perceived image with a pre-established model. It analyzes information such as feature points, edges, and colors in an image and compares it with a known model to achieve the identification, localization, and tracking of target objects. This technology is widely used in various fields, including industrial automation, robot navigation, medical imaging, augmented reality, and virtual reality.

[0003] In industrial automation, visual alignment technology can be used for product quality inspection, automatic part positioning, and assembly. In robot navigation, it helps robots locate and navigate in complex environments. In medical imaging, visual alignment technology can be used for surgical navigation and tumor localization. In augmented reality and virtual reality applications, it enables the alignment and interaction of virtual objects with the real environment. Visual alignment technology provides precise image analysis and recognition capabilities, offering greater possibilities and convenience for applications across various fields.

[0004] Visual alignment bonding refers to using the marks on the workpieces to align the upper and lower workpieces to be bonded, thereby achieving bonding between the upper and lower workpieces. The upper and lower workpieces to be bonded each have two marks on the left and right sides, and it is required that the upper and lower marks on the left and right sides are aligned.

[0005] Practical high-precision visual alignment and bonding systems require not only higher accuracy in target identification and localization, but also high real-time performance of the algorithms. Existing calibration and localization bonding methods suffer from problems such as low accuracy, reliability, or poor real-time performance, and also have significant requirements on the imaging environment, limiting the versatility of the algorithms. Therefore, developing high-precision, real-time visual localization and bonding methods that meet the requirements of complex environments is of great research significance. Summary of the Invention

[0006] This invention addresses the shortcomings of existing visual alignment and bonding technologies by proposing a high-precision visual alignment and bonding method. This invention solves the problem of inaccurate bonding of upper and lower workpieces in microelectronic product processing through high-precision visual alignment.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a high-precision visual alignment and bonding method, specifically comprising the following steps:

[0008] S1. Regional calibration: Establish the relationship between image coordinates and world coordinates in different regions;

[0009] S2. Nine-point repeat calibration combined with accuracy verification;

[0010] S3. Perform subpixel interpolation processing on the upper and lower images of the workpiece to be bonded.

[0011] S3-1. Take pictures of the left and right marks of the lower layer workpiece to be bonded to obtain images X2 and X3 respectively, and take pictures of the left and right marks of the upper layer workpiece to be bonded to obtain images X4 and X5 respectively. Perform grayscale processing and threshold segmentation on the lower layer images X2 and X3 and the upper layer images X4 and X5 respectively.

[0012] S3-2. Perform subpixel interpolation on images X2, X3, X4, and X5 respectively to obtain images X′2, X′3, X′4, and X′5;

[0013] S3-3. For images X′2, X′3, X′4, and X′5, perform erosion followed by dilation to remove small noise regions, then apply Gaussian filtering. The generation equation for the 3x3 Gaussian filter kernel used is:

[0014]

[0015] S4. Create a template image and match it with the interpolated images above and below;

[0016] S4-1. Create a shape contour template that fits the workpiece mark. Take a picture of the mark to be identified on the workpiece, and perform grayscale processing and threshold segmentation to obtain the template image X1 of the mark. Then, find the contour of the template image X1 through the contour detection algorithm, name it contour1, and calculate the contour area S1.

[0017] S4-2. Find all contours of images X′2, X′3, X′4, and X′5 through contour detection, and name them contour2, contour3, contour4, and contour5. Calculate the corresponding contour areas S2, S3, S4, and S5.

[0018] S4-3. Perform shape matching between the contour contour1 obtained in S4-1 and the four contours contour2, contour3, contour4, and contour5 obtained in S4-2, respectively. Find the contours corresponding to the four minimum matching values, and denote them as the best matching contours contour2m, contour3m, contour4m, and contour5m. Based on the best matching contours, the four marks that match the corresponding image can be found. Add an area constraint area before matching, that is, set the area to be based on the contour area S1 of the template image X1:

[0019] S1-500 <area<S1+500

[0020] S4-4. Calculate the spatial moments m of each best-matching contour contour2m, contour3m, contour4m, and contour5m according to the following formula. ji :

[0021]

[0022] Where x and y are the coordinates of the contour, and i and j are the image order, i,j∈{0,1};

[0023] S4-5, Using m obtained from S4-4 00 m 10 m 01 Calculate the centroids center1(x,y), center2(x,y), center3(x,y), and center4(x,y) of contours contour2m, contour3m, contour4m, and contour5m respectively:

[0024]

[0025] S4-6. Use the contour drawing function to mark the best matching contour found in images X′2, X′3, X′4, and X′5, and draw center1(x,y), center2(x,y), center3(x,y), and center4(x,y).

[0026] S5. Alignment of bonding points;

[0027] S5-1, Fitting Position Calculation: Based on the calibration results of S1, use the coordinate transformation matrix H to convert the image coordinates center1(x,y) obtained in S4-6 into world coordinates P1, center2(x,y) into world coordinates P2, center3(x,y) into world coordinates P3, and center4(x,y) into world coordinates P4.

[0028] S5-2, Calculate the rotation center: Rotate and photograph the upper bonding workpiece at a custom angle to obtain images X6 and X7. Perform operations S3-2 to S5-1 on X6 and X7 respectively to obtain the centroids center5(x,y) and center6(x,y) of the bonding position; Calculate the straight line L1 formed by center3 and center4 and the straight line L2 formed by center5 and center6, and calculate the angle between lines L1 and L2 and the rotation center;

[0029] S5-3. Based on the rotation center obtained in S5-2, use P1 and P2 to calculate the angle angel1 and center point P(x, y) of the lower layer of the workpiece, and use P3 and P4 to calculate the angle angel2 and center point M(x, y) of the upper layer of the workpiece.

[0030] S5-4. Rotate the upper-layer bonding workpiece by a rotation angle ΔA = angle1 - angel2, with a distance [Δx, Δy] = x P -x M The lower layer of the workpiece is moved along the X and Y axes respectively.

[0031] Furthermore, the specific steps for the regional calibration are as follows:

[0032] S1-1, A nine-point calibration plate designed to match the dimensions of the mechanical platform, with a 3×3 shape and an actual distance D between two points;

[0033] S1-2. Fix the nine-point calibration plate onto the mechanical platform and adjust the focus according to the alignment mark height on the calibration plate so that the camera can clearly display the workpiece.

[0034] S1-3, Three coordinate transformation matrices H are obtained from the regional calibration. n(n∈1,2,3) .

[0035] Furthermore, the coordinate transformation matrix H n(n∈1,2,3) The specific steps are as follows:

[0036] First, based on the robotic arm's range of motion, the bonding station is divided into working areas 1 to 3. Calibration plates are then placed in each of the three areas, and images X are captured. The coordinates of the nine-point image (x, y) are calculated. nij y nij ), n∈(1,2,3), i∈(1,2,3), j∈(1,2,3);

[0037] Secondly, for each region, read the coordinates (x, y) of any point among the nine points on the calibration board in the mechanical platform coordinate system. n11 y n11 Let this position be the center point, and calculate the mechanical coordinates (x, y) of other points. nij y nij );

[0038]

[0039] Where m∈{0,1,2};

[0040] Finally, using Zheng's nine-point calibration method, the coordinate transformation matrix between each image X and the corresponding mechanical platform region was calculated, resulting in the coordinate transformation matrix H for the three regions. n(n∈1,2,3) .

[0041] Furthermore, the specific steps for the nine-point repetition calibration are as follows:

[0042] S2-1. Fix the verification calibration board onto the platform and take a picture. Determine the area where the calibration board Mark is located based on the robot arm's motion coordinates, and calculate its image coordinates. Use the coordinate transformation matrix H obtained in step S1-3. n(n∈1,2,3) Find the position (x, y) of each mark on the calibration plate on the mechanical platform, and calculate the distance D1 between any two marks;

[0043] S2-2. Compare D1 with D to obtain the calibration accuracy. If the accuracy requirement is met, the coordinate transformation matrix H can be used; otherwise, recalibrate.

[0044] Compared with existing technologies, the present invention improves the accuracy, reliability and real-time performance of calibration, alignment and bonding algorithms in the following four aspects.

[0045] 1. This invention improves calibration accuracy by designing a calibration plate of fixed size. After calibration, the calibration accuracy is verified.

[0046] 2. Regional calibration: Calibration is performed in different regions according to different working areas and calibration distances, based on the size of different products.

[0047] 3. Subpixel interpolation: Subpixel interpolation is used to process the image of the workpiece to improve the alignment and bonding accuracy by subdividing pixels.

[0048] 4. Calculate the rotation center: The product is rotated within the accuracy range, a second photo is taken, the rotation center is determined, and the angle is calibrated and the alignment accuracy is improved. Attached Figure Description

[0049] Figure 1 This is a flowchart of the present invention.

[0050] Figure 2 This is a design drawing for the calibration board.

[0051] Figure 3 The result is shown in the calibration accuracy diagram.

[0052] Figure 4 This is a schematic diagram of biline interpolation.

[0053] Figure 5 This is a diagram of the template matching results.

[0054] Figure 6 The image shows the matching results for the mark.

[0055] Figure 7 This is a schematic diagram for calculating the center of rotation.

[0056] Figure 8 This is a diagram showing the alignment results. Detailed Implementation

[0057] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples, so as to fully understand how the present invention uses technical means to solve technical problems and achieve technical effects and to implement it accordingly.

[0058] like Figure 1 As shown, a high-precision visual alignment and bonding method includes the following steps:

[0059] S1. Regional calibration: Establish the relationship between image coordinates and world coordinates in different regions.

[0060] S1-1, A nine-point calibration plate designed to match the dimensions of the mechanical platform, with a 3×3 shape and an actual distance D between two points (e.g., ...). Figure 2 (as shown);

[0061] S1-2. Fix the nine-point calibration plate onto the mechanical platform and adjust the focus according to the alignment mark height on the calibration plate so that the camera can clearly display the workpiece.

[0062] S1-3, Three coordinate transformation matrices H are obtained from the regional calibration. n(n∈1,2,3) :

[0063] First, based on the robotic arm's range of motion, the bonding station is divided into working areas 1 to 3. Calibration plates are then placed in each of the three areas, and images X are captured. The coordinates of the nine-point image (x, y) are calculated. nij y nij ), n∈(1,2,3), i∈(1,2,3), j∈(1,2,3);

[0064] Secondly, for each region, read the coordinates (x, y) of any point among the nine points on the calibration board in the mechanical platform coordinate system. n11 y n11 Let this position be the center point, and calculate the mechanical coordinates (x, y) of other points. nij y nij )

[0065]

[0066] Where m∈{0,1,2};

[0067] Finally, using Zheng's nine-point calibration method, the coordinate transformation matrix between each image X and the corresponding mechanical platform region was calculated, resulting in the coordinate transformation matrix H for the three regions. n(n∈1,2,3) .

[0068] After the camera captures motion coordinates, it determines which working area the coordinates belong to and then calls the coordinate transformation matrix H for that working area. n(n∈1,2,3) .

[0069] S2. Nine-point repeat calibration combined with accuracy verification.

[0070] S2-1. Fix the verification calibration board onto the platform and take a picture. Determine the area where the Mark on the calibration board is located based on the robot arm's motion coordinates, and calculate its image coordinates. Use the coordinate transformation matrix H obtained in step S1-3. n(n∈1,2,3) Find the position (x, y) of each mark on the calibration plate on the mechanical platform, and calculate the distance D1 between any two marks (e.g., ...). Figure 3 (as shown);

[0071] S2-2. Compare D1 with D to obtain the calibration accuracy. If the accuracy requirement is met, the coordinate transformation matrix H can be used; otherwise, recalibrate.

[0072] S3. Perform subpixel interpolation processing on the upper and lower images of the workpiece to be bonded.

[0073] S3-1. Take pictures of the left and right marks of the lower layer workpiece to be bonded to obtain images X2 and X3 respectively, and take pictures of the left and right marks of the upper layer workpiece to be bonded to obtain images X4 and X5 respectively. Perform grayscale processing and threshold segmentation on the lower layer images X2 and X3 and the upper layer images X4 and X5 respectively.

[0074] S3-2. Subpixel interpolation is performed on images X2, X3, X4, and X5 respectively to obtain images X′2, X′3, X′4, and X′5. For example... Figure 4 As shown, using bilinear interpolation, first in adjacent pixel A 11 and A 21 Perform linear interpolation between them to find the pixel value B1 at the midpoint, and similarly find B2. Then, perform interpolation between B1 and B2 to obtain the pixel C that needs to be added.

[0075] S3-3. For images X′2, X′3, X′4, and X′5, perform erosion followed by dilation to remove small noise regions, then apply Gaussian filtering. The generation equation for the 3x3 Gaussian filter kernel used is:

[0076]

[0077] S4. Create a template image and match it with the interpolated images above and below.

[0078] S4-1. Create a shape contour template for the mark on the workpiece. Take a picture of the mark to be identified on the workpiece, and perform grayscale processing and threshold segmentation to obtain the template image X1 of the mark; then, find the contour of the template image X1 through the contour detection algorithm, name it contour1, and calculate the contour area S1;

[0079] S4-2. Find all contours of images X′2, X′3, X′4, and X′5 through contour detection, and name them contour2, contour3, contour4, and contour5. Calculate the corresponding contour areas S2, S3, S4, and S5.

[0080] S4-3. Perform shape matching between contour1 obtained in S4-1 and the four contours contour2, contour3, contour4, and contour5 obtained in S4-2, respectively, and find the four minimum matching values ​​(e.g., ...). Figure 5 The contours corresponding to the images shown are denoted as the best matching contours contour2m, contour3m, contour4m, and contour5m. Based on these best matching contours, four marks matching the corresponding images can be found. During contour matching, the excessive number of contours in images X′2, X′3, X′4, and X′5 affects matching efficiency. Therefore, an area constraint area is added before matching to reduce the number of contours involved in the matching. Specifically, based on the contour area S1 of the template image X1, the area is set as follows:

[0081] S1-500 <area<S1+500

[0082] S4-4. According to equation (4), calculate the spatial moments m of each best matching contour contour2m, contour3m, contour4m, and contour5m. ji :

[0083]

[0084] Where x and y are the coordinates of the contour, and i and j are the image order, i,j∈{0,1};

[0085] S4-5, Using m obtained from S4-4 00 m 10 m 01 Calculate the centroids center1(x,y), center2(x,y), center3(x,y), and center4(x,y) of contours contour2m, contour3m, contour4m, and contour5m respectively:

[0086]

[0087] S4-6, such as Figure 6 As shown, the best matching contours found in images X′2, X′3, X′4, and X′5 are marked using the contour drawing function, and center1(x,y), center2(x,y), center3(x,y), and center4(x,y) are drawn.

[0088] S5, Alignment of bonding points.

[0089] S5-1, Fitting Position Calculation: Based on the calibration results of S1, use the coordinate transformation matrix H to convert the image coordinates center1(x,y) obtained in S4-6 into world coordinates P1, center2(x,y) into world coordinates P2, center3(x,y) into world coordinates P3, and center4(x,y) into world coordinates P4.

[0090] S5-2, Calculate the rotation center: Rotate and photograph the upper-layer bonding workpiece at a custom angle to obtain images X6 and X7. Perform operations S3-2 to S5-1 on X6 and X7 respectively to obtain the centroids of the bonding position center5(x,y) and center6(x,y); calculate the straight line L1 formed by center3 and center4 and the straight line L2 formed by center5 and center6, and calculate the angle between lines L1 and L2 and the rotation center (e.g., Figure 7 (as shown);

[0091] S5-3. Based on the rotation center obtained in S5-2, use P1 and P2 to calculate the angle angel1 and center point P(x, y) of the lower layer of the workpiece, and use P3 and P4 to calculate the angle angel2 and center point M(x, y) of the upper layer of the workpiece.

[0092] S5-4. Rotate the upper-layer bonding workpiece by a rotation angle ΔA = angle1 - angel2, with a distance [Δx, Δy] = x P -x M The lower layer of the workpiece is moved along the X and Y axes respectively.

[0093] In this specific embodiment, the alignment result is as follows: Figure 8 As shown, by finding the best matching mark through template matching and visually positioning it, the angle difference and moving distance of the workpiece can be planned.

[0094] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A high-precision visual alignment and bonding method, characterized in that: Specifically, the following steps are included: S1. Regional calibration: Establish the relationship between image coordinates and world coordinates in different regions; S2. Nine-point repeat calibration combined with accuracy verification; S3. Perform subpixel interpolation processing on the upper and lower images of the workpiece to be bonded. S3-1. Take pictures of the left and right marks of the lower layer workpiece to be bonded to obtain images X2 and X3 respectively, and take pictures of the left and right marks of the upper layer workpiece to be bonded to obtain images X4 and X5 respectively. Perform grayscale processing and threshold segmentation on the lower layer images X2 and X3 and the upper layer images X4 and X5 respectively. S3-2. Perform subpixel interpolation on images X2, X3, X4, and X5 respectively to obtain images X′2, X′3, X′4, and X′5; S3-3. Images X′2, X′3, X′4, and X′5 are first eroded, then dilated, and finally Gaussian filtered. The generation equation for the 3x3 Gaussian filter kernel used is: S4. Create a template image and match it with the interpolated images above and below. The specific steps are as follows: S4-1. Create a shape contour template that fits the workpiece mark. Take a picture of the mark to be identified on the workpiece, and perform grayscale processing and threshold segmentation to obtain the template image X1 of the mark. Then, find the contour of the template image X1 through the contour detection algorithm, name it contour1, and calculate the contour area S1. S4-2. Find all contours of images X′2, X′3, X′4, and X′5 through contour detection, and name them contour2, contour3, contour4, and contour5. Calculate the corresponding contour areas S2, S3, S4, and S5. S4-3. Perform shape matching between the contour contour1 obtained in S4-1 and the four contours contour2, contour3, contour4, and contour5 obtained in S4-2 respectively. Find the contours corresponding to the four minimum matching values, and denote them as the best matching contours contour2m, contour3m, contour4m, and contour5m. Find the four matching marks based on the best matching contours. Add an area constraint area before matching, that is, set the area to be based on the contour area S1 of the template image X1: S1-500 <area<S1+500 S4-4. Calculate the spatial moments m of each best-matching contour contour2m, contour3m, contour4m, and contour5m according to the following formula. ji : Where x and y are the coordinate values ​​of the contour, and i and j are the image order, i,j∈{0,1}; S4-5, Using m obtained from S4-4 00 m 10 m 01 Calculate the centroids center1(x,y), center2(x,y), center3(x,y), and center4(x,y) of contours contour2m, contour3m, contour4m, and contour5m respectively: S4-6. Use the contour drawing function to mark the best matching contour found in images X′2, X′3, X′4, and X′5, and draw center1(x,y), center2(x,y), center3(x,y), and center4(x,y). S5. Alignment of bonding points; S5-1, Fitting Position Calculation: Based on the calibration results of S1, use the coordinate transformation matrix H to convert the image coordinates center1(x,y) obtained in S4-6 into world coordinates P1, center2(x,y) into world coordinates P2, center3(x,y) into world coordinates P3, and center4(x,y) into world coordinates P4. S5-2, Calculate the rotation center: Rotate and photograph the upper bonding workpiece at a custom angle to obtain images X6 and X7. Perform operations S3-2 to S5-1 on X6 and X7 respectively to obtain the centroids center5(x,y) and center6(x,y) of the bonding position; Calculate the straight line L1 formed by center3 and center4 and the straight line L2 formed by center5 and center6, and calculate the angle between lines L1 and L2 and the rotation center; S5-3. Based on the rotation center obtained in S5-2, use P1 and P2 to calculate the angle angel1 and center point P(x, y) of the lower layer of the workpiece, and use P3 and P4 to calculate the angle angel2 and center point M(x, y) of the upper layer of the workpiece. S5-4. Rotate the upper-layer bonding workpiece by a rotation angle ΔA = angle1 - angel2, with a distance [Δx, Δy] = x P -x M The lower layer of the workpiece is moved along the X and Y axes respectively.

2. The high-precision visual alignment and bonding method according to claim 1, characterized in that: The specific steps for the regional calibration are as follows: S1-1, A nine-point calibration plate designed to match the dimensions of the mechanical platform, with a 3×3 shape and an actual distance D between two points; S1-2. Fix the nine-point calibration plate onto the mechanical platform and adjust the focus according to the alignment mark height on the calibration plate so that the camera can clearly display the workpiece. S1-3, Three coordinate transformation matrices H are obtained from the regional calibration. n(n∈1,2,3) .

3. The high-precision visual alignment and bonding method according to claim 2, characterized in that: Regional coordinate transformation matrix H n(n∈1,2,3) The specific steps are as follows: First, based on the robotic arm's range of motion, the bonding station is divided into working areas 1 to 3. Calibration plates are then placed in each of the three areas, and images X are captured. The coordinates of the nine-point image (x, y) are calculated. nij y nij ), n∈(1,2,3), i∈(1,2,3), j∈(1,2,3); Secondly, for each region, read the coordinates (x, y) of any point among the nine points on the calibration board in the mechanical platform coordinate system. n11 y n11 Let this position be the center point, and calculate the mechanical coordinates (x, y) of other points. nij y nij ); Where m∈{0,1,2}; Finally, using Zheng's nine-point calibration method, the coordinate transformation matrix between each image X and the corresponding mechanical platform region was calculated, resulting in the coordinate transformation matrix H for the three regions. n(n∈1,2,3) .

4. The high-precision visual alignment and bonding method according to claim 3, characterized in that: The specific steps for the nine-point repeated calibration are as follows: S2-1. Fix the verification calibration board onto the platform and take a picture. Determine the area where the calibration board Mark is located based on the robot arm's motion coordinates, and calculate its image coordinates. Use the coordinate transformation matrix H obtained in step S1-3. n(n∈1,2,3) Find the position (x, y) of each mark on the calibration plate on the mechanical platform, and calculate the distance D1 between any two marks; S2-2. Compare D1 with D to obtain the calibration accuracy. If the accuracy requirement is met, the coordinate transformation matrix H can be used; otherwise, recalibrate.

Images