Camera inclination angle calibration method based on high-precision thermocompression bonding equipment

Abstract

The invention discloses a camera inclination angle calibration method based on high-precision thermocompression bonding equipment, and the method comprises the following steps: 1, enabling the installation angle of a camera to be parallel to a reference plane through adjusting the installation angle of the camera; 2, adjusting the inclination angle of the camera mounting angle relative to the reference plane within a set range along the X direction, wherein one unit inclination angle is adjusted each time; the average value is calculated, the leveling value serves as the proportion value Xp of the current inclination angle to the inclination angle in the X-axis direction, and therefore a curve of each inclination angle pair in the X-axis direction and the corresponding proportion value Xp in the set range is obtained; adopting the same method to obtain a curve of each inclination angle pair and the corresponding proportion value Yp in the set range along the Y direction; and 3, carrying out linear fitting on the curve of the proportion value Xp and the curve of the proportion value Yp to obtain a linear relationship between the inclination angle of the camera and the proportion value Xp and the proportion value Yp so as to realize calibration of the inclination angle of the camera.

Description

Technical Field

[0001] This invention relates to the field of chip testing technology, and specifically to a camera tilt angle calibration method based on a high-precision thermo-bonding device. Background Technology

[0002] For high-precision thermoforming (bonding) equipment for semiconductor chips, a camera is typically used for bonding alignment. Thermoforming bonding equipment for chips requires extremely high bonding accuracy; after removing the effects of vibration and temperature, the error at the visual resolution is ±0.3µm. If the camera's mounting position is tilted, a significant error will occur. For example, if the camera is required to capture a vertical image at a 90° angle, but due to the camera's tilt, the projected image will be transformed into an obtuse or acute angle. This will ultimately alter the visual rotation center compensation algorithm within the equipment, making the compensation algorithm difficult to determine and thus making the final error difficult to assess.

[0003] On the other hand, camera tilt angle causes physical distortion in the captured image, and general distortion correction algorithms use camera imaging models for correction. However, these correction algorithms do not include models of irregular distortion caused by camera tilt. Therefore, before operating high-precision thermosetting bonding equipment, the camera tilt angle needs to be detected, and then corrected using the camera's own angle adjustment device or a camera mount.

[0004] To accurately correct the camera tilt angle, some researchers have proposed measuring the camera tilt angle using captured images. For example, Chinese invention patent application CN109506626A discloses a method for measuring the camera tilt angle. In short, this method measures the camera tilt angle based on image sharpness. However, due to the poor accuracy of sharpness-based measurement methods and the requirement for small depth of field and large field of view, it is difficult to apply to the 2mm small field-of-view camera model in high-speed hot-press bonding equipment. Therefore, a new method for measuring the camera tilt angle is needed. To address this, the inventors propose the following technical solution.

[0005] Other relevant patent documents: Chinese invention patent with patent document number CN114638899B discloses a method for extracting feature points of a checkerboard calibration board using a camera and the corresponding joint calibration method. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and to propose a camera tilt angle calibration method based on a high-precision thermo-bonding device.

[0007] To solve the above technical problems, the present invention adopts the following technical solution: a camera tilt angle calibration method based on a high-precision thermosetting bonding device. In this method, the camera tilt angle is measured using a calibration plate. The camera has tilt angle adjustment along the X / Y directions with an adjustment accuracy ≤0.1°. The calibration plate surface serves as a reference surface, and the tilt angle of the camera relative to the reference surface is measured. The calibration plate uses a standard black and white checkerboard calibration plate. The calibration method includes the following steps: First, adjusting the camera's mounting angle to keep it parallel to the reference surface; Second, adjusting the camera's mounting angle relative to the reference surface within a set range along the X direction, adjusting by one unit tilt angle each time. The process involves several steps: 1) After adjusting the tilt angle by one unit, the camera continuously acquires N images of the calibration plate. The scale values ​​x1…xn of the images relative to the X-axis are calculated, and their average value is used as the scale value Xp of the current tilt angle in the X-axis direction. This yields a curve representing each tilt angle along the X-axis within the set range, corresponding to the scale value Xp. The same method is used to obtain a curve representing each tilt angle along the Y-axis within the set range, corresponding to the scale value Yp. 2) The curves of the scale values ​​Xp and Yp are fitted with straight lines to obtain a linear relationship between the camera tilt angle and the scale values ​​Xp and Yp, thus calibrating the camera tilt angle.

[0008] Furthermore, in the above technical solution, the setting range is -1° to 1°; the unit tilt angle is 0.1°.

[0009] Furthermore, in the above technical solution, N is a natural number greater than 5.

[0010] Furthermore, in the second step of the above technical solution, the following steps are used to calculate the relative proportion of the image along the X and Y directions: Step 1, after the calibration board and the camera are positioned relative to each other, the camera takes an image of the calibration board containing at least part of the checkerboard from any angle; Step 2, the captured image is processed to segment each black square in the image, and the connected component centers of all segmented black squares are extracted to obtain a series of center point sets (P). All center points are traversed to find the point closest to the image center, denoted as Pc; Step 3, preliminary positioning: with the center point Pc as the reference, the corresponding center points distributed in the center point set (P) are searched along the four directions of up, down, left, and right; Step 4, establishment of positioning points: based on Step 3, among the searched center points, the black square closest to the image edge is used to search inward along the four directions of up, down, left, and right. Step 1: Shrink the image and remove images with incomplete black squares at the edges. Determine the largest image containing the complete black scheme. Use the center point Pc as the center positioning point and find the farthest positioning points along the top, bottom, left, and right directions, for a total of five positioning points. Step 5: Select the analysis area: Based on the five positioning points determined in Step 4, calculate the center point of the black square containing each positioning point as a reference, and use it as a fine positioning point. Finally, determine five fine positioning points: center fine positioning point, top fine positioning point, bottom fine positioning point, left fine positioning point, and right fine positioning point. Step 6: Calculate the values: Based on the five fine positioning points in Step 5, calculate the distances Lup, Ldown, Lleft, and Lright between the center fine positioning point and the top, bottom, left, and right fine positioning points, respectively. Step 7: Calculate the scale: The scale value along the X direction is X = Lright / Lleft; the scale value along the Y direction is Y = Lup / ldown.

[0011] Furthermore, in the above technical solution, the method used in step 2 is as follows: First, the image is segmented by thresholding to form a binary image with only black and white; second, morphological erosion is performed on the image after thresholding to segment out each black square in the image; then, the connected component centers of all the segmented black squares are extracted to obtain a series of center point sets (P), and all center points are traversed to find the point closest to the center of the image, denoted as Pc.

[0012] Furthermore, in the above technical solution, step 3 uses the following method: Step 3a: Using the center line passing through the center point Pc as a reference, calculate the distance from all points in the center point set (P) to the center line, denoted as Pd-c; Step 3b: Find points in the four directions of up, down, left, and right that meet the following condition: Pd-c – d0 ≤ 0.1 * d0. These points are distributed in the four directions of up, down, left, and right, where d0 is the distance of the black square calculated based on the pixel.

[0013] Furthermore, in the above technical solution, after the camera tilt angle calibration method is completed, the tilt angle of the camera is determined during use using the following method: First, the camera takes N images at the current position, and calculates the scale value Xi along the X direction and the scale value Yi along the Y direction of the image. Second, based on the linear relationship between the camera tilt angle and the scale values ​​Xp and Yp, the corresponding camera tilt angle value is found. Finally, the camera tilt angle is adjusted according to the found camera tilt angle value.

[0014] After adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art: In the process of calibrating the camera tilt angle, the present invention improves the accuracy of the final calibration by processing and selecting images. Based on the method of this patent, the error can be reduced to about 0.2° after testing.

[0015] This invention is applicable to high-precision bonding equipment and can greatly reduce the impact of product photography distortion on positioning. Instruction manual illustrations

[0016] Figure 1 This is a schematic diagram showing the relative positions of the calibration plate and the camera in this invention; Figure 2 This is a schematic diagram of an image of the calibration board captured in this invention; Figure 3 This is a schematic diagram of the selected area of ​​the image in this invention. Detailed Implementation

[0017] The present invention will be further described below with reference to specific embodiments and accompanying drawings.

[0018] This invention measures the tilt angle of the camera using a calibration plate, so that the tilt angle of the camera can be accurately corrected in subsequent high-precision thermosetting bonding equipment. I. Preparatory Work

[0019] 1. For example Figure 1As shown, prepare a standard black and white checkerboard calibration board 1 and a camera 2 to be measured. For example, the squares in the calibration board 1 are 50µm in length and width, the line width is 5µm, and there are 32 squares in both the X and Y directions. The calibration board 1 can be placed arbitrarily in the field of view of the camera 2, that is, the camera 2, which needs to detect the tilt angle, can take an image containing the complete checkerboard calibration board from any angle.

[0020] The camera 2 should be adjustable in the X / Y direction with a tilt angle of ≤0.1°. The tilt angle adjustment of the camera 2 can be achieved using a camera bracket or a tilt angle adjustment device built into the camera.

[0021] If we take the checkerboard calibration plate 1 as the reference surface, the final measurement is the tilt angle of the reference surface, thereby determining the tilt angle of camera 2. II. The camera tilt angle is calibrated by measuring the tilt angle of the camera using calibration plate 1. This includes the following steps:

[0022] Step 1: Adjust the camera's mounting angle to ensure it is parallel to the reference plane. This can be done using the following method.

[0023] a. Randomly place a mark near the center of the calibration plate; b. Adjust the camera or angle table to tilt in the X direction to -1° and locate the center of the marker point c1.

[0024] c. Adjust the camera or angle table to tilt in the X direction to -1° and locate the center of the marker point c2.

[0025] d. The angle between the two X-axis is theta = atan(c2.y - c1.y, c2.x - c1.x)*180 / PI; e. Determine the tilt angle. If the tilt angle is >5°, it means the installation angle is too large. Manually rotate the camera or angle table to make them as close to 0° as possible.

[0026] Step 2: Adjust the camera mounting angle relative to the reference surface within a set range along the X direction, adjusting by one unit tilt angle each time. After each unit tilt angle adjustment, the camera continuously acquires N images of the calibration plate, calculates the proportional values ​​x1...xn of the images relative to the X direction, calculates their average value, and uses this average value as the proportional value Xp of the current tilt angle in the X-axis direction. This yields the curve of each tilt angle along the X direction relative to the corresponding proportional value Xp within the set range. The same method is used to obtain the curve of each tilt angle along the Y direction relative to the corresponding proportional value Yp within the set range.

[0027] The setting range is -1° to 1°; the unit tilt angle is 0.1°.

[0028] N is a natural number greater than 5, such as N=10.

[0029] In the second step, the following steps are used to calculate the relative scale values ​​of the image along the X and Y directions: Step 1: Combining Figure 2 As shown, after the calibration plate 1 and the camera 2 are positioned relative to each other, the camera 2 takes an image of the calibration plate 1 from any angle that contains a complete checkerboard pattern, or at least contains a partial black pattern. The more black squares included, the more accurate the final measurement value will be.

[0030] Step 2: Combining Figure 2 , Figure 3 As shown, the method used to find the black square closest to the center of the image is as follows: First, the image is segmented using thresholding. Since the original captured image is mostly grayscale, thresholding segments the image to form a binary image with only black and white values. This eliminates almost all irrelevant interference information and improves the reliability, accuracy, and speed of subsequent measurements.

[0031] Secondly, morphological erosion is performed on the image after thresholding to segment out each black square in the image.

[0032] Next, extract the centers of the connected components of all the segmented black squares to obtain a set of center points (P). Traverse all the center points and find the point closest to the center of the image, denoted as Pc.

[0033] Step 3: Preliminary positioning, the method used is as follows: Since the camera's pixels are known, the approximate distance d0 of each black checkerboard square in the image can be determined from the pixels. Using the center point Pc as a reference, the search is performed in the center point set (P) along four directions: top, bottom, left, and right, to find the corresponding center points distributed in these four directions. The following is a preferred implementation scheme: Step 3a: Using the median line passing through the center point Pc as the reference, calculate the distance from all points in the center point set (P) to the median line, denoted as Pd-c.

[0034] Step 3b: Find points in the four directions of up, down, left, and right that meet the following condition: Pd-c – d0 ≤ 0.1 *d0. These points are distributed in the four directions of up, down, left, and right.

[0035] This step finds the corresponding center points of center point Pc in the center point set (P) along the top, bottom, left, and right directions. These center points are roughly distributed along the horizontal (left-right) and vertical (top-bottom) midlines. Based on their positions, they are assigned to Pup, Pright, Pleft, and Pdown respectively. This process is repeated, starting from the midline point Pc and proceeding along the top, bottom, left, and right directions until reaching the image edge, to obtain all Pup, Pright, Pleft, and Pdown point sets in the same way.

[0036] Step 4, Establishment of Positioning Points: Based on Step 3, in the set of points Pup, Pright, Pleft, and Pdown found, the black square closest to the image edge is used to shrink inward along the four directions of up, down, left, and right to remove images with incomplete black squares at the edges. The largest image containing complete black squares is determined, and the center point Pc is used as the center positioning point. The farthest positioning points are found along the directions of up, down, left, and right, for a total of five positioning points.

[0037] This step involves removing blurred, missing, or incomplete image areas at the image edges caused by the shooting process, preserving as large an area as possible. Five positioning points are then determined within the Pup, Pright, Pleft, and Pdown point set, oriented in five directions: top, bottom, left, right, and center. The top, bottom, left, and right positioning points are located at the image edges. Determining these five points maximizes the analysis of differences caused by image tilt, using them as analytical samples to improve the accuracy of subsequent analyses.

[0038] Step 5: Select the analysis area. Based on the five positioning points determined in Step 4, calculate the center point of the black square containing each positioning point as a reference, and use it as the fine positioning point. Finally, determine five fine positioning points: center fine positioning point, upper fine positioning point, lower fine positioning point, left fine positioning point, and right fine positioning point.

[0039] The positioning points selected in step 4 are preliminary selections. Based on the five positioning points determined in step 4, the center point of the black square where each positioning point is located is calculated as a reference and used as a fine positioning point. Finally, five fine positioning points are determined: center fine positioning point, upper fine positioning point, lower fine positioning point, left fine positioning point, and right fine positioning point.

[0040] The following is combined Figure 2 , Figure 3 The diagram illustrates the process of determining the precision positioning point: Step 5a: Select the analysis area. Using the center positioning point as a reference, determine the approximate size of the selection box based on the pixel size (roughly the same size as the black scheme). Then, enlarge the selection box by 1.5 times as the analysis area of ​​the current box.

[0041] Step 5b: Perform Gaussian blur processing on the selected analysis area image; Step 5c: Extract the edges of the image after Gaussian blurring; Step 5d: Subpixel contour extraction is performed on the image extracted in step 5c; Step 5e: Hough line detection is performed on the image extracted in step 5d; Step 5f: Calculate the intersection point of the lines in step 5e; Step 5g: Calculate the center of the four points at the center of the four vertices of the black square in the analysis area; x = (x1 + x2 + x3 + x4); y = (y1 + y2 + y3 + y4).

[0042] Steps 5b-5e involve further graphic processing of the selected area to ultimately determine the four vertices of the black square within the area. Then, the center point is precisely calculated; this calculated center point is the precise positioning point mentioned earlier. Based on the above method, five precise positioning points are ultimately determined: the center precise positioning point, the upper precise positioning point, the lower precise positioning point, the left precise positioning point, and the right precise positioning point.

[0043] Step 6: Calculate the values. Based on the five precision positioning points in Step 5, calculate the distances Lup, Ldown, Lleft, and Lright between the middle precision positioning point and the upper, lower, left, and right precision positioning points, respectively.

[0044] Step 7, calculate the proportion. The scale value along the X direction is X = Lright / Lleft; The scaling factor along the Y direction is Y = Lup / ldown.

[0045] Based on the five precise positioning points mentioned above, a rough coordinate system for the image can be established. If the shooting angle is tilted, the ratios of Lup and Ldown, as well as Lleft and Lright, in this coordinate system will differ. These differences will be defined as proportional values.

[0046] Using the above method for calculating the ratio values, the ratio values ​​x1...xn of the image relative to different images at the same tilt angle along the X direction are calculated, and their average value is calculated. This average value is used as the ratio value Xp of the current tilt angle in the X-axis direction, thereby obtaining the curve of each tilt angle along the X direction with respect to the corresponding ratio value Xp within the set range; the same method is used to obtain the curve of each tilt angle along the Y direction with respect to the corresponding ratio value Yp within the set range.

[0047] Step 3: Perform linear fitting on the curves of the scale values ​​Xp and Yp to obtain the linear relationship between the camera tilt angle and the scale values ​​Xp and Yp, thereby calibrating the camera tilt angle.

[0048] After the camera tilt angle calibration method is completed, the tilt angle of the camera during actual use can be determined based on the linear relationship between the camera tilt angle and the scale values ​​Xp and Yp.

[0049] The following method is used to determine the tilt angle of the camera during use: First, the camera takes N (e.g., N=10) images at the current position. Based on the calibration method described above, the relative scale value Xi along the X direction and the relative scale value Yi along the Y direction of each image are calculated. Secondly, based on the linear relationship between the camera tilt angle and the scale values ​​Xp and Yp, find the corresponding camera tilt angle value; Finally, the camera tilt angle is adjusted based on the found camera tilt angle value.

[0050] Of course, the above description is only a specific embodiment of the present invention and is not intended to limit the scope of the present invention. All equivalent changes or modifications made to the structure, features and principles described in the claims of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A camera tilt angle calibration method based on a high-precision thermosetting bonding device, wherein the camera tilt angle is measured by a calibration plate, the camera has tilt angle adjustment along the X / Y direction with an adjustment accuracy ≤0.1°, the surface of the calibration plate is used as a reference surface, and the tilt angle of the camera relative to the reference surface is measured; the calibration plate adopts a standard black and white checkerboard calibration plate, characterized in that: The calibration method includes the following steps: Step 1: Adjust the camera's mounting angle to ensure it remains parallel to the reference plane. Step 2: Adjust the camera mounting angle relative to the reference surface within a set range along the X direction, adjusting by one unit tilt angle each time. After each unit tilt angle adjustment, the camera continuously acquires N images of the calibration plate, calculates the proportional values ​​x1...xn of the images relative to the X direction, and calculates their average value. This average value is used as the proportional value Xp of the current tilt angle in the X-axis direction, thus obtaining the curve of each tilt angle along the X direction relative to the corresponding proportional value Xp within the set range. The same method is used to obtain the curve of each tilt angle along the Y direction relative to the corresponding proportional value Yp within the set range. Step 3: Perform linear fitting on the curves of the scale values ​​Xp and Yp to obtain the linear relationship between the camera tilt angle and the scale values ​​Xp and Yp, thereby calibrating the camera tilt angle.

2. The camera tilt angle calibration method based on a high-precision hot-press bonding device according to claim 1, characterized in that: The setting range is -1° to 1°; the unit tilt angle is 0.1°.

3. The camera tilt angle calibration method based on a high-precision thermo-bonding device according to claim 1, characterized in that: N is a natural number greater than 5.

4. The camera tilt angle calibration method based on a high-precision hot-press bonding device according to claim 1, characterized in that: In the second step, the following steps are used to calculate the relative scale values ​​of the image along the X and Y directions: Step 1: After the calibration board is positioned relative to the camera, the camera takes an image of the calibration board from any angle that includes at least part of the checkerboard pattern. Step 2: Process the captured image, segment each black square in the image, extract the center of the connected component of all segmented black squares to obtain a set of center points (P), traverse all center points, find the point closest to the center of the image, and denote it as Pc. Step 3, preliminary positioning: Using the center point Pc as a reference, search for the corresponding center points distributed in the center point set (P) along the four directions of its top, bottom, left and right. Step 4, Establishment of positioning points: Based on step 3, among the searched center points, take the black square closest to the image edge and shrink inward along the four directions of up, down, left, and right respectively to remove images with incomplete black squares at the edges, determine the largest image containing the complete black scheme, take the center point Pc as the center positioning point, and find the farthest upper positioning point, lower positioning point, left positioning point, and right positioning point along the up, down, left, and right directions, for a total of five positioning points; Step 5, Select the analysis area: Based on the five positioning points determined in Step 4, calculate the center point of the black square where each positioning point is located, using each positioning point as a reference, and use it as the fine positioning point. Finally, determine the five fine positioning points: center fine positioning point, upper fine positioning point, lower fine positioning point, left fine positioning point, and right fine positioning point. Step 6: Calculate the values. Based on the five precision positioning points in Step 5, calculate the distances Lup, Ldown, Lleft, and Lright between the middle precision positioning point and the upper, lower, left, and right precision positioning points, respectively. Step 7, calculate the proportion. The scale value along the X direction is X = Lright / Lleft; The scaling factor along the Y direction is Y = Lup / ldown.

5. The camera tilt angle calibration method based on a high-precision thermo-bonding device according to claim 4, characterized in that: The method used in step 2 is as follows: First, the image is segmented using thresholding to form a binary image consisting only of black and white; Secondly, morphological erosion is performed on the image after thresholding to segment out each black square in the image. Next, extract the centers of the connected components of all the segmented black squares to obtain a set of center points (P). Traverse all the center points and find the point closest to the center of the image, denoted as Pc.

6. The camera tilt angle calibration method based on a high-precision hot-press bonding device according to claim 4, characterized in that: The method used in step 3 is as follows: Step 3a: Using the median line passing through the center point Pc as the reference, calculate the distance from all points in the center point set (P) to the median line, denoted as Pd-c; Step 3b: Find points in the top, bottom, left, and right directions that meet the following condition: Pd-c – d0 ≤ 0.1 * d0. These points are distributed in the top, bottom, left, and right directions, where d0 is the distance of the black squares calculated based on the pixels.

7. A camera tilt angle calibration method based on a high-precision thermo-bonding device according to claims 1-6, characterized in that: After the camera tilt angle calibration method is completed, the following method is used to determine the tilt angle of the camera during use: First, the camera takes N images at the current position, and calculates the scale value Xi along the X direction and the scale value Yi along the Y direction of each image. Secondly, based on the linear relationship between the camera tilt angle and the scale values ​​Xp and Yp, find the corresponding camera tilt angle value; Finally, the camera tilt angle is adjusted based on the found camera tilt angle value.

Images