A multi-layer chip stacking visual inspection method based on binocular vision

Abstract

The application discloses a kind of multilayer chip stack visual inspection methods based on binocular vision, belong to chip stack process detection technical field.The method includes: when placing first layer chip, start six-axis leveling mechanism and adjust suction head position, chip is placed to bonding table;When placing subsequent chip, the image of chip stack area is synchronously collected from double visual angle by binocular vision module, image coordinate information is converted into world coordinate information using parallax calculation and triangulation principle, and the accurate coordinate of chip to be placed position is obtained, then chip is accurately placed on bonding table in combination with chip suction head, two-axis leveling mechanism, six-axis leveling mechanism and longitudinal displacement table;The above steps are repeated to realize the accurate stacking of multilayer chip.The application has the advantages of non-contact, real-time online detection, high three-dimensional positioning accuracy and wide detection range.

Description

Technical Field

[0001] This invention relates to the field of process inspection technology for multilayer chip stacking and flip bonding processes, and more specifically to a visual inspection method for multilayer chip stacking based on binocular vision. Background Technology

[0002] Multilayer chip stacking is a key technology for achieving high-density integration in the field of advanced integrated circuit packaging. By vertically stacking multiple functional chips according to a preset logic and precisely interconnecting them, core devices with small size, low power consumption, and high integration can be constructed. This not only releases stronger performance in a limited space, but also reduces the stringent requirements on the single chip manufacturing process, achieving a balance between performance and cost.

[0003] However, with the increase in the number of stacked layers and the continuous reduction in chip spacing, the probability of defects such as interlayer alignment misalignment, bonding misalignment, and poor solder joints increases significantly. Traditional optical inspection is limited by single-view imaging, making it difficult to accurately capture deep defects and spatial position deviations in three-dimensional stacked structures, resulting in insufficient inspection accuracy and seriously affecting product yield and production efficiency. Therefore, there is an urgent need for a technical solution that can perform non-contact, high-precision, real-time online inspection of the multi-layer chip stacking process.

[0004] Visual inspection technology is a rapidly developing discipline that deeply integrates with computer vision, image processing, pattern recognition, and optical engineering. It is widely used in various fields such as electronics manufacturing, precision machining, semiconductor packaging, the automotive industry, and quality control. Its basic principle is to acquire two-dimensional or three-dimensional images of a target object using image acquisition devices such as cameras, and then use algorithms to extract and analyze features such as grayscale, contours, texture, and spatial coordinates to accurately determine the object's shape, position, size, and defects. Visual inspection technology has core advantages such as being non-contact, having high inspection speed, high accuracy, and strong repeatability.

[0005] In recent years, with the continuous iteration of artificial intelligence and machine vision technologies, visual inspection technology has shown a broader prospect for development. For example, it can achieve automatic classification and accurate identification of complex defects through deep learning algorithms, thereby improving the level of intelligent inspection; it can break through the limitations of two-dimensional inspection by combining binocular vision, structured light and other three-dimensional imaging technologies, and achieve all-round measurement of three-dimensional structures; and it can integrate multi-sensor data fusion technology to integrate multi-dimensional information such as optics and infrared, thereby further enhancing the reliability and robustness of inspection in complex scenarios.

[0006] Binocular vision technology, as an important means of 3D imaging, simultaneously acquires target images from different perspectives using two cameras, and reconstructs the 3D spatial information of the target object using parallax calculation and triangulation principles. This technology boasts significant advantages such as non-contact operation, real-time online detection, high 3D positioning accuracy, and a wide detection range, and has broad application prospects in precision manufacturing and semiconductor packaging. Summary of the Invention

[0007] The purpose of this invention is to solve the problem of inaccurate chip alignment caused by the increase of the number of stacked layers during the multi-layer chip stacking process. It provides a multi-layer chip stacking visual inspection method and system based on binocular vision, which realizes real-time capture, accurate positioning and quantitative analysis of defects during the stacking process through dual-view image fusion and three-dimensional reconstruction technology.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A visual inspection method for multi-layer chip stacks based on binocular vision includes the following steps:

[0010] Step 1: Adjust the bonding stage and chip picker to the preset positions, and move the chip picker vertically to slowly place the chip onto the bonding stage;

[0011] Step 2: When placing the second layer of chips, first move the binocular vision module above the first layer of chips, acquire chip stacking images from dual perspectives, convert the image coordinates of the stacked area into world coordinates, and obtain the precise coordinates of the chip placement position.

[0012] Step 3: Adjust the chip picker to the preset position, move the stacked chips under the chip picker according to the precise coordinates, and slowly place the subsequent chips on the previous chips;

[0013] When multiple chips need to be placed later, repeat steps two and three to achieve precise stacking of multiple layers of chips.

[0014] Before placing the chip, fine-tune the chip's horizontal orientation and correct the tilt deviation in the x and y axes to ensure the bonding accuracy between layers.

[0015] The binocular vision module simultaneously acquires two-dimensional images of multi-layer chip stacks from different perspectives.

[0016] During the process of repeating steps two and three for multi-layer chip stacking, after each layer of chips is placed, the stacked chip images are acquired in real time through a binocular vision module and three-dimensional reconstruction is performed to detect the stacking quality online. If the interlayer alignment deviation or bonding offset exceeds the standard, an alarm signal is issued and the stacking operation is suspended.

[0017] The steps to obtain precise coordinates include:

[0018] The first step is to correct the pixel coordinates and convert the corrected pixel coordinates into three-dimensional coordinates in the corresponding camera coordinate system;

[0019] The second step is to select one of the camera coordinate systems as the reference camera coordinate system and convert the three-dimensional coordinates in the other camera coordinate system into normalized three-dimensional coordinates in the reference camera coordinate system.

[0020] The third step is to establish a world coordinate system with the camera optical center corresponding to the reference camera coordinate system as the origin, and convert the normalized three-dimensional coordinates into absolute three-dimensional coordinates in the world coordinate system, so as to obtain the precise coordinates of the chip placement position.

[0021] The process of converting the image coordinates of the stacked region to world coordinates in step two is as follows:

[0022] Distortion correction is performed on the pixel coordinates of chip feature points in the binocular image, and the corrected pixel coordinates are converted into normalized coordinates in the corresponding camera coordinate system. Based on the extrinsic parameters obtained from the binocular stereo calibration, the dual-camera coordinate system is aligned with the reference camera coordinate system. Combining the camera baseline length, effective focal length, and the disparity value obtained from the matching, the feature point depth is calculated by triangulation. The normalized coordinates in the reference camera coordinate system are multiplied by the depth value and converted into absolute three-dimensional coordinates in the world coordinate system with the optical center of the reference camera as the origin, so as to obtain the precise coordinates of the chip placement position, which serves as the basis for motion control of chip alignment.

[0023] The specific process of 3D reconstruction in step two is as follows: First, the dual-view image is corrected by calibrated distortion coefficients, and stereo matching algorithm is used to complete stereo matching and generate disparity map; a hierarchical method combining sparse reconstruction of reference features and dense reconstruction of key areas is adopted to perform sparse high-precision positioning of chip reference points and dense point cloud reconstruction of key areas such as stack bonding surfaces; combined with camera intrinsic parameters and binocular extrinsic parameters, the depth is calculated by triangulation to complete the conversion from image coordinates to world coordinates, and the 3D coordinates, contour and depth features of chip stack are reconstructed, thereby quantitatively determining whether the interlayer alignment deviation and bonding offset exceed the standard.

[0024] The first step, which converts the corrected pixel coordinates into three-dimensional coordinates in the corresponding camera coordinate system, is as follows:

[0025] The pixel coordinates of the chip reference feature points in the binocular image are obtained, and the pixel coordinates are distorted according to the calibrated camera intrinsic distortion coefficients. Based on the effective focal length of the camera and the coordinates of the imaging principal point, the rectified pixel coordinates are converted into normalized three-dimensional coordinates in the corresponding camera coordinate system. The normalized three-dimensional coordinates in another camera coordinate system are unified to the reference camera coordinate system after binocular extrinsic transformation, providing a reference for subsequent depth calculation.

[0026] In the second step, the transformation process of converting the three-dimensional coordinates in another camera coordinate system into normalized three-dimensional coordinates in the reference camera coordinate system is as follows:

[0027] A reference camera coordinate system is selected as the reference. Based on the rotation matrix and translation vector obtained from the stereo calibration, the normalized 3D coordinates of the feature points in the other camera coordinate system are mapped to the reference camera coordinate system through rigid body coordinate transformation, thus completing the reference unification of the dual-view coordinates. Taking the parallel optical axis stereo configuration as an example, the two cameras do not rotate relative to each other. The other camera is translated relative to the reference camera along the baseline direction. The normalized coordinates of the chip reference points below it are unified to the reference camera coordinate system after transformation, forming a matching pair with the reference camera coordinates, providing a unified reference for subsequent depth calculation.

[0028] In the third step, the process of converting normalized 3D coordinates into absolute 3D coordinates in the world coordinate system is as follows: a world coordinate system is established with the camera optical center corresponding to the reference camera coordinate system as the origin. The feature point depth is calculated by triangulation by combining the camera baseline length, effective focal length and the matched disparity value. The normalized coordinates in the reference camera coordinate system are multiplied by the depth value to convert them into absolute 3D coordinates in the world coordinate system, thus obtaining the precise coordinates of the chip placement position.

[0029] A multi-layer chip stack visual inspection system based on binocular vision, comprising:

[0030] A bonding stage, comprising a vacuum adsorption fixing structure, is used to support and fix multilayer stacked chips, providing a stable reference platform for chip stacking and testing.

[0031] The chip pick-up tip is used to accurately pick up and transfer chips and assist in interlayer stacking positioning. The chip pick-up tip has anti-scratch and anti-static properties.

[0032] The binocular vision module includes two industrial cameras and matching lenses, which are used to simultaneously acquire two-dimensional images of the chip stack from two perspectives and convert the light signals into electrical signals and transmit them to the host computer.

[0033] The longitudinal displacement stage, connected to the base, is used to drive the chip pick-up head, industrial camera and lens to move longitudinally and adjust the imaging distance;

[0034] The two-axis leveling mechanism is connected to the chip picker head and is used to fine-tune the horizontal attitude of the chip picker head and correct the tilt deviation in the x-axis and y-axis directions.

[0035] The six-axis leveling mechanism is fixed to the bottom of the bonding stage and is used to perform all-round attitude calibration of the bonding stage.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. This invention uses dual cameras to simultaneously acquire two-dimensional images of multi-layer chip stacks from different perspectives, and reconstructs the three-dimensional spatial information of the chip stack using parallax calculation and triangulation principles. It can capture deep defects and spatial position deviations in the three-dimensional stacked structure, and has the advantage of high three-dimensional positioning accuracy.

[0038] 2. This invention uses a 3D reconstruction algorithm to convert the image coordinate information of the stacked area into world coordinate information, thereby obtaining the precise coordinates of the chip placement position and solving the problem of inaccurate chip alignment caused by the increase in the number of layers during multi-layer chip stacking.

[0039] 3. This invention uses a six-axis leveling mechanism to perform all-round attitude calibration of the overall testing platform, compensating for multi-directional positional deviations that occur during the stacking process; and uses a two-axis leveling mechanism to fine-tune the horizontal attitude of the chip, correcting the tilt deviations in the x-axis and y-axis directions, and ensuring the bonding accuracy between layers.

[0040] 4. This invention enables real-time online detection, and obtains precise coordinates through a binocular vision module when placing each layer of chips, thereby achieving precise stacking of multiple layers of chips.

[0041] 5. This invention has the advantages of being non-contact and having a wide detection range; the chip pick-up head has anti-scratch and anti-static properties to avoid damaging the chip surface; the industrial camera has high frame rate and high resolution characteristics. Attached Figure Description

[0042] The present invention will now be described in further detail with reference to the accompanying drawings and specific implementation methods.

[0043] Figure 1 This is a schematic diagram of the structure of the multi-layer chip stacked visual inspection system based on binocular vision described in this invention.

[0044] Figure 2 This is a flowchart of the multi-layer chip stacking visual inspection method based on binocular vision described in this invention.

[0045] The markings in the diagram are as follows: 1. Bonding stage; 2. Chip pick-up head; 3. Lens; 4. Industrial camera; 5. Longitudinal displacement stage; 6. Two-axis leveling mechanism; 7. Six-axis leveling mechanism. Detailed Implementation

[0046] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0047] A visual inspection method for multi-layer chip stacks based on binocular vision includes the following steps:

[0048] Step 1: When placing the first layer of chips, activate the six-axis leveling mechanism 7 and adjust the position of the chip pick-up head 2 to adjust the bonding stage 1 and the chip pick-up head 2 to the preset position. Then, activate the longitudinal displacement stage 5 to slowly place the chip onto the bonding stage 1. The binocular vision module is not fixed. After the chip has been placed on the bonding stage 1, it can be driven by the six-axis leveling mechanism 7 to achieve position changes. It can also rotate around the entire rotation axis to rotate above the bonding stage 1.

[0049] In step one, the specific process of adjusting the bonding stage 1 and the chip picker head 2 to the preset position is as follows: First, the six-axis leveling mechanism 7 is reset and initially calibrated. Using a high-precision reference mark installed on the surface of the bonding stage 1 in conjunction with a binocular vision module, the six-degree-of-freedom deviation between the current spatial pose of the bonding stage 1 and the target zero position is measured. The compensation amount for each axis is calculated based on the kinematic model of each axis, driving the six axes of the six-axis leveling mechanism 7 to sequentially perform translational and rotational compensation, so that the bonding stage 1 reaches the predetermined horizontal calibration state. Simultaneously, the preset longitudinal position of the chip picker head 2 is determined by the encoder feedback signal of the longitudinal displacement stage 5 and the limit sensor signal. During chip pickup, the chip picker head 2 uses vacuum adsorption to pick up the chip. The adsorption vacuum degree is controlled within the range of -60kPa to -80kPa. This vacuum degree range ensures stable chip pickup while avoiding stress damage to the chip surface and internal structure. The end contact surface of the chip picker head 2 is made of anti-static polymer material and has an elastic buffer structure to effectively absorb minor contact impacts during pickup and placement. After the first layer of chips is placed, images of the chip placement area are acquired using a binocular vision module. Once it is confirmed that the chip position and orientation are normal, its coordinates are used as a reference for subsequent stacking layers.

[0050] Step 2: When placing the second layer of chips, first move the binocular vision module above the first layer of chips, adjust the lens focal length 3 to focus on the chip stacking area, and simultaneously acquire chip stacking images from dual perspectives; use the semi-global block matching SGBM algorithm to calculate disparity in the acquired images and generate a sub-pixel level disparity map; based on the parallel binocular pinhole camera model, combined with the pre-calibrated camera baseline length, effective focal length, and disparity value, calculate the 3D depth information of the corresponding pixels using the triangulation formula; adopt a hierarchical 3D reconstruction method that combines sparse reconstruction of reference features with dense reconstruction of key regions, and complete coordinate transformation with camera calibration parameters to convert the image coordinate information of the stacked area into world coordinate information, obtaining the precise coordinates of the chip placement position; world coordinates refer to the global reference coordinate system that describes the absolute position of an object in real 3D space, which does not change with the camera viewpoint or imaging position, and is used to uniformly express the real spatial position and geometric relationship of all objects in the entire scene.

[0051] In step two, the parameters of the semi-global block matching (SGBM) algorithm are set as follows: the SAD matching window size is 5×5 to 9×9 pixels, the number of path directions is 4 to 8, the P1 penalty coefficient is 8 × window size² × number of image channels, the P2 penalty coefficient is 32 × window size² × number of image channels, the uniqueness ratio threshold is 5% to 15%, and the sub-pixel interpolation adopts the parabolic fitting method. The disparity search range is pre-calculated and determined based on the chip stacking height range and system parameters. Before SGBM stereo matching, the original images acquired from both perspectives are first subjected to Gaussian filtering to reduce noise and suppress camera sensor noise, adaptive histogram equalization to enhance the edge contrast of the Mark points on the chip surface, and epipolar correction based on calibration extrinsic parameters to accurately align corresponding rows of the left and right images. This reduces the two-dimensional search of stereo matching to a one-dimensional search, significantly improving matching efficiency and reducing the mismatch rate. The extraction of chip feature points adopts the Harris corner detection algorithm or the template matching algorithm based on normalized cross-correlation (NCC). The sub-pixel level feature point coordinates are automatically located in the local area of ​​the preset Mark point on the chip surface, providing accurate feature references for subsequent sparse reconstruction and dense reconstruction.

[0052] Step 3: After obtaining the coordinates, move the chip pick-up head 2 to the corresponding position and start the six-axis leveling mechanism 7 to move the stacked chips under the chip pick-up head 2. Start the longitudinal displacement stage 5 to slowly place the chips onto the bonding stage 1.

[0053] In step three, the host computer, based on the precise world coordinates calculated in step two, maps the world coordinates into target displacements for each axis of the longitudinal displacement stage 5 and the six-axis leveling mechanism 7 through inverse kinematics, generating a smooth motion trajectory and driving the actuator to translate the chip pick-up head 2 directly above the target position. Subsequently, the longitudinal displacement stage 5 is started to descend slowly and uniformly, with the descent speed controlled within the range of 0.1 mm / s to 0.5 mm / s. Just before contacting the target surface, a force sensor mounted on the top of the chip pick-up head 2 monitors the contact force between the chip and the bonding stage 1 or the surface of the preceding chip in real time. When the contact force rises and exceeds a preset contact threshold, the control system immediately stops the longitudinal feed motion and releases the vacuum suction, completing the gentle placement of the chip and avoiding overshoot that could damage the chip or cause positional displacement. After placement, the binocular vision module once again acquires dual-view images of the chip stacking area to quickly perform 3D positioning of the newly placed chip and verify the deviation between the actual placement position and the target position. If the deviation is within the allowable range, the placement data is recorded and the next layer of stacking continues. If the deviation exceeds the allowable range, the chip pick-up head 2 picks up the chip again, corrects the position based on the latest measured deviation, and places it again.

[0054] When multiple chips need to be placed later, repeat steps two and three to achieve precise stacking of multiple layers of chips.

[0055] During the repeated execution of steps two and three for multi-layer chip stacking, the working distance between the binocular vision module and the chip surface gradually increases as the number of stacked layers increases. To maintain the field of view and imaging clarity, the position of the longitudinal displacement stage 5 is automatically adjusted after each additional chip layer to compensate for the change in working distance. Simultaneously, the focal length and focus position of the lens 3 are fine-tuned based on the current working distance. Furthermore, an inter-layer cumulative error compensation strategy is employed: after each chip layer is stacked, the actual placement position of that layer is remeasured using the binocular vision module. The three-dimensional deviation between the actual position and the target position is used as a compensation amount and corrected in the target placement coordinates of the next chip layer. This prevents the deviation from accumulating with each layer, ensuring the final alignment accuracy of the multi-layer stack. When the number of stacked layers exceeds a preset value, an intermediate quality inspection layer can be inserted, pausing the regular stacking process for a comprehensive three-dimensional reconstruction and alignment accuracy assessment. Subsequent stacking operations can only continue after confirming that there is no accumulated deviation exceeding the limit.

[0056] Before placing the chip onto the bonding stage 1, the horizontal orientation of the chip is finely adjusted by the two-axis leveling mechanism 6 to correct the tilt deviation in the x-axis and y-axis directions, ensuring the bonding accuracy between layers.

[0057] In step two, the binocular camera simultaneously acquires two-dimensional images of the multi-layered chip stack from different perspectives using two industrial cameras 4. The industrial cameras 4 possess high frame rate and high resolution characteristics to meet the high-speed online inspection requirements of the chip stacking process. Different perspectives employ a parallel optical axis horizontal baseline configuration, with the optical axes of the two cameras parallel to each other and the baselines arranged horizontally. Synchronous acquisition is achieved through hardware synchronization triggering. The specifications and imaging parameters of the left and right industrial cameras 4 and their matching lenses 3 are completely identical, and the chip stacking inspection area is entirely within the common field of view of the two industrial cameras 4, ensuring the accuracy of stereo matching and 3D reconstruction.

[0058] Before the binocular vision module is put into use, it must be calibrated by the system. The calibration process includes the following steps: (1) Monocular camera calibration: Using a high-precision dot or checkerboard calibration board, images are collected from no less than 15 different perspectives by changing the pose of the calibration board. The intrinsic parameter matrix of each industrial camera 4 is calculated based on the Zhang Zhengyou calibration method, including the effective focal length, principal point coordinates, radial distortion coefficients k1, k2, k3 and tangential distortion coefficients p1, p2; (2) Binocular stereo calibration: After completing the monocular calibration, the relative poses of the two industrial cameras 4 are kept fixed, and the same calibration board is photographed at the same time. The rotation matrix R and translation vector T between the two cameras are calculated using multiple sets of images to obtain the binocular stereo extrinsic parameters; (3) Epipolar correction: The left and right images are rotated and translated using the binocular stereo extrinsic parameters so that the rows of the two image planes are aligned with the same epipolar line, ensuring that the corresponding points are located on the same image row when stereo matching is performed. Hardware synchronization triggering uses an external hardware trigger to generate a synchronization pulse signal, simultaneously triggering the exposure of two industrial cameras 4. The synchronization error does not exceed 1 microsecond, avoiding synchronization mismatch caused by moving targets. After calibration, all calibration parameters are written to the host computer configuration file and automatically loaded each time the system starts or performs a test. If the calibration parameters drift due to temperature changes or vibrations during operation, the drift can be detected by monitoring the deviation between the 3D reconstructed coordinates of the known reference point and the measured value. When the deviation exceeds a preset threshold, an online recalibration process is triggered or the drift is corrected in real time through a calibration compensation algorithm.

[0059] In step two, image data is processed through disparity calculation, triangulation principles, and 3D reconstruction algorithms. Specifically, this includes: using binocular vision to calculate disparity to obtain the 3D spatial information of the chip stack, and combining camera calibration parameters to reconstruct the 3D coordinates, contours, and depth features of the chip stack, thereby accurately judging the stacking quality and defects. The camera calibration parameters include monocular camera intrinsic parameters, binocular stereo extrinsic parameters, and lens 3 distortion coefficients. The monocular camera intrinsic parameters are the effective focal length and principal point coordinates of the camera, the binocular stereo extrinsic parameters are the rotation matrix and translation vector between the left and right industrial cameras 4, and the lens 3 distortion coefficients are the radial distortion coefficient and the tangential distortion coefficient.

[0060] The triangulation depth calculation formula is: Z = f·b / d, where Z is the depth value of the feature point along the optical axis, f is the effective focal length of the camera, b is the binocular baseline length, and d is the disparity value obtained from stereo matching; the depth direction measurement accuracy σz = (Z²·σd) / (f·b), where σd is the standard deviation of the disparity measurement. Therefore, the depth accuracy is directly proportional to the baseline length f·b and inversely proportional to the square of the measurement distance Z². In system design, the matching relationship between the baseline length and the working distance should be reasonably selected according to the accuracy requirements. The horizontal coordinates of the feature point are calculated using X = (u - cx)·Z / f ​​and Y = (v - cy)·Z / f, where (u, v) are the pixel coordinates after distortion correction, (cx, cy) are the principal imaging point coordinates, and (X, Y, Z) are the three-dimensional coordinates of the feature point in the reference camera coordinate system. The sub-pixel accuracy of the disparity value d is obtained by parabolic interpolation or isoangular straight line fitting of the integer-pixel disparity map output by the SGBM algorithm, so that the depth measurement accuracy reaches the sub-millimeter level; the confidence evaluation of stereo matching combines the ratio of the minimum to the second minimum matching cost and the left-right consistency test to filter out unreliable matching points in occluded areas and low-texture areas.

[0061] In step two, when converting the image coordinate information of the stacked area into world coordinate information, the coordinate transformation is performed based on the disparity calculation and triangulation principles of binocular vision, combined with camera calibration parameters, to obtain the precise coordinates of the chip's placement location; the coordinate transformation is performed sequentially according to the following steps:

[0062] The first step is to correct the pixel coordinates and convert the corrected pixel coordinates into three-dimensional coordinates in the corresponding camera coordinate system;

[0063] The second step, binocular coordinate system one: Based on the rotation matrix and translation vector in the binocular stereo extrinsic parameters, the coordinates of the feature points in the right industrial camera 4 coordinate system are transformed to the left industrial camera 4 coordinate system, thus completing the benchmark unification of the dual-view coordinates.

[0064] The third step is to establish a world coordinate system with the optical center of the left industrial camera 4 as the origin. Based on the parallel binocular pinhole camera model, the feature point depth is calculated by combining the camera baseline length, effective focal length and parallax value through the principle of triangulation. The normalized three-dimensional coordinates under the left industrial camera 4 coordinate system are multiplied by the depth value and converted into absolute three-dimensional coordinates under the world coordinate system to obtain the precise coordinates of the chip placement position.

[0065] Camera coordinates to world coordinates: A world coordinate system is established with the optical center of the left industrial camera as the origin. The feature point depth values ​​obtained by triangulation are combined to convert the normalized coordinates in the left industrial camera 4 coordinate system into absolute three-dimensional coordinates in the world coordinate system, and finally obtain the accurate world coordinates of the chip placement position.

[0066] The process of converting the image coordinates of the stacked region to world coordinates in step two is as follows:

[0067] The scheme uses a 1920×1200 resolution industrial camera 4 and a 24mm focal length lens 3, with a binocular baseline length of 8mm. The calibration yields an effective focal length of 6956 pixels and imaging principal point coordinates of (960, 600) pixels. For the Mark point at the upper left corner of the first-layer chip, the pixel coordinates of the left image are (1400, 520) and the pixel coordinates of the right image are (544, 520). After SGBM matching, a disparity value of 856 pixels is obtained. After completing distortion correction, pixel coordinate to camera normalized coordinate conversion, and binocular coordinate benchmark unification, the depth value of 65.0mm is calculated using triangulation formula. Finally, the absolute coordinates of this point in the world coordinate system with the optical center of the left industrial camera 4 as the origin are (4.11mm, -0.75mm, 65.0mm), which can be directly used as the motion control basis for chip alignment.

[0068] The specific process of 3D reconstruction in step two is as follows: First, the dual-view image is corrected by calibrated distortion coefficients. The SGBM algorithm is used to complete stereo matching and generate a sub-pixel disparity map. A hierarchical method combining sparse reconstruction of reference features and dense reconstruction of key areas is adopted to perform sparse high-precision positioning of chip reference points and dense point cloud reconstruction of key areas such as stacked bonding surfaces. Combining camera intrinsic parameters and binocular extrinsic parameters, the depth is calculated by triangulation to complete the conversion from image coordinates to world coordinates, and the 3D coordinates, contour and depth features of the chip stack are reconstructed. The stacked chip is detected by a parallel binocular system with a baseline of 8mm. The chip reference marker points are sparsely reconstructed and matched to obtain a disparity value of 856 pixels. Combined with the effective focal length of the camera of 6956 pixels, the world coordinates of the point are calculated to be (4.11mm, -0.75mm, 65.0mm), which can quantitatively determine whether the interlayer alignment deviation and bonding offset exceed the standard.

[0069] The first step, which converts the corrected pixel coordinates into three-dimensional coordinates in the corresponding camera coordinate system, is as follows:

[0070] If the pixel coordinates of the chip reference Mark point in the left image are (1400, 520), after distortion correction, combined with the effective focal length of 6956 pixels and the principal point coordinates (960, 600), the normalized coordinates of the left industrial camera 4 are obtained as (0.06325, -0.01150, 1). The corresponding coordinates of the right industrial camera 4 are unified to the coordinate system of the left industrial camera 4 after binocular extrinsic transformation, providing a reference for subsequent depth calculation.

[0071] In the second step, the transformation process of converting the three-dimensional coordinates in another camera coordinate system into normalized three-dimensional coordinates in the reference camera coordinate system is as follows:

[0072] The coordinate system of the left industrial camera 4 is selected as the reference. Based on the rotation matrix and translation vector obtained from the stereo calibration, the normalized 3D coordinates of the feature points in the coordinate system of the right industrial camera 4 are mapped to the coordinate system of the left industrial camera 4 through rigid body coordinate transformation, thus completing the reference unification of the dual-view coordinates. Taking the parallel optical axis stereo configuration as an example, the two industrial cameras 4 have no relative rotation, and the right industrial camera 4 is translated -8mm relative to the left industrial camera 4 along the x-axis. The normalized coordinates of the chip reference point under the right industrial camera 4 are unified to the coordinate system of the left industrial camera 4 after transformation, forming a matching pair with the coordinate system of the left industrial camera 4, providing a unified reference for subsequent depth calculation.

[0073] In the third step, the process of converting normalized 3D coordinates to absolute 3D coordinates in the world coordinate system is as follows: the baseline length of the binocular system is 8mm, the effective focal length of the camera is 6956 pixels, the disparity value of the chip reference Mark point is 856 pixels, and the depth value is obtained by triangulation as 65.0mm; the normalized coordinates (0.06325, -0.01150, 1) of the left industrial camera 4 are multiplied by the depth value to finally obtain the world coordinates of the point as (4.11mm, -0.75mm, 65.0mm).

[0074] The above embodiment employs a hardware solution with a 1920×1200 resolution industrial camera 4, a 24mm focal length lens 3, and a binocular baseline length of 8mm. This solution is suitable for small to medium-sized chip stacking scenarios with chip sizes ranging from 3mm×3mm to 25mm×25mm and a total stacking height not exceeding 20mm. For applications with larger sizes or higher stacking heights, the hardware parameters can be adjusted accordingly to meet different detection requirements: the camera resolution can be selected as 2448×2048 pixels or 4096×3072 pixels, the lens focal length can be selected as needed within the range of 16mm to 50mm, and the binocular baseline length can be adjusted within the range of 5mm to 30mm, thereby achieving a balance between field of view, detection distance, and depth measurement accuracy. The shape of the reference mark point on the chip surface can be any of a cross, circle, or rectangle, with a minimum size of not less than 0.1mm×0.1mm and a grayscale contrast with the background area of ​​the chip surface not less than 30%, to ensure the stability and reliability of sub-pixel level feature point extraction during binocular stereo matching. The Mark points can be flexibly placed at the four corners or edges of the chip according to the chip layout design, with no fewer than two to provide redundancy. Practical testing has verified that, under the conditions of the above embodiments, the 3D positioning accuracy of the chip reference markers is better than ±5μm, and the interlayer alignment accuracy is better than ±10μm, meeting the high-precision alignment requirements of multilayer chip stacking processes. For chips of different materials (such as silicon-based chips, silicon carbide chips, gallium nitride chips, etc.), due to differences in surface reflection characteristics, the brightness and angle of the illumination source and the camera exposure parameters can be adjusted accordingly to ensure that the contrast and clarity of the acquired images meet the requirements for stereo matching.

[0075] A multi-layer chip stack visual inspection system based on binocular vision, comprising:

[0076] Bonding stage 1, which includes a vacuum adsorption fixing structure, is used to support and fix multilayer stacked chips, providing a stable reference platform for chip stacking and testing.

[0077] Chip pick-up tip 2 is used to accurately pick up and transfer chips and assist in completing interlayer stacking positioning. The chip pick-up tip 2 has anti-scratch and anti-static properties.

[0078] The binocular vision module includes two industrial cameras 4 and matching lenses 3, which are used to simultaneously acquire two-dimensional images of the chip stack from two perspectives and convert the light signals into electrical signals and transmit them to the host computer.

[0079] The longitudinal displacement stage 5 is connected to the base and is used to drive the chip suction head 2, industrial camera 4 and lens 3 to move longitudinally and adjust the imaging distance.

[0080] The two-axis leveling mechanism 6 is connected to the chip pick-up head 2 and is used to fine-tune the horizontal posture of the chip pick-up head 2 and correct the tilt deviation in the x-axis and y-axis directions.

[0081] The six-axis leveling mechanism 7 is fixed to the bottom of the bonding stage 1 and is used to perform all-round attitude calibration of the bonding stage 1.

[0082] The complete workflow of this invention is as follows: After the system is powered on and initialized, the calibration process is first executed, including the zero-reset of the six-axis leveling mechanism 7, loading and online verification of calibration parameters for the binocular vision module, and horizontal calibration and flatness detection of the reference surface of the bonding stage 1. After the calibration verification is passed, the main chip stacking process is entered, sequentially executing the reference placement of the first layer of chips, the precise stacking of the second and subsequent layers of chips, and the online 3D reconstruction quality detection after each layer is stacked, until all chips are stacked. After stacking is completed, a stacking quality report is generated, recording the placement coordinates of each layer of chips, interlayer alignment deviation, and bonding offset data. If the online detection process finds that the interlayer alignment deviation exceeds the preset threshold or the bonding offset exceeds the standard, the system automatically pauses the stacking operation and issues an audible and visual alarm signal, and the operator confirms the subsequent handling method; the operator can choose to stop the current batch, manually correct the deviation and continue, or skip the current layer and mark it in the report. The specific value of the deviation threshold is determined according to the chip type, size and packaging process requirements: for conventional chips, the interlayer alignment deviation threshold is generally set to ±5μm to ±50μm, and the bonding offset threshold is generally set to ±10μm to ±100μm; for high-precision RF or optoelectronic chips, the above thresholds can be tightened to ±1μm to ±10μm.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A visual inspection method for multi-layer chip stacks based on binocular vision, characterized in that, Includes the following steps: Step 1: Adjust the bonding stage (1) and the chip picker (2) to the preset position, and move the chip picker (2) vertically to slowly place the chip onto the bonding stage (1); Step 2: When placing the second layer of chips, first move the binocular vision module above the first layer of chips, acquire chip stacking images from dual perspectives, convert the image coordinates of the stacked area into world coordinates, and obtain the precise coordinates of the chip placement position. Step 3: Adjust the chip picker (2) to the preset position, move the stacked chips to the bottom of the chip picker (2) according to the precise coordinates, and slowly place the subsequent chips on the previous chips; When multiple chips need to be placed later, repeat steps two and three to achieve precise stacking of multiple layers of chips.

2. The multi-layer chip stacking visual inspection method based on binocular vision according to claim 1, characterized in that, Before placing the chip, fine-tune the chip's horizontal orientation and correct the tilt deviation in the x and y axes to ensure the bonding accuracy between layers.

3. The multi-layer chip stacking visual inspection method based on binocular vision according to claim 1, characterized in that, The binocular vision module simultaneously acquires two-dimensional images of multi-layer chip stacks from different perspectives.

4. The multi-layer chip stacking visual inspection method based on binocular vision according to claim 1, characterized in that, During the process of repeating steps two and three to stack multiple layers of chips, after each layer of chips is placed, the stacked chip images are acquired in real time through a binocular vision module and three-dimensional reconstruction is performed to detect the stacking quality online. If the interlayer alignment deviation or bonding offset exceeds the standard, an alarm signal is issued and the stacking operation is suspended.

5. The multi-layer chip stacking visual inspection method based on binocular vision according to claim 1, characterized in that, The steps to obtain precise coordinates include: The first step is to correct the pixel coordinates and convert the corrected pixel coordinates into three-dimensional coordinates in the corresponding camera coordinate system; The second step is to select one of the camera coordinate systems as the reference camera coordinate system and convert the three-dimensional coordinates in the other camera coordinate system into normalized three-dimensional coordinates in the reference camera coordinate system. The third step is to establish a world coordinate system with the camera optical center corresponding to the reference camera coordinate system as the origin, and convert the normalized three-dimensional coordinates into absolute three-dimensional coordinates in the world coordinate system, so as to obtain the precise coordinates of the chip placement position.

6. The multi-layer chip stacking visual inspection method based on binocular vision according to claim 1, characterized in that, The process of converting the image coordinates of the stacked region to world coordinates in step two is as follows: Distortion correction is performed on the pixel coordinates of chip feature points in the binocular image, and the corrected pixel coordinates are converted into normalized coordinates in the corresponding camera coordinate system. Based on the extrinsic parameters obtained from the binocular stereo calibration, the dual-camera coordinate system is aligned with the reference camera coordinate system. Combining the camera baseline length, effective focal length, and the disparity value obtained from the matching, the feature point depth is calculated by triangulation. The normalized coordinates in the reference camera coordinate system are multiplied by the depth value and converted into absolute three-dimensional coordinates in the world coordinate system with the optical center of the reference camera as the origin, so as to obtain the precise coordinates of the chip placement position, which serves as the basis for motion control of chip alignment.

7. The multi-layer chip stacking visual inspection method based on binocular vision according to claim 4, characterized in that, The specific process of 3D reconstruction in step two is as follows: First, the dual-view image is corrected by calibrated distortion coefficients, and stereo matching algorithm is used to complete stereo matching and generate disparity map; a hierarchical method combining sparse reconstruction of reference features and dense reconstruction of key areas is adopted to perform sparse high-precision positioning of chip reference points and dense point cloud reconstruction of key areas such as stack bonding surfaces; combined with camera intrinsic parameters and binocular extrinsic parameters, the depth is calculated by triangulation to complete the conversion from image coordinates to world coordinates, and the 3D coordinates, contour and depth features of chip stack are reconstructed, thereby quantitatively determining whether the interlayer alignment deviation and bonding offset exceed the standard.

8. The multi-layer chip stacking visual inspection method based on binocular vision according to claim 5, characterized in that, The first step, which converts the corrected pixel coordinates into three-dimensional coordinates in the corresponding camera coordinate system, is as follows: The pixel coordinates of the chip reference feature points in the binocular image are obtained, and the pixel coordinates are distorted according to the calibrated camera intrinsic distortion coefficients. Based on the effective focal length of the camera and the coordinates of the imaging principal point, the rectified pixel coordinates are converted into normalized three-dimensional coordinates in the corresponding camera coordinate system. The normalized three-dimensional coordinates in another camera coordinate system are unified to the reference camera coordinate system after binocular extrinsic transformation, providing a reference for subsequent depth calculation.

9. The multi-layer chip stacking visual inspection method based on binocular vision according to claim 5, characterized in that, In the second step, the transformation process of converting the three-dimensional coordinates in another camera coordinate system into normalized three-dimensional coordinates in the reference camera coordinate system is as follows: The reference camera coordinate system is selected as the reference. Based on the rotation matrix and translation vector obtained by binocular stereo calibration, the normalized three-dimensional coordinates of feature points in another camera coordinate system are mapped to the reference camera coordinate system through rigid body coordinate transformation, thus completing the reference unification of the dual-view coordinates. Taking the parallel optical axis binocular configuration as an example, the two cameras do not rotate relative to each other. The other camera translates relative to the reference camera along the baseline direction. The normalized coordinates of the chip reference point are transformed and unified to the reference camera coordinate system, forming a matching pair with the reference camera coordinates, providing a unified reference for subsequent depth calculation. In the third step, the process of converting normalized 3D coordinates into absolute 3D coordinates in the world coordinate system is as follows: a world coordinate system is established with the camera optical center corresponding to the reference camera coordinate system as the origin. The feature point depth is calculated by triangulation by combining the camera baseline length, effective focal length and the matched disparity value. The normalized coordinates in the reference camera coordinate system are multiplied by the depth value to convert them into absolute 3D coordinates in the world coordinate system, thus obtaining the precise coordinates of the chip placement position.

10. A multi-layer chip stacked visual inspection system based on binocular vision, characterized in that, include: Bonding stage (1), the bonding stage (1) includes a vacuum adsorption fixing structure for carrying and fixing multilayer stacked chips, providing a stable reference platform for chip stacking and testing; The chip pick-up tip (2) is used to accurately pick up and transfer chips and assist in completing interlayer stacking positioning. The chip pick-up tip (2) has anti-scratch and anti-static properties. The binocular vision module includes two industrial cameras (4) and matching lenses (3), which are used to simultaneously acquire two-dimensional images of the chip stack from two perspectives and convert the light signals into electrical signals and transmit them to the host computer. The longitudinal displacement stage (5) is connected to the base and is used to drive the chip pick-up head (2), industrial camera (4) and lens (3) to move longitudinally and adjust the imaging distance. The two-axis leveling mechanism (6) is connected to the chip pick-up head (2) and is used to fine-tune the horizontal posture of the chip pick-up head (2) and correct the tilt deviation in the x-axis and y-axis directions. A six-axis leveling mechanism (7) is fixed at the bottom of the bonding stage (1) and is used to perform all-round attitude calibration of the bonding stage (1).

Images