Chip bonding wire three-dimensional measurement method and system based on inner cavity reflection camera system

Abstract

The invention provides a chip bonding wire three-dimensional measurement method and system based on an inner cavity reflection camera system, and relates to the technical field of three-dimensional measurement. The method comprises the following steps: performing one-time exposure imaging on a chip bonding wire to be detected through an inner cavity reflection camera system to obtain at least three mutually independent visual angle images in the same frame; performing brightness normalization compensation and geometric calibration on the visual angle image to obtain a visual angle data set with uniform brightness and correlation; pairing the view angle data sets in pairs, extracting parallax information by using a feature matching algorithm based on optical flow, and fusing multiple pairs of parallax to generate a lead three-dimensional point cloud; performing noise filtering and curved surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional grid model of a lead; and performing texture mapping on the three-dimensional grid model by adopting any visual angle in the visual angle image or the combination of the visual angles, and outputting a lead three-dimensional reconstruction result with textures. According to the invention, high-precision, large-depth-of-field and rapid three-dimensional measurement of the chip bonding wire is realized.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional measurement technology, and in particular to a chip bonding lead three-dimensional measurement method and system based on an intracavity reflection camera system. Background Art

[0002] With the widespread adoption of high-end chips such as automotive-grade and multi-layer interconnect packaging, the number of bonding wire layers has rapidly expanded from the traditional one or two to four or even five, placing unprecedented demands on the accuracy and efficiency of micron-level 3D measurement. The industry currently relies on solutions such as structured light / binocular vision, laser line scanning, white light interferometry, spectral confocal, and light field cameras for 3D inspection. However, these solutions are generally limited by factors such as depth of field, occlusion, scanning speed, and cost, making it difficult to achieve the four core requirements of microscale, large depth of field, large area, and high throughput.

[0003] To achieve both measurement accuracy and production line speed, inspection equipment is evolving toward single-shot exposure, multi-view acquisition, lightweight optics, and algorithm-driven approaches. Data processing emphasizes a combination of hardware and software, including dual calibration of brightness and geometry, deep learning and optical flow fusion, and integrated point cloud-mesh reconstruction. This aims to achieve large depth of field and online three-dimensional reconstruction of multi-layer gold wires while keeping hardware costs under control.

[0004] The existing technology has the following defects

[0005] (1) The structured light / binocular method has a small depth of field and is easily blocked by the upper wires;

[0006] (2) Laser lines and white light interferometry require line-by-line or point-by-point scanning, which is slow and costly;

[0007] (3) Spectral confocal and light field cameras are still at the laboratory stage at micron-level resolution, and their resolution or real-time performance is insufficient.

[0008] Therefore, the industry urgently needs a new three-dimensional measurement method for chip bonding wires that can obtain multi-view information in a single exposure and is combined with efficient calibration and reconstruction algorithms. Summary of the Invention

[0009] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a chip bonding wire three-dimensional measurement method and system based on an intracavity reflection camera system, which combines single-exposure multi-view acquisition with precise calibration and efficient reconstruction to achieve high-precision, large depth of field, and fast three-dimensional measurement of chip bonding wires.

[0010] To achieve the above object, the present invention provides the following solutions:

[0011] A three-dimensional measurement method for chip bonding wires based on an intracavity reflection camera system, comprising:

[0012] Performing a single exposure imaging of the bonding wires of the chip to be tested by an intracavity reflection camera system to obtain at least three independent perspective images in the same frame;

[0013] Performing brightness normalization compensation and geometric calibration on the perspective images to obtain a perspective data set with uniform brightness and mutual correlation;

[0014] Pairing the perspective data sets in pairs, extracting disparity information using a feature matching algorithm based on optical flow, and fusing multiple pairs of disparities to generate a lead 3D point cloud;

[0015] Performing noise filtering and surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional mesh model of the lead;

[0016] Texture mapping is performed on the three-dimensional mesh model using any one of the perspectives in the perspective images or a combination thereof, and a three-dimensional reconstruction result of the lead with texture is output.

[0017] Preferably, the bonding wires of the chip to be tested are imaged once by an intracavity reflection camera system to obtain at least three independent perspective images in the same frame, including:

[0018] Fix the chip to be tested on the stage and adjust the stage along the main optical axis to align with the object conjugate plane of the intracavity reflection camera system so that the bonding wire is in the optimal focal plane;

[0019] Synchronously triggering the image sensor shutter within a single exposure cycle and driving the first plane reflector, the second plane reflector, and the half-mirror disposed in the inner cavity so that the light beam from the area to be measured forms a multi-field light beam along at least three different reflection paths;

[0020] Using a combination of a microscope main lens and a one-time magnification relay lens, the multi-field light beam and the direct field of view are simultaneously projected onto the same image sensor surface to obtain an original image frame containing multiple sub-views;

[0021] The original image frame is subjected to sub-view decoupling and storage according to a preset field of view segmentation template to obtain at least three non-overlapping and independent perspective images.

[0022] Preferably, performing brightness normalization compensation and geometric calibration on the viewing angle images to obtain a viewing angle data set with uniform brightness and mutual correlation includes:

[0023] Place a Lambert white reference plate at the workstation where the chip to be tested is located, and obtain uniformly illuminated reference images at each viewing angle.

[0024] Calculating a pixel-level gain-offset correction matrix according to the reference image, and performing pixel-by-pixel brightness correction on the original image of each viewing angle based on the pixel-level gain-offset correction matrix to obtain a brightness normalized image;

[0025] Place a flat stereo calibration plate with a checkerboard or coded dots at the same workstation and collect calibration image sequences from various viewpoints.

[0026] The calibration image sequence is used to solve the intrinsic parameter matrix and extrinsic parameter pose of each view, perform lens distortion removal and epipolar correction, resample the brightness normalized image to a common image plane, and generate a geometrically calibrated view data set.

[0027] Preferably, the calculation formula of the pixel-level gain-offset correction matrix is:

[0028]

[0029] Wherein, M(u,v) is the pixel-level gain-offset correction matrix; F(u,v) is the flat-field pixel intensity obtained under the Lambert white reference plate condition; is the global average value of F(u,v), which is used to provide a global energy benchmark; D(u,v) is the dark field reference pixel intensity, which reflects the sensor dark current and fixed pattern noise; is the global average value of D(u,v), which is used to unify the dark field baseline;

[0030] The correction formula for the pixel-by-pixel brightness correction is:

[0031]

[0032] is the grayscale value of the corresponding pixel in the corrected brightness normalized image; I(u,v) is the pixel intensity to be corrected in the original image of each viewing angle.

[0033] Preferably, the calibration image sequence is used to solve the intrinsic parameter matrix and extrinsic parameter pose of each view, lens distortion removal and epipolar correction are performed, and the brightness normalized image is resampled to a common image plane to generate a geometrically calibrated view data set, including:

[0034] Capturing the calibration image sequence for each viewing angle; the calibration image sequence is a plurality of frames of images captured continuously when the same calibration plate is in multiple known spatial postures; the calibration plate surface has a checkerboard or coded dot pattern;

[0035] Based on minimizing the reprojection error of the feature points in the calibration image sequence, solving the camera intrinsic parameter matrix and the radial and tangential distortion coefficients of each viewpoint respectively;

[0036] Select a view angle as a reference view angle, and use the coordinates of the corresponding feature points in the calibration image of the same frame to calculate the extrinsic pose parameters of the remaining view angles relative to the reference view angle;

[0037] Performing inverse distortion mapping on the brightness normalized image of each viewing angle according to the distortion coefficient to obtain a dedistorted image;

[0038] Calculating an epipolar correction homography matrix from each viewpoint to a common virtual image plane according to the intrinsic parameter matrix and the extrinsic parameter posture parameters, and performing homography transformation and bilinear interpolation resampling on the dedistorted image;

[0039] The resampled view images are cropped into common overlapping view areas according to the view index to generate a geometrically calibrated view dataset.

[0040] Preferably, based on minimizing the reprojection error of the feature points in the calibration image sequence, solving the camera intrinsic parameter matrix and the radial and tangential distortion coefficients of each viewing angle respectively includes:

[0041] Construct a reprojection error function E; the expression of the reprojection error function E is:

[0042] in,

[0043] E is the global reprojection error; M is the number of calibration images; N is the number of calibration feature points detected in each image; is the measured pixel coordinate of the nth feature point in the mth image; is the calculated theoretical projection pixel coordinate; is the camera intrinsic parameter matrix; f x is the equivalent focal length in the x direction; f y is the equivalent focal length in the y direction; s is the tilt of the principal point; u0, v0 are the principal point coordinates; k = [k1, k2, k3, p1, p2] T is the distortion coefficient vector; k1, k2, k3 are the first, second, and third order radial distortion coefficients; p1, p2 are the first and second order tangential distortion coefficients; R m is the rotation matrix that transforms the world coordinates to the camera coordinate system of the mth image; t m is the translation vector of the same transformation; P n =[X n ,Y n ,Z n ] T is the three-dimensional coordinate of the nth calibration feature point in the calibration plate coordinate system; n ,y n x is the undistorted pixel coordinate obtained by projection and normalization with the optical axis as the center; d ,y d is the pixel coordinate after adding radial-tangential distortion; Z c is the depth of the three-dimensional point in the camera coordinate system; r2 is the square of the distance from the normalized coordinate to the optical axis;

[0044] By using nonlinear least squares iterative optimization on the reprojection error function E, the intrinsic parameter K and distortion coefficient k of each view are obtained at the same time, and the posture parameter R of each image is updated synchronously during the iteration m , t m , to ensure the best agreement between the projected model and the measured feature points.

[0045] Preferably, the perspective data sets are paired in pairs, disparity information is extracted using a feature matching algorithm based on optical flow, and multiple pairs of disparities are fused to generate a lead 3D point cloud, including:

[0046] For the geometrically calibrated view dataset, an ordered view pair list is constructed according to the view number, and an epipolar search window is established for each pair of images in the reference view-auxiliary view manner;

[0047] For each view pair in the ordered view pair list, a pyramid cascade optical flow algorithm is used to perform dense pixel matching from coarse to fine to obtain a sub-pixel optical flow field, and epipolar constraints of the same pair of views are used to eliminate flow vectors that do not satisfy homography consistency;

[0048] The filtered sub-pixel optical flow field is converted into a disparity map, and the corresponding pixels are triangulated line-by-line using the intrinsic-extrinsic matrix of each view to generate a local 3D point cloud represented in the reference view coordinate system.

[0049] Perform voxel binning and normal vector consistency detection on the local 3D point cloud of all view pairs, and remove outliers with significantly low spatial density or normal vector deviation exceeding a preset threshold;

[0050] The retained local three-dimensional point cloud is overlapped and fused according to the reference perspective, and the repeated points are merged and the weighted average depth value is used to form a unified and deduplicated three-dimensional point cloud of the lead.

[0051] Preferably, performing noise filtering and surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional mesh model of the lead includes:

[0052] For each point in the three-dimensional point cloud, the average Euclidean distance of its k nearest neighbors is calculated. If the average Euclidean distance is greater than the global distance average plus α times the standard deviation, the point is determined to be an outlier and deleted, thereby obtaining a purified point cloud.

[0053] The purified point cloud is divided into two parts with a side length of l v For each voxel, the geometric centroid of all points belonging to the voxel is taken as the representative point to form a sparse point cloud P with uniform density. s ;

[0054] In the sparse point cloud P s The normal vector of each representative point is estimated by the k-nearest neighbor method and the moving least squares surface fitting method is used to locally smooth the coordinates and normal vectors of the representative points to obtain a smooth point cloud P s,MLS ;

[0055] Smooth point cloud P s,MLS and its normal vector as input, set the octree depth d and solve the Poisson equation to generate the initial triangular mesh M0;

[0056] Perform Laplace smoothing on the initial mesh M0 and combine it with edge folding and edge flipping simplification operations to reduce the number of facets while keeping the curvature change less than the threshold κ, obtaining the optimized mesh M1;

[0057] Detect the area of ​​the optimized grid M1 that is less than the threshold A min The holes are automatically filled, and then the topological connectivity and normal consistency are checked. If there are any anomalies, local triangulation is performed, and finally the continuous three-dimensional mesh model M with topologically closed, connected and smooth leads is output. final .

[0058] Preferably, texture mapping is performed on the three-dimensional mesh model using any one of the perspectives or a combination thereof in the perspective images to output a textured three-dimensional reconstruction result of the lead wire, including:

[0059] Perform fast visibility statistics on all view images after geometric calibration, sort them by the number of visible mesh faces, and automatically select the single view or two-view combination with the best visibility as the texture source;

[0060] Perspectively projecting the vertices of the three-dimensional mesh model onto a pixel plane of a selected viewing angle to obtain corresponding UV coordinates;

[0061] If it is a two-view combination, for each triangular face in the 3D mesh model, the angle between the face normal vector and the sight line vectors of the two cameras' optical centers pointing to the face is calculated, and the triangular face is assigned to the view with the smallest angle, and only the UV coordinates corresponding to the view are retained;

[0062] The pixel color is directly sampled from the selected perspective image according to the generated UV coordinates, written into a single texture map, and the texture map is bound to the three-dimensional mesh model to output the textured lead three-dimensional reconstruction result.

[0063] A chip bonding wire 3D measurement system based on an intracavity reflection camera system, comprising:

[0064] A multi-view intracavity imaging module is used to perform a single exposure imaging of the bonding wires of the chip to be tested using an intracavity reflection camera system, thereby obtaining at least three independent viewpoint images in the same frame;

[0065] An image normalization and geometric calibration module, configured to perform brightness normalization compensation and geometric calibration on the view images to obtain a view data set with uniform brightness and correlation;

[0066] A disparity extraction and point cloud fusion module is used to pair the view data sets in pairs, extract disparity information using a feature matching algorithm based on optical flow, and fuse multiple pairs of disparities to generate a lead 3D point cloud;

[0067] A point cloud filtering and surface reconstruction module, configured to perform noise filtering and surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional mesh model of the lead;

[0068] The texture mapping and model output module is used to perform texture mapping on the three-dimensional mesh model by adopting any one of the perspectives in the perspective images or a combination thereof, and output a three-dimensional reconstruction result of the lead with texture.

[0069] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0070] The present invention uses an intracavity reflection camera to synchronously capture at least three non-overlapping perspective images in a single exposure, avoiding mechanical jitter and alignment errors caused by multiple shots. At the same time, brightness normalization compensation and geometric calibration are used to map each perspective to a unified luminosity and image plane, achieving high consistency across perspectives. Dense disparity calculation and multi-pair disparity fusion based on pyramid optical flow can obtain sub-pixel accurate three-dimensional point clouds under large depth of field and complex occlusion conditions, improving the detection integrity of multi-layer bond wire microstructures. Subsequently, through statistical denoising, voxel averaging, and Poisson surface reconstruction, the sparse noisy point cloud is converted into a topologically connected, surface-continuous mesh model, providing a reliable geometric basis for dimensional measurement, stress analysis, and defect assessment. Finally, texture mapping is performed at the optimal visibility perspective to generate a textured model with consistent color and rich details, facilitating manual inspection and automated comparison. Overall, the method takes into account multiple requirements such as high resolution, online speed, and cost control. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0072] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0073] Figure 2 A schematic diagram of the system structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0075] The purpose of the present invention is to provide a chip bonding wire three-dimensional measurement method and system based on an intracavity reflection camera system, which combines single-exposure multi-view acquisition with precise calibration and efficient reconstruction to achieve high-precision, large depth of field, and fast three-dimensional measurement of chip bonding wires.

[0076] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0077] Figure 1 A flow chart of the method provided in the embodiment of the present invention is shown in FIG. Figure 1 As shown, the present invention provides a three-dimensional measurement method for chip bonding wires based on an intracavity reflection camera system, comprising:

[0078] Step 100: Performing a single exposure imaging on the bonding wires of the chip to be tested using an intracavity reflection camera system to obtain at least three independent perspective images in the same frame;

[0079] Step 200: performing brightness normalization compensation and geometric calibration on the view image to obtain a view data set with uniform brightness and mutual correlation;

[0080] Step 300: Pair the view data sets in pairs, extract disparity information using a feature matching algorithm based on optical flow, and fuse multiple pairs of disparities to generate a lead 3D point cloud;

[0081] Step 400: performing noise filtering and surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional mesh model of the lead;

[0082] Step 500: Use any one of the perspectives in the perspective image or a combination thereof to perform texture mapping on the three-dimensional mesh model, and output a textured three-dimensional reconstruction result of the lead.

[0083] Preferably, the bonding wires of the chip to be tested are imaged once by an intracavity reflection camera system to obtain at least three independent perspective images in the same frame, including:

[0084] Fix the chip to be tested on the stage and adjust the stage along the main optical axis to align with the object conjugate plane of the intracavity reflection camera system so that the bonding wire is in the optimal focal plane;

[0085] Synchronously triggering the image sensor shutter within a single exposure cycle and driving the first plane reflector, the second plane reflector, and the half-mirror disposed in the inner cavity so that the light beam from the area to be measured forms a multi-field light beam along at least three different reflection paths;

[0086] Using a combination of a microscope main lens and a one-time magnification relay lens, the multi-field light beam and the direct field of view are simultaneously projected onto the same image sensor surface to obtain an original image frame containing multiple sub-views;

[0087] The original image frame is subjected to sub-view decoupling and storage according to a preset field of view segmentation template to obtain at least three non-overlapping and independent perspective images.

[0088] In a preferred embodiment, the chip under test is mounted on a high-precision five-degree-of-freedom micro-displacement stage and fine-tuned along the optical axis to ensure that the geometric center of the bond wire is coplanar with the object-side conjugate plane of the intracavity reflection camera system, thereby locating the optimal focal plane of the microscope's main lens. The camera system consists of a main microscope lens, a single-magnification relay lens, two plane mirrors symmetrically arranged about the main optical axis, and a semi-transparent mirror located on the optical axis. The relay lens is coaxially mounted with the main lens, and the distance between the mirrors is calibrated to ensure focal overlap. The two plane mirrors are symmetrically positioned at an angle of approximately 25 degrees, and the semi-transparent mirror is positioned perpendicular to the main optical axis. This allows the direct beam from the bond wire and the double-reflected left and right beams to be combined in the image direction to form three independent fields of view. During a single exposure, the image sensor's global shutter is synchronously triggered by a control system, while a voltage is applied to the two mirrors and the semi-transparent mirror to maintain their position. This ensures that the three optical paths are stable and in phase throughout the exposure, avoiding sub-pixel errors caused by any optical path jitter.

[0089] After the exposure is completed, the microscope main lens and the relay lens will simultaneously image the direct field of view and the two reflected fields of view onto the same image sensor surface, forming an original image frame containing three sub-views. The system pre-solidifies the field of view segmentation template in the logic device. The template divides the original image frame into three non-overlapping areas of the upper direct view, the left reflected view, and the right reflected view according to the optical axis coordinates, and writes them into the cache in a predetermined order and saves them as independent image files. The boundary coordinates of the segmentation template are obtained through grid target calibration when assembled at the factory, which can ensure that the three areas completely cover their respective fields of view without cross-overlap. With the help of the above process, at least three independent perspective images can be synchronously acquired in a single frame, providing temporally and spatially consistent multi-perspective raw data for subsequent brightness normalization, geometric calibration, and three-dimensional reconstruction.

[0090] Preferably, performing brightness normalization compensation and geometric calibration on the viewing angle images to obtain a viewing angle data set with uniform brightness and mutual correlation includes:

[0091] Place a Lambert white reference plate at the workstation where the chip to be tested is located, and obtain uniformly illuminated reference images at each viewing angle.

[0092] Calculating a pixel-level gain-offset correction matrix according to the reference image, and performing pixel-by-pixel brightness correction on the original image of each viewing angle based on the pixel-level gain-offset correction matrix to obtain a brightness normalized image;

[0093] Place a flat stereo calibration plate with a checkerboard or coded dots at the same workstation and collect calibration image sequences from various viewpoints.

[0094] The calibration image sequence is used to solve the intrinsic parameter matrix and extrinsic parameter pose of each view, perform lens distortion removal and epipolar correction, resample the brightness normalized image to a common image plane, and generate a geometrically calibrated view data set.

[0095] In this embodiment, a Lambert white reference plate preheated in a mirror black box is first placed at the same workstation as the chip to be tested, with its normal substantially parallel to the optical axis. The camera system is then triggered to capture a three-view flat-field reference image, and the exposure parameters are kept consistent with subsequent measurements. The system then calls the dark field database embedded in the FPGA, reads the dark field template at the corresponding exposure time, and performs pixel-by-pixel dark current subtraction on the flat-field reference image. Subsequently, the GPU is used to calculate the dark current. The formula generates a gain-offset correction matrix in real time, where F(u,v) is the flat-field pixel intensity and D(u,v) is the dark-field pixel intensity. and is the global average of each channel. After being written to the on-chip video memory, this matrix directly participates in the image flow pipeline, performing pixel-by-pixel multiplication-addition compensation on the three-view original images. This outputs a brightness-normalized image that eliminates uneven illumination, inconsistent gain, and fixed pattern noise at the pixel level.

[0096] After brightness normalization, the assembled checkerboard calibration plate is placed on the same working plane as the chip. A multi-pose calibration image sequence is continuously acquired, and the corner pixel coordinates are detected using OpenCL acceleration. Based on nonlinear optimization that minimizes reprojection error, the intrinsic parameter matrix, radial and tangential distortion coefficients, and the relative extrinsic parameter poses of each view are simultaneously calculated. The system uses the calculated distortion coefficients to perform inverse distortion mapping on the brightness-normalized image. Then, using the intermediate view as a reference, the rectification transformation matrix from the remaining views to the common virtual image plane is calculated based on the homography relationship, and resampling is performed using bilinear interpolation within the FPGA core. Finally, the common overlapping areas of the three rectified images are cropped and synchronously written to the cache, forming a geometrically calibrated view dataset with uniform brightness, consistent coordinates, and epipolar alignment, providing a unified image foundation for subsequent optical flow disparity calculations.

[0097] Preferably, the calculation formula of the pixel-level gain-offset correction matrix is:

[0098]

[0099] Wherein, M(u,v) is the pixel-level gain-offset correction matrix; F(u,v) is the flat-field pixel intensity obtained under the Lambert white reference plate condition; is the global average value of F(u,v), which is used to provide a global energy benchmark; D(u,v) is the dark field reference pixel intensity, which reflects the sensor dark current and fixed pattern noise; is the global average value of D(u,v), which is used to unify the dark field baseline;

[0100] The correction formula for the pixel-by-pixel brightness correction is:

[0101]

[0102] is the grayscale value of the corresponding pixel in the corrected brightness normalized image; I(u,v) is the pixel intensity to be corrected in the original image of each viewing angle.

[0103] In this preferred solution, the system first places a Lambert white reference plate at the site to be measured and captures a flat-field reference image, while simultaneously calling a dark-field template with the same exposure time. For each pixel coordinate position, the grayscale value F(u,v) under flat-field conditions and the grayscale value D(u,v) under dark-field conditions can be read. Subsequently, the channel average of the entire flat-field image and dark-field image is calculated to obtain the global energy reference. and dark field baseline The physical meaning of the gain-offset correction matrix M(u,v) is: if the response of a pixel in the flat field is lower than the global average, a gain greater than 1 is assigned to it; if it is higher than the average, a gain less than 1 is assigned to it; at the same time, D(u,v) and This matrix is ​​generated in real time on a pixel-by-pixel basis in the GPU and fully cached in on-chip memory for subsequent image stream calls.

[0104] After the matrix calculation is completed, the dark field value D(u,v) is first deducted from the intensity of each pixel I(u,v) of any subsequent original measurement frame, and then multiplied by the gain-offset coefficient M(u,v) at the corresponding position to obtain the grayscale value after brightness normalization. In this process, D(u,v) ensures that the dark current and column fixed noise are completely eliminated, while M(u,v) compresses or stretches the effective dynamic range of the pixel to the global energy reference. After this pixel-by-pixel correction, the brightness differences between different viewing angles and different areas are smoothed out. The resulting brightness normalized image not only removes fixed pattern noise but also provides input data with consistent luminosity and sufficient contrast for subsequent geometric calibration and disparity calculation.

[0105] Preferably, the calibration image sequence is used to solve the intrinsic parameter matrix and extrinsic parameter pose of each view, lens distortion removal and epipolar correction are performed, and the brightness normalized image is resampled to a common image plane to generate a geometrically calibrated view data set, including:

[0106] Capturing the calibration image sequence for each viewing angle; the calibration image sequence is a plurality of frames of images captured continuously when the same calibration plate is in multiple known spatial postures; the calibration plate surface has a checkerboard or coded dot pattern;

[0107] Based on minimizing the reprojection error of the feature points in the calibration image sequence, solving the camera intrinsic parameter matrix and the radial and tangential distortion coefficients of each viewpoint respectively;

[0108] Select a view angle as a reference view angle, and use the coordinates of the corresponding feature points in the calibration image of the same frame to calculate the extrinsic pose parameters of the remaining view angles relative to the reference view angle;

[0109] Performing inverse distortion mapping on the brightness normalized image of each viewing angle according to the distortion coefficient to obtain a dedistorted image;

[0110] Calculating an epipolar correction homography matrix from each viewpoint to a common virtual image plane according to the intrinsic parameter matrix and the extrinsic parameter posture parameters, and performing homography transformation and bilinear interpolation resampling on the dedistorted image;

[0111] The resampled view images are cropped into common overlapping view areas according to the view index to generate a geometrically calibrated view dataset.

[0112] In this implementation, a planar calibration plate with a checkerboard pattern is first placed at the workstation where the chip to be tested resides. A robotic arm then sequentially positions the plate in multiple known poses within the three degrees of freedom (pitch, yaw, and translation) to cover different depths and angles in the camera's field of view. The camera system simultaneously captures brightness-normalized images from three perspectives at each pose, automatically detecting the sub-pixel coordinates of the checkerboard corners or coded dots, and pairing the pixel coordinates of the same physical corner point in different image frames with the corresponding spatial coordinates. Subsequently, a calibration algorithm based on least-squares optimization of the reprojection error is used to independently calculate internal parameters such as focal length, principal point position, and shear coefficient, as well as first- and second-order radial and tangential distortion coefficients, for each perspective. The perspective at the center of the field of view is then used as a reference. The position of the three-dimensional corner points in that perspective's coordinate system is then rigidly aligned with the pixel observations of the remaining perspectives to obtain the rotation matrix and translation vector for each auxiliary perspective relative to the reference perspective.

[0113] After completing the parameter solution, the system calls the distortion reverse mapping lookup table to perform pixel-level distortion correction on the three-view brightness normalized image and output an undistorted image. Then, the epipolar correction homography matrix from each view to the common virtual image plane is calculated based on the intrinsic and extrinsic parameters, and bilinear interpolation is used on the FPGA image pipeline to complete the rectification resampling. The rectified image has a unified pixel coordinate system and row-aligned epipolar lines, which can be directly used for dense disparity search. Finally, the software module detects the common field of view boundary of the three rectified images, crops the redundant areas, retains only the overlapping parts of the three views, and writes them into the cache according to the view index, forming a geometrically calibrated view data set, which provides an optically consistent and coordinate-aligned input basis for subsequent optical flow matching and 3D point cloud reconstruction.

[0114] Preferably, based on minimizing the reprojection error of the feature points in the calibration image sequence, solving the camera intrinsic parameter matrix and the radial and tangential distortion coefficients of each viewing angle respectively includes:

[0115] Construct a reprojection error function E; the expression of the reprojection error function E is:

[0116] in,

[0117]

[0118] E is the global reprojection error; M is the number of calibration images; N is the number of calibration feature points detected in each image; is the measured pixel coordinate of the nth feature point in the mth image; is the calculated theoretical projection pixel coordinate; is the camera intrinsic parameter matrix; f xis the equivalent focal length in the x direction; f y is the equivalent focal length in the y direction; s is the tilt of the principal point; u0, v0 are the principal point coordinates; k = [k1, k2, k3, p1, p2] T is the distortion coefficient vector; k1, k2, k3 are the first, second, and third order radial distortion coefficients; p1, p2 are the first and second order tangential distortion coefficients; R m is the rotation matrix that transforms the world coordinates to the camera coordinate system of the mth image; t m is the translation vector of the same transformation; P n =[X n ,Y n ,Z n ] T is the three-dimensional coordinate of the nth calibration feature point in the calibration plate coordinate system; n ,y n x is the undistorted pixel coordinate obtained by projection and normalization with the optical axis as the center; d ,y d is the pixel coordinate after adding radial-tangential distortion; Z c is the depth of the three-dimensional point in the camera coordinate system; r 2 is the square of the distance from the normalized coordinate to the optical axis;

[0119] By using nonlinear least squares iterative optimization on the reprojection error function E, the intrinsic parameter K and distortion coefficient k of each view are obtained at the same time, and the posture parameter R of each image is updated synchronously during the iteration m , t m , to ensure the best agreement between the projected model and the measured feature points.

[0120] In this embodiment, the system converts each three-dimensional feature point Pn on the calibration plate (whose spatial coordinates are measured by a high-precision fixture) into the camera coordinate system using the camera pose Rm and tm of the mth calibration image, and normalizes it with the depth Zc to obtain the distortion-free pixel coordinates (xn,yn). Subsequently, radial distortion coefficients k1, k2, k3 and tangential distortion coefficients p1 and p2 are introduced to perturb (xn,yn) to obtain the distorted coordinates (xd,yd). The camera intrinsic parameter matrix K = [[fxsu0], [0fyv0],

[001] ] is then used to map (xd,yd) to the theoretical projected pixel coordinates \hatxmn; where fx and fy are the equivalent focal lengths along the row and column directions, respectively, s describes the shearing caused by pixel non-orthogonality, and (u0,v0) are the principal point coordinates. The global reprojection error E is obtained by taking the difference between the theoretical projection coordinates and the measured pixel coordinates xmn of the calibration plate and calculating the square sum of all image frames (number is M) and their detected feature points (number per frame is N). 2 =xn 2 +yn 2It is the square of the distance from the normalized coordinate to the optical axis in the distortion calculation.

[0121] To align the mathematical model with the camera's true imaging characteristics, the system uses an improved Levenberg–Marquardt nonlinear least squares algorithm to iteratively optimize the reprojection error E. In each iteration, the intrinsic parameters K, the distortion coefficient vectors [k1, k2, k3, p1, p2], and the extrinsic pose parameters (Rm, tm) of each view are simultaneously updated. The algorithm computes the Jacobian matrix in parallel on the GPU and dynamically adjusts the damping factor until E drops to a preset threshold or the convergence difference between two consecutive rounds is less than 10- 6 Pixel 2 At the end of the optimization, the precise intrinsic parameter matrix and distortion coefficients for each viewpoint can be obtained simultaneously, and the pose solutions of all calibrated image frames meet the minimum reprojection error condition, laying a unified geometric foundation for subsequent distortion removal, epipolar correction, and parallax calculation.

[0122] Preferably, the perspective data sets are paired in pairs, disparity information is extracted using a feature matching algorithm based on optical flow, and multiple pairs of disparities are fused to generate a lead 3D point cloud, including:

[0123] For the geometrically calibrated view dataset, an ordered view pair list is constructed according to the view number, and an epipolar search window is established for each pair of images in the reference view-auxiliary view manner;

[0124] For each view pair in the ordered view pair list, a pyramid cascade optical flow algorithm is used to perform dense pixel matching from coarse to fine to obtain a sub-pixel optical flow field, and epipolar constraints of the same pair of views are used to eliminate flow vectors that do not satisfy homography consistency;

[0125] The filtered sub-pixel optical flow field is converted into a disparity map, and the corresponding pixels are triangulated line-by-line using the intrinsic-extrinsic matrix of each view to generate a local 3D point cloud represented in the reference view coordinate system.

[0126] Perform voxel binning and normal vector consistency detection on the local 3D point cloud of all view pairs, and remove outliers with significantly low spatial density or normal vector deviation exceeding a preset threshold;

[0127] The retained local three-dimensional point cloud is overlapped and fused according to the reference perspective, and the repeated points are merged and the weighted average depth value is used to form a unified and deduplicated three-dimensional point cloud of the lead.

[0128] In a specific implementation, this embodiment first reads three geometrically calibrated rectified view images and automatically generates an ordered view pair list (<0-1>, <0-2>, <1-2>) according to the default view number (e.g., 0, 1, 2). For each pair of images in the list, the system uses the pre-calculated epipolar correction homography to establish a pixel-precision epipolar search window in each row of the reference view image. It then activates the pyramid cascade optical flow module and iterates layer by layer from 1 / 8 scale to the original resolution: first, dense optical flow is estimated at a coarse scale with a pixel step size, and then, at a fine scale, the optical flow field is sampled from the previous layer as the initial value and refined at the sub-pixel level. After each layer of refinement, epipolar consistency detection is used to eliminate vectors whose flow direction deviates from the theoretical epipolar line by more than two pixels. A photometric consistency check of the same source block is then used to further filter out unreliable matches in dynamic texture or low-texture areas, thereby obtaining a dense optical flow field that satisfies the homography constraint and has an accuracy better than 0.1 pixel.

[0129] The optical flow field verified by bidirectional consistency is directly converted into a disparity map; the system calls the intrinsic and extrinsic parameter matrices of each view, performs line-line triangulation on the light pairs corresponding to the reference pixel and the auxiliary pixel, calculates the 3D coordinates of the nearest point and projects it back to the reference view coordinate system to obtain the local point cloud of the view pair. The local point clouds of all view pairs then enter voxelization processing: first, the 5μm 3 Voxels are binned by side length, retaining only the center coordinate of the point with the most points as a representative and counting its normal vector. Any voxel with a normal-to-normal difference exceeding 20 degrees or a voxel with fewer than three points is considered an outlier and deleted. The coordinates of the selected representative voxels are then overlapped and fused according to the viewpoint label. When the same spatial location appears in multiple viewpoint pairs, a depth-weighted average is taken to suppress random errors. The resulting unified point cloud has uniform density, no duplication or redundancy, and a reduced error level from 2.5μm in single parallax to 1.2μm, sufficient for subsequent surface reconstruction and dimensional measurement.

[0130] Preferably, performing noise filtering and surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional mesh model of the lead includes:

[0131] For each point in the three-dimensional point cloud, the average Euclidean distance of its k nearest neighbors is calculated. If the average Euclidean distance is greater than the global distance average plus α times the standard deviation, the point is determined to be an outlier and deleted, thereby obtaining a purified point cloud.

[0132] The purified point cloud is divided into two parts with a side length of l v For each voxel, the geometric centroid of all points belonging to the voxel is taken as the representative point to form a sparse point cloud P with uniform density. s ;

[0133] In the sparse point cloud P sThe normal vector of each representative point is estimated by the k-nearest neighbor method and the moving least squares surface fitting method is used to locally smooth the coordinates and normal vectors of the representative points to obtain a smooth point cloud P s,MLS ;

[0134] Smooth point cloud P s,MLS and its normal vector as input, set the octree depth d and solve the Poisson equation to generate the initial triangular mesh M0;

[0135] Perform Laplace smoothing on the initial mesh M0 and combine it with edge folding and edge flipping simplification operations to reduce the number of facets while keeping the curvature change less than the threshold κ, obtaining the optimized mesh M1;

[0136] Detect the area of ​​the optimized grid M1 that is less than the threshold A min The holes are automatically filled, and then the topological connectivity and normal consistency are checked. If there are any anomalies, local triangulation is performed, and finally the continuous three-dimensional mesh model M with topologically closed, connected and smooth leads is output. final .

[0137] In this implementation, the system first performs statistical outlier removal on the fused 3D lead point cloud: for each point, the system retrieves its twenty nearest neighbors and calculates their average Euclidean distance. The global mean and standard deviation of all average distances across the entire cloud are then calculated. If a point's average distance exceeds the global mean plus twice the standard deviation, it is considered an outlier and removed, typically filtering out 3%–5% of stray noise. The cleansed point cloud is then voxelized and downsampled, with a voxel edge length of 3μm. The algorithm traverses the cubic voxel grid, replacing all points falling within the same voxel with a single representative point using its geometric centroid, resulting in a sparse point cloud with uniform density. Normal vectors are then estimated for the sparse point cloud using its twenty nearest neighbors. Local smoothing of the coordinates and normal vectors is performed using a moving least squares surface fitting window with a 6μm radius, suppressing measurement noise while preserving subtle curvature variations on the lead surface, resulting in a smooth point cloud.

[0138] The Poisson surface reconstruction module is then called, taking the smoothed point cloud and its normal vector as input, setting the maximum octree depth to 10, and extracting the zero level set to generate the initial triangular mesh. After the initial mesh is smoothed ten times by Laplace, the number of facets is reduced by about 40% using edge folding and edge flipping methods, and the curvature change is monitored in real time to ensure that the curvature increment caused by any edge operation does not exceed 2°. The simplified mesh is inspected for all edges less than 100μm. 2 The system automatically fills holes in the mesh. It then checks the overall topological connectivity and normal consistency, automatically retriangulating any localized flips or non-manifold structures. The resulting 3D mesh is visually continuous and smooth, with uniform normal orientation and no isolated faces or openings, making it ready for dimensional measurement, finite element analysis, or interactive visualization.

[0139] Preferably, texture mapping is performed on the three-dimensional mesh model using any one of the perspective images or a combination thereof to output a textured three-dimensional reconstruction result of the lead wire, including:

[0140] Perform fast visibility statistics on all view images after geometric calibration, sort them by the number of visible mesh faces, and automatically select the single view or two-view combination with the best visibility as the texture source;

[0141] Perspectively projecting the vertices of the three-dimensional mesh model onto a pixel plane of a selected viewing angle to obtain corresponding UV coordinates;

[0142] If it is a two-view combination, for each triangular face in the 3D mesh model, the angle between the face normal vector and the sight line vectors of the two cameras' optical centers pointing to the face is calculated, and the triangular face is assigned to the view with the smallest angle, and only the UV coordinates corresponding to the view are retained;

[0143] The pixel color is directly sampled from the selected perspective image according to the generated UV coordinates, written into a single texture map, and the texture map is bound to the three-dimensional mesh model to output the textured lead three-dimensional reconstruction result.

[0144] In this implementation, GPU-based facet-by-face raycasting is first performed on three geometrically calibrated, rectified view images. This method counts the number of visible mesh faces and the average angle of incidence for each view within 30ms. The three images are then sorted in descending order by the number of visible faces. If the top view covers at least 85% of the mesh faces, it is selected as the sole texture source. If coverage is insufficient, the top two views are automatically combined to form a texture view pair. After view selection, the system uses camera intrinsics and extrinsic parameters to perspective-project the 3D mesh vertices onto the pixel plane of the selected view, generating vertex-level UV coordinates in real time and caching them in GPU memory.

[0145] If the system selects a two-view combination, each triangle face of the mesh is traversed in the rendering thread, and the angle difference between the face normal and the sight line vector of the optical center of the two cameras pointing to the face is calculated to determine which perspective is "facing" the face; the side with the smaller angle is defined as the main texture perspective, and the UV coordinates of the other side are immediately discarded. The renderer then samples the color data pixel by pixel directly from the corresponding perspective image according to the final retained UV coordinates, writes all the sampling results into a 4K×4K single universal texture map, and binds the texture map to the mesh model through OpenGL during the loading phase. The textured 3D reconstruction results output by the above process can control the cross-face brightness gap within 2% in the lighting consistency test, which not only meets the requirements of industrial size detection for texture positioning, but also provides visual inspection personnel with a visual model of real details.

[0146] Corresponding to the above method, such as Figure 2 As shown, this embodiment also provides a chip bonding wire 3D measurement system based on an intracavity reflection camera system, comprising:

[0147] A multi-view intracavity imaging module is used to perform a single exposure imaging of the bonding wires of the chip to be tested using an intracavity reflection camera system, thereby obtaining at least three independent viewpoint images in the same frame;

[0148] An image normalization and geometric calibration module, configured to perform brightness normalization compensation and geometric calibration on the view images to obtain a view data set with uniform brightness and correlation;

[0149] A disparity extraction and point cloud fusion module is used to pair the view data sets in pairs, extract disparity information using a feature matching algorithm based on optical flow, and fuse multiple pairs of disparities to generate a lead 3D point cloud;

[0150] A point cloud filtering and surface reconstruction module, configured to perform noise filtering and surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional mesh model of the lead;

[0151] The texture mapping and model output module is used to perform texture mapping on the three-dimensional mesh model by adopting any one of the perspectives in the perspective images or a combination thereof, and output a three-dimensional reconstruction result of the lead with texture.

[0152] The beneficial effects of the present invention are as follows:

[0153] (1) The present invention uses an intracavity reflection camera system to synchronously capture at least three non-overlapping perspective images in a single exposure, eliminating the mechanical scanning and alignment steps required for traditional multiple shooting, significantly reducing error sources such as vibration and thermal drift, and compressing the measurement cycle to milliseconds, greatly improving the rhythm and stability of online detection on the production line.

[0154] (2) The present invention utilizes a two-stage calibration strategy combining Lambert white reference plate flat-field correction with checkerboard multi-pose calibration to achieve pixel-level brightness balance and sub-pixel geometric alignment; combined with an epipolar consistency pyramid optical flow algorithm, dense parallax with photometric and geometric consistency can be obtained under complex occlusion and large depth of field conditions, thereby controlling the three-dimensional reconstruction error of bonded multi-layer gold wires to the micron level.

[0155] (3) After statistical outlier removal, voxel averaging, moving least squares smoothing and Poisson surface reconstruction, the output mesh model of the present invention maintains continuous surface and topological closure characteristics, and can be directly used in size measurement, weld height difference evaluation and finite element thermal-stress analysis without the need for additional data cleaning or format conversion, significantly saving post-processing time and labor costs.

[0156] (4) The present invention uses fast visibility statistics and adaptive viewpoint allocation strategies, and the system can complete the full model texture baking with only a single or double rectified image; the generated textured three-dimensional model has a color gap of less than 2%, which is convenient for manual visual inspection of welding line defects and provides high-quality training data for machine learning algorithms. Overall, it takes into account the three key requirements of high resolution, fast output and low equipment cost.

[0157] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0158] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A three-dimensional measurement method for chip bonding wires based on an intracavity reflection camera system, characterized in that: include: Performing a single exposure imaging of the bonding wires of the chip to be tested by an intracavity reflection camera system to obtain at least three independent perspective images in the same frame; Performing brightness normalization compensation and geometric calibration on the perspective images to obtain a perspective data set with uniform brightness and mutual correlation; Pairing the perspective data sets in pairs, extracting disparity information using a feature matching algorithm based on optical flow, and fusing multiple pairs of disparities to generate a lead 3D point cloud; Performing noise filtering and surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional mesh model of the lead; Texture mapping is performed on the three-dimensional mesh model using any one of the perspectives in the perspective images or a combination thereof, and a three-dimensional reconstruction result of the lead with texture is output.

2. The chip bonding wire three-dimensional measurement method based on the intracavity reflection camera system according to claim 1, characterized in that: The intracavity reflection camera system is used to perform a single exposure imaging of the bonding wires of the chip to be tested, obtaining at least three independent perspective images in the same frame, including: Fix the chip to be tested on the stage and adjust the stage along the main optical axis to align with the object conjugate plane of the intracavity reflection camera system so that the bonding wire is in the optimal focal plane; Synchronously triggering the image sensor shutter within a single exposure cycle and driving the first plane reflector, the second plane reflector, and the half-mirror disposed in the inner cavity so that the light beam from the area to be measured forms a multi-field light beam along at least three different reflection paths; Using a combination of a microscope main lens and a one-time magnification relay lens, the multi-field light beam and the direct field of view are simultaneously projected onto the same image sensor surface to obtain an original image frame containing multiple sub-views; The original image frame is subjected to sub-view decoupling and storage according to a preset field of view segmentation template to obtain at least three non-overlapping and independent perspective images.

3. The chip bonding wire three-dimensional measurement method based on the intracavity reflection camera system according to claim 1, characterized in that: Performing brightness normalization compensation and geometric calibration on the perspective image to obtain a perspective data set with uniform brightness and mutual correlation, including: Place a Lambert white reference plate at the workstation where the chip to be tested is located, and obtain uniformly illuminated reference images at each viewing angle. Calculating a pixel-level gain-offset correction matrix according to the reference image, and performing pixel-by-pixel brightness correction on the original image of each viewing angle based on the pixel-level gain-offset correction matrix to obtain a brightness normalized image; Place a flat stereo calibration plate with a checkerboard or coded dots at the same workstation and collect calibration image sequences from various viewpoints. The calibration image sequence is used to solve the intrinsic parameter matrix and extrinsic parameter pose of each view, perform lens distortion removal and epipolar correction, resample the brightness normalized image to a common image plane, and generate a geometrically calibrated view data set.

4. The chip bonding wire three-dimensional measurement method based on the intracavity reflection camera system according to claim 3, characterized in that: The calculation formula of the pixel-level gain-offset correction matrix is: Wherein, M(u,v) is the pixel-level gain-offset correction matrix; F(u,v) is the flat-field pixel intensity obtained under the Lambert white reference plate condition; is the global average value of F(u,v), which is used to provide a global energy benchmark; D(u,v) is the dark field reference pixel intensity, which reflects the sensor dark current and fixed pattern noise; is the global average value of D(u,v), which is used to unify the dark field baseline; The correction formula for the pixel-by-pixel brightness correction is: is the grayscale value of the corresponding pixel in the corrected brightness normalized image; I(u,v) is the pixel intensity to be corrected in the original image of each viewing angle.

5. The chip bonding wire three-dimensional measurement method based on the intracavity reflection camera system according to claim 3, characterized in that: The calibration image sequence is used to solve the intrinsic parameter matrix and extrinsic parameter pose of each view, perform lens distortion removal and epipolar correction, resample the brightness normalized image to a common image plane, and generate a geometrically calibrated view data set, including: Capturing the calibration image sequence for each viewing angle; the calibration image sequence is a plurality of frames of images captured continuously when the same calibration plate is in multiple known spatial postures; the calibration plate surface has a checkerboard or coded dot pattern; Based on minimizing the reprojection error of the feature points in the calibration image sequence, solving the camera intrinsic parameter matrix and the radial and tangential distortion coefficients of each viewpoint respectively; Select a view angle as a reference view angle, and use the coordinates of the corresponding feature points in the calibration image of the same frame to calculate the extrinsic pose parameters of the remaining view angles relative to the reference view angle; Performing inverse distortion mapping on the brightness normalized image of each viewing angle according to the distortion coefficient to obtain a dedistorted image; Calculating an epipolar correction homography matrix from each viewpoint to a common virtual image plane according to the intrinsic parameter matrix and the extrinsic parameter posture parameters, and performing homography transformation and bilinear interpolation resampling on the dedistorted image; The resampled view images are cropped into common overlapping view areas according to the view index to generate a geometrically calibrated view dataset.

6. The chip bonding wire three-dimensional measurement method based on the intracavity reflection camera system according to claim 5, characterized in that: Based on minimizing the reprojection error of the feature points in the calibration image sequence, the camera intrinsic parameter matrix and the radial and tangential distortion coefficients of each view are solved respectively, including: Construct a reprojection error function E; the expression of the reprojection error function E is: in, E is the global reprojection error; M is the number of calibration images; N is the number of calibration feature points detected in each image; is the measured pixel coordinate of the nth feature point in the mth image; is the calculated theoretical projection pixel coordinate; is the camera intrinsic parameter matrix; f x is the equivalent focal length in the x direction; f y is the equivalent focal length in the y direction; s is the tilt of the principal point; u0, v0 are the principal point coordinates; k = [k1, k2, k3, p1, p2] T is the distortion coefficient vector; k1, k2, k3 are the first, second, and third order radial distortion coefficients; p1, p2 are the first and second order tangential distortion coefficients; R m is the rotation matrix that transforms the world coordinates to the camera coordinate system of the mth image; t m is the translation vector of the same transformation; P n =[X n ,Y n ,Z n ] T is the three-dimensional coordinate of the nth calibration feature point in the calibration plate coordinate system; n ,y n x is the undistorted pixel coordinate obtained by projection and normalization with the optical axis as the center; d ,y d is the pixel coordinate after adding radial-tangential distortion; Z c is the depth of the three-dimensional point in the camera coordinate system; r 2 is the square of the distance from the normalized coordinate to the optical axis; By using nonlinear least squares iterative optimization on the reprojection error function E, the intrinsic parameter K and distortion coefficient k of each view are obtained at the same time, and the posture parameter R of each image is updated synchronously during the iteration m , t m , to ensure the best agreement between the projected model and the measured feature points.

7. The chip bonding wire three-dimensional measurement method based on the intracavity reflection camera system according to claim 1, characterized in that: The perspective data sets are paired with each other, and the disparity information is extracted using a feature matching algorithm based on optical flow. Multiple pairs of disparities are fused to generate a lead 3D point cloud, including: For the geometrically calibrated view dataset, an ordered view pair list is constructed according to the view number, and an epipolar search window is established for each pair of images in the reference view-auxiliary view manner; For each view pair in the ordered view pair list, a pyramid cascade optical flow algorithm is used to perform dense pixel matching from coarse to fine to obtain a sub-pixel optical flow field, and epipolar constraints of the same pair of views are used to eliminate flow vectors that do not satisfy homography consistency; The filtered sub-pixel optical flow field is converted into a disparity map, and the corresponding pixels are triangulated line-by-line using the intrinsic-extrinsic matrix of each view to generate a local 3D point cloud represented in the reference view coordinate system. Perform voxel binning and normal vector consistency detection on the local 3D point cloud of all view pairs, and remove outliers with significantly low spatial density or normal vector deviation exceeding a preset threshold; The retained local three-dimensional point cloud is overlapped and fused according to the reference perspective, and the repeated points are merged and the weighted average depth value is used to form a unified and deduplicated three-dimensional point cloud of the lead.

8. The chip bonding wire three-dimensional measurement method based on the intracavity reflection camera system according to claim 1, characterized in that: Performing noise filtering and surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional mesh model of the lead, including: For each point in the three-dimensional point cloud, the average Euclidean distance of its k nearest neighbors is calculated. If the average Euclidean distance is greater than the global distance average plus α times the standard deviation, the point is determined to be an outlier and deleted, thereby obtaining a purified point cloud. The purified point cloud is divided into two parts with a side length of l v For each voxel, the geometric centroid of all points belonging to the voxel is taken as the representative point to form a sparse point cloud P with uniform density. s ; In the sparse point cloud P s The normal vector of each representative point is estimated by the k-nearest neighbor method and the moving least squares surface fitting method is used to locally smooth the coordinates and normal vectors of the representative points to obtain a smooth point cloud P s,MLS ; Smooth point cloud P s,MLS and its normal vector as input, set the octree depth d and solve the Poisson equation to generate the initial triangular mesh M0; Perform Laplace smoothing on the initial mesh M0 and combine it with edge folding and edge flipping simplification operations to reduce the number of facets while keeping the curvature change less than the threshold κ, obtaining the optimized mesh M1; Detect the area of ​​the optimized grid M1 that is less than the threshold A min The holes are automatically filled, and then the topological connectivity and normal consistency are checked. If there are any anomalies, local triangulation is performed, and finally the continuous three-dimensional mesh model M with topologically closed, connected and smooth leads is output. final .

9. The chip bonding wire three-dimensional measurement method based on the intracavity reflection camera system according to claim 1, characterized in that: Using any one of the perspective images or a combination thereof to perform texture mapping on the three-dimensional mesh model and outputting a textured three-dimensional reconstruction result of the lead wire, including: Perform fast visibility statistics on all view images after geometric calibration, sort them by the number of visible mesh faces, and automatically select the single view or two-view combination with the best visibility as the texture source; Perspectively projecting the vertices of the three-dimensional mesh model onto a pixel plane of a selected viewing angle to obtain corresponding UV coordinates; If it is a two-view combination, for each triangular face in the 3D mesh model, the angle between the face normal vector and the sight line vectors of the two cameras' optical centers pointing to the face is calculated, and the triangular face is assigned to the view with the smallest angle, and only the UV coordinates corresponding to the view are retained; The pixel color is directly sampled from the selected perspective image according to the generated UV coordinates, written into a single texture map, and the texture map is bound to the three-dimensional mesh model to output the textured lead three-dimensional reconstruction result.

10. A chip bonding wire three-dimensional measurement system based on an intracavity reflection camera system, characterized in that: include: A multi-view intracavity imaging module is used to perform a single exposure imaging of the bonding wires of the chip to be tested using an intracavity reflection camera system, thereby obtaining at least three independent viewpoint images in the same frame; An image normalization and geometric calibration module, configured to perform brightness normalization compensation and geometric calibration on the view images to obtain a view data set with uniform brightness and correlation; A disparity extraction and point cloud fusion module is used to pair the view data sets in pairs, extract disparity information using a feature matching algorithm based on optical flow, and fuse multiple pairs of disparities to generate a lead 3D point cloud; A point cloud filtering and surface reconstruction module, configured to perform noise filtering and surface reconstruction on the three-dimensional point cloud to obtain a continuous three-dimensional mesh model of the lead; The texture mapping and model output module is used to perform texture mapping on the three-dimensional mesh model by adopting any one of the perspectives in the perspective images or a combination thereof, and output a three-dimensional reconstruction result of the lead with texture.

Images