3D microscopic measurement method based on shift-axis projector and double telecentric camera

Abstract

The application provides a 3D microscopic measurement method based on a shift lens projector and a double telecentric camera, comprising the following steps: 1. Pre-distortion fringe pattern generation. In order to offset the measurement error caused by the lens distortion and shift of the pinhole shift light field projector, the inverse distortion processing is performed on the original ideal projection pattern according to the distortion model given by formula (4) and the calibration parameters of the pinhole shift light field projector, and the optimized light field projection fringe pattern is obtained through an iterative optimization algorithm; 2. Calibration image acquisition. A plane target is placed in the measurement volume, the pinhole shift light field projector projects two groups of horizontal and vertical three-frequency N-step phase shift fringe patterns in turn, and at the same time, the camera captures the modulation deformation fringe on the target, and the process is repeated in several different target attitudes f={1, 8, 57}, and the fringe phase shift step is N={4, 4, 10}. The application can obtain similar measurement accuracy as the PHM method, but the cost is lower, the flexibility and efficiency of the 3D microscopic measurement are improved, and the application can be used for online microscopic industrial measurement.

Description

Technical Field

[0001] This invention belongs to the field of optical measurement technology, and particularly relates to a 3D microscopic measurement method based on a tilt-shift projector and a dual telecentric camera. Background Technology

[0002] Fringe projection profilometry (FPP) offers advantages such as non-contact measurement, high speed, high accuracy, full-field measurement capability, and the ability to obtain dense point clouds. This has led to its widespread application in numerous fields, including industrial inspection, healthcare, reverse engineering, cultural heritage, and entertainment. Combining fringe projection equipment with microscopic optics allows the projection of fringe patterns onto a small area, enabling the measurement of the three-dimensional morphology of minute devices. Telecentric lenses, with their advantages of orthographic projection, high resolution, near-zero distortion, constant magnification, and large depth of field, have seen widespread use in microscopic measurement in recent years. The mainstream configuration for FPP microscopic measurement systems is a pinhole projector + camera, which offers advantages such as low cost, simple structure, and flexible field of view. For this configuration, system calibration is a crucial step in the entire microscopic measurement process. Existing calibration methods for projector-camera microscopic measurement systems are mainly divided into two categories: methods based on phase height mapping (PHM) and methods based on stereo vision. Because stereoscopic vision methods require calibrating the projector as a reverse camera and involve time-consuming stereo matching steps, the accuracy and efficiency of microscopic measurements are difficult to guarantee. Therefore, 3D measurement systems based on a single camera and a projector primarily employ the PHM (Probe-Modified Measurement) method. However, due to the constant magnification of the telecentric camera, the PHM method requires a precision displacement stage (usually manually operated). While this ensures measurement accuracy, it increases the system's hardware costs and the manual labor costs associated with system calibration.

[0003] Technical solution of existing technology 1

[0004] 3D microscopy measurement methods based on stereo vision are grounded in triangulation theory. The projector is typically viewed as a reverse camera, and the projection process is described using a mathematical model of camera imaging. Specifically, the projector, with the aid of the camera, "captures" the image of the calibration target. Therefore, each pixel in the camera image and each pixel in the projector image can be marked with phase values ​​in both the horizontal and vertical directions by projecting orthogonal fringes. This phase marking establishes a mapping relationship between the camera image and the projector image, thus obtaining the projector image. Subsequently, stereo vision technology allows for flexible calibration of the pinhole projector-camera system without any practical limitations. However, 3D reconstruction is based on system parameters, so the measurement accuracy of this method largely depends on the calibration accuracy of these parameters.

[0005] Disadvantages of existing technology 1

[0006] A pinhole projector-camera system can essentially be considered a stereo vision system, but the dual telecentric cameras are not sensitive to changes in depth. This characteristic increases the difficulty of system calibration. However, it heavily relies on precise projector and camera parameters, and many factors such as gamma and lens distortion often affect the final calibration accuracy, making sufficiently accurate system calibration difficult to achieve. Furthermore, the subsequent stereo matching process is also quite cumbersome, limiting the measurement efficiency of stereo vision methods. In contrast, the PHM method is more robust to the influence of system parameters and does not require projector calibration, making it a better choice than stereo vision-based methods.

[0007] Technical solution of existing technology 2

[0008] The PHM-based method establishes a mapping relationship between phase and height, eliminating the need to calculate projector parameters and allowing direct application to the calibration of projector-camera systems. The system calibration using this method mainly consists of two parts: (1) camera calibration; and (2) PHM calibration, i.e., pixel-by-pixel PHM coefficient calculation. These two calibrations are performed separately; PHM calibration requires manual movement of a precision displacement stage. Subsequently, the X and Y coordinates can be calculated using the camera calibration parameters, and the Z coordinate can be calculated using the PHM coefficients, thus enabling 3D topographic measurement.

[0009] Disadvantages of existing technology 2

[0010] It typically requires a precision stage to achieve accurate PHM calibration. Each time the stage's height changes during translation, it needs to be moved precisely, a process that is usually quite time-consuming. The additional stage not only increases hardware costs but also reduces the flexibility of system calibration due to the increased manual operation required. This makes system calibration more complex and expensive. Figure 1 As shown, PHM calibration and camera calibration are performed separately. The movement of the translation stage will generate motion errors, which will affect the final measurement accuracy. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a 3D microscopic measurement method based on a tilt-shift projector and a dual telecentric camera.

[0012] Phase-height mapping (PHM)

[0013] Phase-to-Z-coordinate conversion (PZC)

[0014] Look-up table (LUT)

[0015] Fringe projection profilometry (FPP)

[0016] Three-dimensional (or simply 3D).

[0017] The principle of this invention is as follows: a planar target is used to simultaneously calibrate the camera and projector; then, the extrinsic parameters of the projector and camera are used to reconstruct the target at different poses. Subsequently, based on the reconstruction results of the target, a clear pixel-by-pixel phase-to-Z coordinate transformation lookup table (LUT) is established within the system's measurement volume. The Z-axis coordinate is calculated based on the PZC-LUT and the phase value carried by the measured point, and the (X,Y) coordinates can be derived from the undistorted camera pixel coordinates.

[0018] The present invention adopts the following technical solution:

[0019] 3D microscopic measurement methods based on tilt-shift projectors and dual telecentric cameras include:

[0020] Step 1. Generation of pre-distortion fringe pattern

[0021] To compensate for the lens distortion and measurement error caused by the shift of the pinhole axis-shifting light field projector, the inverse distortion processing of the original projection pattern is performed according to the distortion model given by formula (4) and the calibration parameters of the pinhole axis-shifting light field projector, and the optimized light field projection fringe pattern is obtained through iterative optimization algorithm.

[0022]

[0023] Let be the second-order radial distortion coefficient of the projector. Let be the fourth-order radial distortion coefficient of the projector. Let be the sixth-order radial distortion coefficient of the projector. Let be the tangential distortion coefficient in the x-direction of the projector. Let x be the tangential distortion coefficient in the y-direction of the projector. p Let x be the normalized coordinates of the projector image plane in the x-direction, and y be the normalized coordinates of the projector image plane in the x-direction. p Let r be the normalized coordinate of the y-axis of the projector image plane. p The normalized radius;

[0024] Step 2: Calibration Image Acquisition

[0025] A planar target is placed in the measurement volume, and a pinhole tilt-shift optical field projector sequentially projects two sets of horizontal and vertical three-frequency N-step phase-shifted fringe patterns. Simultaneously, a camera captures the modulation-deformed fringes on the target. This process is repeated in several different target orientations. The fringe frequencies used in the experiment are f = {1, 8, 57}, and the fringe shift step size is N = {4, 4, 10}.

[0026] Step 3: Dual telecentric camera calibration

[0027] Using the previously derived closed-loop solution, the dual telecentric cameras are calibrated. This includes estimating the initial parameters of the dual telecentric cameras and then using a constrained optimization algorithm to optimize the initial parameters to minimize the reprojection error. The calibration image is the average value of the high-frequency images captured in step 2.

[0028] Step 4. Calibrate the tilt-shift pinhole projector

[0029] First, the phase-assisted method is used to establish the correspondence between the pixels of the dual telecentric camera image and the pinhole shift light field projector image. Then, based on this relationship, the pixel coordinates of the obtained dual telecentric camera target image feature points are mapped to the image coordinate system of the pinhole shift light field projector. Finally, the projector is calibrated according to formulas (3) and (4), the sub-pixel level projector calibration algorithm in [1], and the Schahm's law distortion correction algorithm in [2]. That is, by inputting the world coordinates of the target feature points and the corresponding feature point coordinates of the pinhole shift light field projector image, the intrinsic and extrinsic parameters of the pinhole shift light field projector are obtained.

[0030] s p (u p ν p 1) T =K p (R wp T wp (XYZ 1) T (3)

[0031]

[0032] Step 5. Target Reconstruction

[0033] Point X on the target at position i i (X i ,Y i Z i The image point of ) is q(u c ,ν c ), will,u c ,ν c Substituting into formula (7), we obtain the coordinate component X. i Y i X i ,Yi Substituting into formula (6), we get Z i Repeat this step for each image point to reconstruct the target at position i. Using the same method, 3D models of all pose targets can be obtained.

[0034]

[0035] Step 6: Calculation of PZC coefficient

[0036] Each image point q(u c ,ν c There are n+1 corresponding 3D world points, located at the poses of the n+1 target points. Their Z-axis coordinate components and the phase values ​​they carry constitute the dataset {(Z...} for fitting the PZC polynomial parameters. wi ,Φ i ),i=0,1,...,n-1,n}, where n+1>M, repeat this operation for each image point to build a pixel-by-pixel PZC coefficient lookup table;

[0037] Step 7: Microdevice Image Acquisition

[0038] A set of orthogonal pre-distorted phase-shifted fringe patterns is projected onto the surface of the microdevice under test, while dual telecentric cameras capture the fringes that are modulated and deformed by the surface profile of the microdevice and obtain an absolute phase map.

[0039] Step 8: Three-dimensional reconstruction of microdevices

[0040] pixel coordinates (u c ,v c Substituting these values ​​into formula (7) yields the coordinates (X, Y). Then, substituting the absolute phase and PZC coefficients into formula (11) yields the corresponding Z-axis coordinates.

[0041]

[0042] Furthermore, in step 1, the total number of iterations is 9. By projecting these pre-distorted horizontal and vertical phase-shifted fringe patterns, the pinhole shift light field projector projects a distortion-free image, which means that the projector distortion is canceled. Thus, the pinhole shift light field projector is regarded as a linear projection system.

[0043] Furthermore, the reconstruction of the target in different poses includes:

[0044] To achieve pixel-level PZC without using a precise translation stage, the coordinates (X, Y, Z) of each point on the target are obtained using the projector extrinsic parameters and the parameters of the dual telecentric cameras for PZC calibration. In fact, the world coordinate system (WCS) at pose i is first calculated. iThe rotation matrix R of the world coordinate system WCS0 at pose 0 i0 Translation vector T i0 Then, the plane equation of the target in WCS0 under other poses is calculated, and then the effective target at different poses is reconstructed.

[0045] Let R wpi and T wpi For WCS i The rotation matrix and translation vector to PCS, R wci and T wci For WCS i The rotation matrix and translation vector to the CCS, along with the constant magnification characteristic of the dual telecentric cameras, make T wci The third component is 0, let (R) pwi ,T pwi Let be the pose parameters of the i-th target relative to the PCS, then:

[0046]

[0047] R pw0 R is the transpose of the rotation matrix from the projector coordinate system to the world coordinate system. wp0 T represents the rotation matrix from the world coordinate system to the initial projector coordinate system. pw0 Let represent the translation vector from the initial projector coordinate system to the world coordinate system. CCS is the coordinate system of the dual telecentric cameras, and PCS is the projector coordinate system. The plane equation of the target plane at pose i in WCS0 is derived, and...

[0048]

[0049] in, For R i0 The last column element, For T i0 The elements in, let Expressing formula (6) as A i x+B i y+C i z+D i =0;

[0050] Let X = (X, Y, Z) be a point on the target, and q(u c ,ν c ) is its distortion-free pixel, determined by the formula In the imaging model of (1), the calibration parameters of the dual telecentric cameras and the pixel coordinates of point q are used to directly calculate X and Y. The calculation formula is as follows:

[0051]

[0052] s c u is the scale factor for a dual telecentric camera. c Let ν be the x-axis pixel coordinate on the image plane of the dual telecentric camera. c The meaning is the y-axis pixel coordinate on the image plane of a dual telecentric camera, K c Let X be the intrinsic parameter matrix of the dual telecentric camera, X be the x-axis coordinate of the target point in the three-dimensional world coordinate system, Y be the y-axis coordinate of the target point in the three-dimensional world coordinate system, and Z be the z-axis coordinate of the target point in the three-dimensional world coordinate system. In formula (7), the extrinsic parameters are the elements of the rotation matrix and translation vector from WCS0 to CCS. Substituting formula (7) into formula (6), The Z-coordinate is obtained by taking the pixel coordinates of the principal point in the camera image. Thus, the three-dimensional coordinates of each point on the calibrated target are obtained. In this way, the target under different poses can be reconstructed.

[0053] Furthermore, the parametric Z-axis coordinate calculation includes:

[0054] O p O c The origins of the projector coordinate system and the dual telecentric camera coordinate system are respectively. The projector projects two sets of orthogonal fringe patterns onto the target, thus forming a stable phase field. This allows each point on the target to be marked with a phase value. Based on the ray path and imaging model, the relationship between the phase and the Z-axis coordinate is derived. The derivation steps are as follows:

[0055] (1). Q0 is a point on the reference plane, and H is O. p and its projection point O' p The distance between them, Φ0 is the phase value carried by Q0, and L is the distance between Q0 and O'. p The distances between them, l', l'0, l1', l'2, are from O p The reflected rays l, l0, l1, l2 are given, and q0 is the image point of Q0. A micro-device of height Z is placed at Q0, causing the phase at point q0 to change from Φ0 to Φ. Assuming that when the phase Φ0 increases by 1, the corresponding change in distance on the reference plane is λ0, according to geometric relationships, we have:

[0056]

[0057] (2). For image I c For any pixel on the graph, once the measurement system is determined, H, L, Φ0, and λ0 are all constants. Therefore, formula (8) can be rewritten as follows:

[0058]

[0059] Where, {c0, c i cm , g} represents the pixel-wise mapping coefficients to be calibrated, M is the degree of the polynomial fitting function, c0 represents the coefficient of the constant term in the polynomial fitting, c i c represents the coefficient of the i-th term in the phase-height mapping relationship. m The coefficients of the highest-degree term of the polynomial are represented by , and g represents the global calibration parameter or system-level parameter. For computational convenience, F(Φ) is expanded using Taylor series at Φ0.

[0060]

[0061] For any pixel F(Φ0), F'(Φ0), F""Φ0",...,F (M) (Φ0) and Φ0 are constants. Formula (10) can be rewritten as follows:

[0062]

[0063] Where ΔΦ=Φ-Φ0, these coefficients are fitted using the obtained 3D point cloud data of the target and the phase carried by the target points;

[0064] (3) The reflected ray intersects the target at position i at point X. i (X i ,Y i Z i ), whose image point is q(u c ,ν c Let Φ i For q and X i The phase carried is calculated by formulas (6) and (7) for Z. i Similarly, calculate points X0, X1, ..., X n-1 ,X n The Z-axis coordinates Z0, Z1, ..., Z in the universal world coordinate system n-1 Z n This yields n+1 sets of Z-axis coordinates and their corresponding phase values ​​{(Z wi , Φ i ), i = 0, 1, ..., n-1, n}, are used to fit the polynomial parameters of PZC, usually n+1 > M.

[0065] The beneficial effects of this invention are:

[0066] This invention utilizes an FPP (Flexible Printed Profilometry) microscopy system to achieve flexible and efficient 3D microscopic measurements. This method eliminates the need for a precision displacement stage, enabling accurate 3D microscopic measurements. Experiments show that this method achieves similar measurement accuracy to the PHM (Profound Physics Method) but at a lower cost, improving the flexibility and efficiency of 3D microscopic measurements and making it suitable for online industrial microscopic measurements.

[0067] This invention eliminates the need for a displacement stage to achieve PZC, reducing hardware and labor costs and improving the flexibility of system calibration. It enables pinhole tilt-shift light field projector, dual telecentric cameras, and PZC calibration through a single acquisition of calibration images, thus improving calibration efficiency. Once calibration is complete, efficient 3D microscopic measurements can be achieved by projecting only vertical stripes.

[0068] Zhang et al. [3] (S. Zhang, “Flexible and high-accuracy method for uni-directional structured light system calibration,” Opt. Lasers Eng. 143, 106637 (2021).) used existing flexible camera calibration methods to extract each calibration pose and determine the three-dimensional coordinates of each pixel, and then established the pixel-level relationship between the three-dimensional coordinates of the measured object point and the phase of the corresponding image point. However, this method is for a pinhole projector + pinhole camera system and cannot be used in a system of a pinhole tilt-shift light field projector + a dual telecentric camera.

[0069] Miao et al. [4] (Y.Miao, Y.Yang, Q.Hou, Z.Wang, X.Liu, Q.Tang, X.Peng, and B.Z.Gao, “High-efficiency 3D reconstruction with a uniaxial mems-based fringeprojection profilometry,” Opt. Express 29(21), 34243–34257(2021).) proposed a flexible calibration method for obtaining 3D mapping coefficients. The mapping relationship between phase and spatial three-dimensional coordinates is directly established by the intersection of the camera reflected light and the equiphase plane. This method improves the flexibility of FPP calibration. However, this method is for a system based on MEMS (Micro-Electro-Mechanical System) projector + pinhole camera, and cannot be used in the system of a pinhole shift-axis light field projector + a dual telecentric camera of this invention.

[0070] Yang et al. [5] (Y.Yang, Y.Miao, X.Liu, G.Pedrini, Q.Tang, W.Osten, and X.Peng, “Intrinsic parameter-free calibration of FPP using a ray phase mapping model,” Opt. Lett. 47(14), 3564–3567(2022).) proposed an FPP ray mapping model that does not require calibration of the system's internal parameters. Although it can be applied to three-dimensional microscopy of telecentric lenses, it has a significant drawback. In this model, camera ray calibration and camera calibration are two independent processes, which leads to computational redundancy and high computational cost. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0072] Figure 2 This is a flowchart of the overall process of the three-dimensional microscopic measurement model proposed in this invention;

[0073] Figure 3 An iterative optimization strategy for distortion elimination in light field projectors;

[0074] Figure 4 The diagram shows the principle of target reconstruction, where (a) is a diagram showing the relationship between PCS, CCS and different WCSs; and (b) is a diagram showing the principle of 3D target reconstruction.

[0075] Figure 5 This relates to the relationship between the absolute phase and the Z-axis coordinate.

[0076] Figure 6(a) shows the fifth-order polynomial fitting curve of the Z coordinate at pixel (1024, 1224) using the PHM method and its residual.

[0077] Figure 6(b) shows the fifth-order polynomial fitting curve and residual of the Z coordinate at pixel (1024, 1224) in the method of the present invention.

[0078] Figure 6(c) shows the fitting residual of the PHM method at pixel (1024, 1224);

[0079] Figure 6(d) shows the fitting residual of the method of the present invention at pixel (1024, 1224);

[0080] Figure 7(a) shows the image of the ceramic plate being tested;

[0081] Figure 7(b) shows the stripe pattern after the surface morphology of the ceramic plate has been modulated and deformed;

[0082] Figure 7(c) shows the point cloud of the ceramic plate using the PHM method;

[0083] Figure 7(d) shows the point cloud of the ceramic plate obtained by the method of the present invention;

[0084] Figure 8(a) shows the color coding height distribution map of the PHM method;

[0085] Figure 8(b) is a color coding height distribution diagram of the method of the present invention;

[0086] Figure 8(c) shows the error histogram of the point cloud of the PHM method relative to the fitted ideal plane;

[0087] Figure 8(d) is the error histogram of the point cloud of the method of the present invention relative to the fitted ideal plane;

[0088] Figure 8(e) is a cross-sectional view of Figures 8(a) and 8(b);

[0089] Figure 9(a) shows an image of the microdevice under test;

[0090] Figure 9(b) shows the stripe pattern after being modulated and deformed by the complex morphology of the microdevice surface;

[0091] Figure 9(c) shows the point cloud of the PHM method;

[0092] Figure 9(d) shows the point cloud of the method of the present invention;

[0093] Figure 10(a) is a profile of the point cloud data in column 1600;

[0094] Figure 10(b) shows the Euclidean distance between corresponding points on the two point cloud curves in column 1600;

[0095] Figure 11 This is a flowchart of the steps of the present invention. Detailed Implementation

[0096] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention are described clearly and completely below. Obviously, the described embodiments are only some embodiments of this invention, not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0097] This method improves the flexibility and efficiency of 3D microscopy, allowing for one-time calibration of the microscopy system and enabling phase-to-Z coordinate conversion (PZC) without the need for a precision displacement stage (see...). Figure 1First, this invention uses the most advanced dual telecentric camera calibration method to accurately calibrate the dual telecentric camera. Second, this invention treats the pinhole tilt-shift light field projector as a reverse pinhole camera. With the help of the phase markers of the pixels, this invention establishes the mapping relationship between the dual telecentric camera image and the tilt-shift light field projector image, thereby successfully extracting the feature points in the projector image. Then, the pinhole tilt-shift light field projector is calibrated using the distortion correction calibration method based on Schahm's law in reference [2]. Due to tilt-shift and lens distortion, the light field projector is nonlinear. In order to obtain a linear projection system, this invention proposes an iterative optimization strategy to eliminate projector distortion and optimize the projection light field. Then, the 3D point cloud of the target under different poses is calculated using the external parameters of the pinhole tilt-shift light field projector and the parameters of the dual telecentric camera. Therefore, this invention fits the relationship between the phase and the Z coordinate based on the Z coordinate and phase value of each point on the target under different poses. In this way, the PZC relationship at each pixel can be obtained without a displacement stage. The (X,Y) coordinates of the measured point can be derived from the undistorted camera pixel coordinates using a dual telecentric imaging model. This establishes a high-precision and efficient 3D microscopic measurement model.

[0098] like Figure 2 , Figure 11 As shown, the 3D microscopic measurement method based on a tilt-shift projector and a dual telecentric camera of the present invention includes:

[0099] Step 1. Generation of pre-distortion fringe pattern

[0100] To compensate for the lens distortion and measurement errors caused by the tilt-shift of the pinhole tilt-shift light field projector, the principle projection image is subjected to inverse distortion processing based on the distortion model given in formula (4) and the calibration parameters of the pinhole tilt-shift light field projector, and then... Figure 3 The iterative optimization algorithm for generating pre-distorted fringes yields the optimized light field projection fringe pattern.

[0101] In this invention, the total number of iterations is 9. By projecting these pre-distorted horizontal and vertical phase-shifted fringe patterns, the pinhole-shift light field projector can project a distortion-free image, which means that the projector distortion is canceled. Thus, the pinhole-shift light field projector can be regarded as a linear projection system.

[0102] Step 2: Image Acquisition for Calibration. Place the planar target within the measurement volume. A pinhole tilt-shift optical field projector sequentially projects two sets of horizontal and vertical three-frequency N-step phase-shift fringe patterns. Simultaneously, the camera captures the modulation deformation fringes on the target, such as... Figure 1This invention repeats this process in several different target postures, using fringe frequencies f = {1, 8, 57}, with corresponding fringe phase shift step sizes N = {4, 4, 10}. f represents the three frequencies of the three sets of projected fringes (low frequency 1, mid frequency 8, high frequency 57), and N represents the phase shift step size corresponding to the fringes at the three frequencies;

[0103] Step 3: Dual Telecentric Camera Calibration. This invention utilizes the previously derived closed-form solution to calibrate the dual telecentric camera. First, the initial parameters of the dual telecentric camera are estimated, and then a constrained optimization algorithm is used to optimize the initial parameters to minimize the reprojection error. The calibration image is the average value of the high-frequency images captured in Step 2.

[0104] Step 4. Calibrate the tilt-shift pinhole projector. First, establish the correspondence between the pixels of the dual telecentric camera image and the pinhole tilt-shift light field projector image using a phase-assisted method. Then, based on this relationship, map the pixel coordinates of the feature points of the dual telecentric camera target image to the image coordinate system of the pinhole tilt-shift light field projector. Finally, according to formulas (3) and (4) and the algorithm described in [1,2], by inputting the world coordinates of the target features and the corresponding image coordinates of the pinhole tilt-shift light field projector, we obtain the intrinsic and extrinsic parameters of the pinhole tilt-shift light field projector.

[0105] s p (u p ν p 1) T =K p (R wp T wp (XYZ 1) T (3)

[0106] In Formula 3, s p As the scaling factor, (u p ν p K represents the pixel coordinates of the image point corresponding to the 3D world point (XYZ). p Let R be the intrinsic parameter matrix of the projector. wp ,T wp These are the rotation matrix and translation vector from the world coordinate system to the projector coordinate system, respectively.

[0107]

[0108] Let be the second-order radial distortion coefficient of the projector. Let be the fourth-order radial distortion coefficient of the projector. Let be the sixth-order radial distortion coefficient of the projector. Let be the tangential distortion coefficient in the x-direction of the projector. Let x be the tangential distortion coefficient in the y-direction of the projector.p Let x be the normalized coordinates of the projector image plane in the x-direction, and y be the normalized coordinates of the projector image plane in the x-direction. p Let r be the normalized coordinate of the y-axis of the projector image plane. p The normalized radius;

[0109] Step 5. Target reconstruction. (e.g.) Figure 4 As shown, point X on the target at position i. i (X i ,Y i Z i The image point of ) is q(u c ,ν c ). Pixel coordinates u c ,ν c And by substituting the calibration parameters into formula (7), we obtain the coordinate components, X. i Y i X i ,Y i Substituting into formula (6), we can obtain Z. i Repeat this step for each image point to reconstruct the target at position i. Using the same method, 3D models of targets in all poses can be obtained.

[0110]

[0111] For R i0 The elements in the last column, For T i0 Elements in R. i0 ,T i0 For WCS i The rotation matrix and translation vector of WCS0 are as follows: Figure 4 As shown in (a).

[0112]

[0113] m is the effective magnification of the dual telecentric lens. The pixel coordinates of the principal point in the camera image. and R represents the pixel size along the u and v directions, respectively. wc and T wc These are the rotation matrix and translation vector from the world coordinate system (WCS) to the dual telecentric camera coordinate system (CCS), respectively. For R wc The element in the i-th row and j-th column, For T wc Elements. Usually used They represent u represents the row direction of the image, and ν represents the column direction of the image.

[0114] Step 6: Calculate the PZC coefficient. For example... Figure 5 As shown, for each image point q(u) c ,ν c There are n+1 corresponding 3D world points, each located on the phase pose carried by one of the n+1 target points. Their Z-axis coordinate components and the phase values ​​they carry constitute the dataset {(Z)} for fitting the PZC polynomial parameters. wi ,Φ i ), i = 0, 1, ..., n-1, n}, where n+1 > M. Repeat this operation for each image point to build a pixel-wise PZC coefficient lookup table (LUT);

[0115] like Figure 4 As shown in (b), the image point q(u) c ,ν c The incident ray corresponding to the i-th pose intersects the target at point X. i (X i ,Y i Z i Z i Let X be the point i The third component. Φ i Let X be the point i The phase value it carries (that is, the phase label; we use the phase of the light field to mark each spatial point).

[0116] Step 7: Microdevice Image Acquisition. A set of orthogonal pre-distorted phase-shifted fringe patterns are projected onto the surface of the microdevice under test. Simultaneously, dual telecentric cameras capture the fringes modulated and deformed by the surface contour of the microdevice and acquire the absolute phase map.

[0117] Step 8: 3D reconstruction of the microdevice. The pixel coordinates (u... c ,v c Substitute the values ​​into formula (7) to obtain the coordinates (X, Y). Then, substitute the absolute phase and PZC coefficients (from LUT) into formula (11) to obtain the corresponding Z-axis coordinates.

[0118]

[0119] F is a polynomial function with independent variable Φ, where Φ is the pixel value (u). c ,v c The phase value corresponding to ) is Φ0, which is the pixel point (u c ,v c The phase value at the reference plane corresponding to (Φ-Φ0) n ΔΦ is (Φ-Φ0) raised to the power of n. n Let ΔΦ be the nth power. {a0, a1, ..., a m} represents the pixel-by-pixel PZC mapping coefficients to be calibrated.

[0120] Furthermore, the reconstruction of the target in different poses includes:

[0121] To achieve pixel-level PZC without using a precise translation stage, the coordinates (X, Y, Z) of each point on the target (in the WCS0 coordinate system) are obtained using the projector extrinsic parameters and the parameters of the dual telecentric cameras. In fact, the world coordinate system (WCS0) at pose i is first calculated. i The rotation matrix R of the world coordinate system (WCS0) at pose 0 i0 Translation vector T i0 Then, the planar equations of the target in WCS0 under other poses are calculated. Subsequently, the target at different poses can be reconstructed.

[0122] Figure 4 The relationship between PCS, CCS, and WCSs under different target attitudes is shown, where WCS0 is regarded as a universal (global) coordinate system.

[0123] Let R wpi and T wpi For WCS i The rotation matrix and translation vector to PCS, R wci and T wci For WCS i The rotation matrix and translation vector to the CCS. The dual telecentric cameras have a constant magnification, which makes T... wci The third component is 0. Let (R) pwi ,T pwi Let be the pose parameters of the i-th target relative to the PCS, then:

[0124]

[0125] R pw0 R is the transpose of the rotation matrix from the projector coordinate system to the world coordinate system. wp0 T represents the rotation matrix from the world coordinate system to the initial projector coordinate system. pw0 This represents the translation vector from the initial projector coordinate system to the world coordinate system. For R wpi The transpose matrix, R wpi For WCS i Rotation matrix to PCS;

[0126] - for Multiply by -1 in front;

[0127] For R pwi The transpose matrix, R pwiFor PCS to WCS i The rotation matrix;

[0128] T pw0 Let be the translation vector from PCS to WCS0;

[0129] R pw0 Let be the rotation matrix from PCS to WCS0;

[0130] Based on Equation 16 in the appendix of reference [6] (A novel projector ray-model for 3D measurement in fringeprojection profilometry), the plane equation of the target plane at pose i in WCS0 can be derived, and we have

[0131]

[0132] in, For R i0 The last column element, For T i0 The elements in. Let Formula (6) can be expressed as A i x+B i y+C i z+D i =0.

[0133] Let X = (X, Y, Z) be a point on the target, and q(u c ,ν c ) is its distortion-free pixel. (From the formula...) In the imaging model in 1), the X and Y coordinates are directly calculated from the calibration parameters of the dual telecentric cameras and the pixel coordinates of point q. The calculation formula is as follows:

[0134]

[0135] s c u is the scale factor for a dual telecentric camera. c Let ν be the x-axis pixel coordinate on the image plane of the dual telecentric camera. c The meaning is the y-axis pixel coordinate on the image plane of a dual telecentric camera, K cLet X be the intrinsic parameter matrix of the dual telecentric cameras, X be the x-axis coordinate of the target point in the 3D world coordinate system, Y be the y-axis coordinate of the target point in the 3D world coordinate system, and Z be the z-axis coordinate of the target point in the 3D world coordinate system. In formula (7), the extrinsic parameters are elements of the rotation matrix and translation vector from WCS0 to CCS. Substituting formula (7) into formula (6) yields the Z-coordinate. Therefore, the 3D coordinates of each point on the calibration target are obtained. Using this method, targets in different poses can be reconstructed.

[0136] Furthermore, the parametric Z-axis coordinate calculation includes:

[0137] O p O c These are the origins of the projector coordinate system and the dual telecentric camera coordinate system, respectively. The projector projects two sets of orthogonal fringe patterns onto the target. This creates a stable phase field, ensuring that every point on the target can be labeled with a phase value. Based on the ray path and imaging model, such as... Figure 5 As shown, the relationship between phase and Z-axis coordinate can be derived. The detailed derivation steps are described below:

[0138] Q0 is a point on the reference plane, and H is O. p and its projection point O' p The distance between them. Φ0 is the phase value carried by Q0, and L is the distance between Q0 and O'. p The distance between them. Figure 5 In the diagram, l', l'0, l1', l'2 represent values ​​from O. p Let $l$, $l0$, $l1$, $l2$ be the reflected rays of $Q$, and $q0$ be the image point of $Q$. A microdevice of height $Z$ is placed at $Q$, causing the phase at $q$ to change from $Φ$ to $Φ$. Assume that when the phase $Φ$ increases by 1, the corresponding change in distance on the reference plane is $λ$. Based on geometric relationships, we have:

[0139]

[0140] For image I c For any pixel on the measurement system, H, L, Φ0, and λ0 are all constants. Therefore, formula (8) can be rewritten as follows:

[0141]

[0142] Where, {c0,c i ,…,c m , g} represents the pixel-wise mapping coefficients to be calibrated, and M is the degree of the polynomial fitting function. c0 represents the coefficients of the constant term in the polynomial fitting, c i c represents the coefficient of the i-th term in the phase-height mapping relationship. mrepresents the coefficient of the highest-degree term of the polynomial, and g represents the global calibration parameter or system-level parameter. For computational convenience, F(Φ) is expanded using Taylor series at Φ0.

[0143]

[0144] For any pixel F(Φ0), F'(Φ0), F""Φ0",...,F (M) (Φ0) and Φ0 are constants, and formula (10) can be rewritten in the following form.

[0145]

[0146] Where ΔΦ = Φ - Φ0. Here, these coefficients are fitted using the obtained 3D point cloud data of the target and the phase carried by the target points.

[0147] like Figure 5 As shown, the reflected ray intersects the target at pose i at point X. i =(X i ,Y i Z i ), whose image point is q(u c ,ν c Let Φ i For q and X i The phase carried by Z can be calculated using formulas (6) and (7). i Similarly, points X0, X1, ..., X can be calculated. n-1 ,X n The Z-axis coordinates Z0, Z1, ..., Z in the universal world coordinate system n-1 Z n This yields n+1 sets of Z-axis coordinates and their corresponding phase values ​​{(Z wi , Φ i The polynomial parameters of PZC can be fitted using the following formula: ), i = 0, 1, ..., n-1, n}, where n+1 > M.

[0148] Example

[0149] The practicality and effectiveness of the proposed three-dimensional microscopy measurement method were experimentally analyzed. The FPP microscopy system included a camera (MER2-502-79U3M, resolution 2448×2048) equipped with dual telecentric lenses (DTAC 230-37C, 0.3×magnigication), a custom-designed tilt-shift light field projector (DLP4500, resolution 912×1140) equipped with a standard lens, and a manual precision stage (GCM-VC13M) with a range of 13 mm and an accuracy of 0.01 mm. The angle between the camera optical axis and the projector optical axis was approximately 22°.

[0150] A planar target was used, with a 9×9 array of white dots evenly distributed on a dark blue background, and the distance between adjacent feature points was 1.5 mm. Furthermore, a ceramic plate was used as the standard plane, and watch parts were used as test samples to evaluate the 3D reconstruction accuracy of the proposed method. Experiments included system calibration and 3D reconstruction.

[0151] System calibration

[0152] During calibration, targets were placed at nine different locations to cover a measurement volume of 24mm × 28mm × 6mm. Each target location could be used to calibrate the dual telecentric camera, projector, and PZC coefficients. Furthermore, to improve the algorithm's accuracy and for comparative analysis using the PHM method, nine additional target images with different calibration poses were collected. These additional poses were obtained by translating the target within the measurement volume along the Z-direction from far to near using a translation stage. Additionally, nine ceramic plate images with the same pose (projecting only vertical phase fringe patterns) were acquired. Specifically, the ceramic plate was placed on a translation stage, and nine fringe images of the ceramic plate at different heights were captured by manually moving the stage to implement the PHM method. The height calculated based on the modulated fringe image of the ceramic plate and the planar equations of the nine additional targets with different poses was used to replace the scale of the translation stage.

[0153] First, calibrate the camera and projector. Then, based on the projector's extrinsic parameters, estimate the coefficients of the target's planar equation in WCS0 at any pose. The calibration parameters for the camera and projector are shown in Table 1, and the coefficients of the target's planar equation are shown in Table 2.

[0154] Table 1. Parameters of the dual telecentric camera and the tilt-shift light field projector

[0155]

[0156] Table 2. Planar equations of the target at different poses

[0157] A B C D A B C D <![CDATA[Pose1]]> 0.01369 0.04003 0.99910 -0.93471 <![CDATA[Pose 10 ]]> 0.00042 -0.00034 1.00000 -3.00008 <![CDATA[Pose2]]> -0.02521 0.07378 0.99695 1.81959 <![CDATA[Pose 11 ]]> 0.00036 -0.00016 1.00000 -2.50562 <![CDATA[Pose3]]> -0.03531 -0.08366 0.99587 0.89024 <![CDATA[Pose 12 ]]> 0.00024 -0.00013 1.00000 -2.00098 <![CDATA[Pose4]]> -0.04540 -0.0308 0.99849 0.08989 <![CDATA[Pose 13 ]]> 0.00009 0.00002 1.00000 -0.99678 <![CDATA[Pose5]]> 0.07461 0.02598 0.99687 -1.97645 <![CDATA[Pose 14 ]]> 0 0 1.00000 0 <![CDATA[Pose6]]> 0.09458 -0.00380 0.99551 -1.00312 <![CDATA[Pose 15 ]]> -0.00020 0.00001 1.00000 1.00552 <![CDATA[Pose7]]> 0.04232 0.02871 0.99869 -0.42192 <![CDATA[Pose 16 ]]> -0.00021 0.00013 1.00000 2.00541 <![CDATA[Pose8]]> 0.02800 0.02247 0.99935 -0.45289 <![CDATA[Pose 17 ]]> -0.00027 0.00023 1.00000 2.51008 <![CDATA[Pose9]]> -0.00857 -0.03697 0.99928 -0.98175 <![CDATA[Pose 18 ]]> -0.00024 0.00042 1.00000 2.99462

[0158] In their paper "Discussion on accurate phase-height mapping in fringeprojection profilometry", Zhao et al.

[10] systematically compared the measurement effects of four polynomial fitting methods based on absolute phase: linear fitting, quadratic fitting, cubic fitting (CF). The results showed that the reconstruction accuracy of PLF was similar to that of CF, and better than that of linear fitting and quadratic fitting. Higher-order polynomial fitting methods often have better fitting effects. Here, let m = 5. The PHM relationship at each pixel is described by the following formula:

[0159] Z = a0 + a1Φ + a2Φ + ... + a M Φ M (12)

[0160] Where Φ is the absolute phase, a0~a M The fitting coefficients are given at each pixel, and M is the degree of the fitting polynomial.

[0161] To ensure the approximation accuracy of the Taylor expansion in equation (10), a Taylor expansion of a 5th-order polynomial function is chosen. The performance of the PHM method will be analyzed in detail in the following experimental process.

[0162] Using the calibration parameters in Tables 1 and 2, the PZC coefficients of the polynomial function were calculated. Sufficient phase values ​​and corresponding Z values ​​were collected by translating the ceramic plate at equal intervals (0.5 mm) along the Z-axis. Table 3 shows the PZC coefficients and RMSEs of the method and the PHM method at image pixels (1024, 1224), as well as the mean μ and standard deviation σ of Φ and ΔΦ. Figures 6(a) and 6(b) show that the phase-Z relationship is approximately linear, and the small fitting residuals (see Figures 6(c) and 6(d)) indicate that the polynomial fitting achieves good results. Once the PZC coefficients are determined, 3D reconstruction can be performed efficiently.

[0163] Table 3 shows the PZC coefficients at pixels (1024, 1224).

[0164] <![CDATA[a0]]> <![CDATA[a1]]> <![CDATA[a2]]> <![CDATA[a3]]> <![CDATA[a4]]> <![CDATA[a5]]> <![CDATA[μ Φ / m ΔΦ ]]> <![CDATA[σ Φ / s ΔΦ ]]> RMSE PHM 0.0379 2.2661 -0.0496 -0.0367 0.0049 0.0168 177.4756 7.0339 0.00056 Ours 0.0413 2.2541 -0.0653 -0.0095 0.0140 0.0046 0.1491 7.0215 0.00045

[0165] Measuring standard plane

[0166] To evaluate the reconstruction accuracy of the proposed 3D microscopy method and compare it with the PHM method, a ceramic plate (7cm × 7cm) was measured within a range of approximately -1.5mm to 1.5mm from the reference plane (located 110mm from the camera). The flatness of the ceramic plate was greater than 10μm. A pre-distorted vertical fringe pattern was projected onto the ceramic plate, and the fringe pattern after ceramic modulation deformation was captured by the camera, as shown in Figure 7(b) (the original image of the ceramic plate is shown in Figure 7(a)). The ceramic plate was first placed parallel to the reference plane and then translated 4 times along the Z-axis. The distance between adjacent positions was 1mm. The point cloud of the ceramic plate at each position was calculated, and the point cloud of the PHM method is shown in Figure 7(c) and the point cloud of the proposed method is shown in Figure 7(d). Then, the average height and absolute error of 500×500 pixels in the central region were calculated (reference value is the displacement stage scale). The relevant results are shown in Table 4, and the data show that the measurement accuracy of the proposed method is comparable to that of the PHM method.

[0167] Furthermore, this invention reconstructed a ceramic plate at a height of 0.5 mm relative to a reference plane, and the height distribution of the color-coded data using the PHM method and the method of this invention is shown in Figures 8(a) and 8(b), respectively. Additionally, Figure 8(e) shows the cross-section of the midline of Figures 8(a) and 8(b), with similar amplitudes for the two curves. The calculated point cloud data was then fitted to an ideal plane using the least squares method. Figures 8(c) and 8(d) show the error histograms between the real point cloud and the ideal plane for both methods. Most errors are within ±10 μm, consistent with the flatness of the ceramic plate. The root mean square error (RMSE) from the point cloud to the ideal plane for each point was also calculated, which are 0.00235 mm (PHM) and 0.00269 mm, respectively. (From Table 4 and...) Figures 8(a)-8(e) As can be seen, the results of the method of the present invention are highly consistent with the results of PHM.

[0168] To verify the practicality of the method of this invention in various complex scenarios in the real world, reconstruction was performed on a complex 3D microdevice, as shown in Figure 9(a). During the experiment, a set of modulation distortion (MDD) fringe images was used, and an example MMD image is shown in Figure 9(b). First, the phase value was converted to the Z coordinate using formula (11) to obtain the Z coordinate. Then, the (X,Y) coordinates were calculated according to formula (7). Thus, the three-dimensional coordinates of each pixel were obtained, forming the final point cloud, as shown in Figures 9(c) and 9(d). In Figure 9(c), the PHM point cloud data corresponding to the 1600 columns of pixels in Figure 9(a) is marked with a dark curve, and in Figure 9(d), these points are marked in the point cloud data with a dark curve.

[0169] The profiles corresponding to the dark curves in Figures 9(c) and 9(d) are shown in Figure 10(a). Figure 10(a) illustrates the relationship between the Z and X values ​​of the point cloud curves, showing that the two dark curves are very close together. For further quantitative analysis, the Euclidean distance between the corresponding 3D points on the curves in Figures 9(c) and 9(d) was calculated, as shown in Figure 10(b). The maximum absolute difference between the two lines is 0.0099 mm, and the average absolute difference is 0.0023 mm. This demonstrates that both PHM and the method of this invention can capture similar levels of detail. Experiments show that even without using a precise displacement stage, the method of this invention can achieve similar accuracy to the PHM method.

[0170] Note:

[0171] [1]: W. Zhang, W. Li, L. Yu, H. Luo, H. Zhao, and H. Xia, "Sub-pixel projectorcalibration method for fringe projection profilometry," Opt. Express 25(16), 19158–19169(2017).

[0172] [2]: J.Peng, M.Wang, D.Deng,

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D microscopic measurement method based on a tilt-shift projector and dual telecentric cameras, characterized in that, include: Step 1. Generation of pre-distortion fringe pattern To compensate for the measurement errors caused by lens distortion and tilt-shift in the pinhole tilt-shift light field projector, the inverse distortion processing of the theoretical projection pattern is performed based on the distortion model given in formula (4) and the calibration parameters of the pinhole tilt-shift light field projector. The optimized light field projection fringe pattern is then obtained through an iterative optimization algorithm for generating pre-distorted fringes. Step 2: Calibration Image Acquisition A planar target is placed in the measurement volume. A pinhole shift-axis light field projector sequentially projects two sets of horizontal and vertical three-frequency N-step phase-shifted fringe patterns. At the same time, a camera captures the modulation and deformation fringes on the target. This process is repeated in several different target postures. The fringe frequency used is f = {1, 8, 57}, and the corresponding fringe phase-shift step size is N = {4, 4, 10}. Step 3: Dual telecentric camera calibration Using the previously derived closed-loop solution, the dual telecentric camera is calibrated, including estimating the initial parameters of the dual telecentric camera, and then using a constrained optimization algorithm to optimize the initial parameters to minimize the reprojection error. The calibration image is the average value of the high-frequency images captured in step 2. Step 4. Calibrate the tilt-shift pinhole projector First, the phase-assisted method is used to establish the correspondence between the pixels of the dual telecentric camera image and the pinhole shift light field projector image. Then, based on this relationship, the pixel coordinates of the obtained dual telecentric camera target image feature points are mapped to the image coordinate system of the pinhole shift light field projector. Finally, according to formulas (3) and (4), as well as the sub-pixel level projector calibration algorithm and the distortion correction algorithm for Sham's law, the projector is calibrated. That is, by inputting the world coordinates of the target feature points and the corresponding pinhole shift light field projector image feature point coordinates, the intrinsic and extrinsic parameters of the pinhole shift light field projector are obtained. s p (u p ν p 1) T =K p (R wp T wp )(X Y Z 1) T , (3) In formula (3), s p As the scaling factor, (u p ν p K represents the pixel coordinates of the image point corresponding to the 3D world point (XYZ). p Let R be the intrinsic parameter matrix of the projector. wp ,T wp These are the rotation matrix and translation vector from the world coordinate system to the projector coordinate system, respectively. Let be the second-order radial distortion coefficient of the projector. Let be the fourth-order radial distortion coefficient of the projector. Let be the sixth-order radial distortion coefficient of the projector. Let be the tangential distortion coefficient in the x-direction of the projector. Let x be the tangential distortion coefficient in the y-direction of the projector. p Let x be the normalized coordinates of the projector image plane in the x-direction, and y be the normalized coordinates of the projector image plane in the x-direction. p Let r be the normalized coordinate of the y-axis of the projector image plane. p The normalized radius; Step 5. Target Reconstruction Point X on the target at position i i (X i ,Y i Z i The image point of ) is q(u c ,ν c ), to set pixel coordinates u c ,ν c And by substituting the calibration parameters into formula (7), we can obtain the coordinate components X. i Y i Then X i ,Y i Substituting into formula (6), we get Z i Repeat this step for each image point to reconstruct the target at position i. Use the same method to obtain the 3D model of all pose targets. The reconstruction of the target in different poses includes: To achieve pixel-level PZC without using a precise translation stage, the coordinates (X, Y, Z) of each point on the target are obtained using the projector extrinsic parameters and the parameters of the dual telecentric cameras. In fact, the world coordinate system (WCS) at pose i is first calculated. i The rotation matrix R of the world coordinate system WCS0 at pose 0 i0 Translation vector T i0 Then, the plane equation of the target in WCS0 under other poses is calculated, and then the target at different poses is reconstructed. Let R wpi and T wpi For WCS i The rotation matrix and translation vector to PCS, R wci and T wci For WCS i The rotation matrix and translation vector to the CCS, along with the constant magnification characteristic of the dual telecentric cameras, make T wci The third component is 0, let (R) pwi ,T pwi Let be the pose parameters of the i-th target relative to the PCS, then: R pw0 R is the transpose of the rotation matrix from the projector coordinate system to the world coordinate system. wp0 T represents the rotation matrix from the world coordinate system to the initial projector coordinate system. pw0 Let represent the translation vector from the initial projector coordinate system to the world coordinate system. Derive the plane equation of the target plane at pose i in WCS0, and have... in, For R i0 The last column element, For T i0 The elements in, let Representing formula (6) as A i x+B i y+C i z+D i =0; Let X = (X, Y, Z) be a point on the target, and q(u c ,ν c ) is its distortion-free pixel, determined by the formula In the imaging model, the X and Y coordinates are directly calculated from the calibration parameters of the dual telecentric cameras and the pixel coordinates of point q. The calculation formula is as follows: s c u is the scale factor for a dual telecentric camera. c Let ν be the x-axis pixel coordinate on the image plane of the dual telecentric camera. c The meaning is the y-axis pixel coordinate on the image plane of a dual telecentric camera, K c Let be the intrinsic parameter matrix of the dual telecentric camera, where X is the x-axis coordinate of the target point in the 3D world coordinate system, Y is the y-axis coordinate of the target point in the 3D world coordinate system, and Z is the z-axis coordinate of the target point in the 3D world coordinate system. In the formula, m is the effective magnification of the dual telecentric lens. The pixel coordinates of the principal point in the camera image. and R represents the pixel size along the u and v directions, respectively. wc and T wc These are the rotation matrix and translation vector from the world coordinate system to the dual telecentric camera coordinate system, respectively. For R wc The element in the i-th row and j-th column, For T wc Elements, usually represented by f u c ,f v c They represent u represents the row direction of the image, and ν represents the column direction of the image; In formula (7), the external parameters are the elements in the rotation matrix and translation vector from WCS0 to CCS. Substituting formula (7) into formula (6) yields the Z coordinate. Thus, the 3D coordinates of each point on the calibration target are obtained. In this way, the target under different postures is reconstructed. CCS is the coordinate system of the dual telecentric camera and PCS is the coordinate system of the projector. Step 6: Calculation of PZC coefficient Each image point q(u c ,ν c There are n+1 corresponding 3D world points, located at n+1 target poses respectively. Their Z-axis coordinate components and the phase values ​​they carry constitute the dataset {(Z)} used to fit the PZC polynomial parameters. wi ,Φ i ),i=0,1,...,n-1,n}, where n+1>M, repeat this operation for each image point to build a pixel-by-pixel PZC coefficient lookup table; Step 7: Microdevice Image Acquisition A set of orthogonal pre-distorted phase-shifted fringe patterns is projected onto the surface of the microdevice under test, while dual telecentric cameras capture the fringes that are modulated and deformed by the surface profile of the microdevice and obtain an absolute phase map. Step 8: Three-dimensional reconstruction of microdevices pixel coordinates (u c ,v c Substituting these values ​​into formula (7) yields the coordinates (X, Y). Then, substituting the absolute phase and PZC coefficients into formula (11) yields the corresponding Z-axis coordinates. F is a polynomial function with independent variable Φ, where Φ is the pixel value (u). c ,v c The phase value corresponding to ) is Φ0, which is the pixel point (u c ,v c The phase value at the reference plane corresponding to (Φ-Φ0) n ΔΦ is (Φ-Φ0) raised to the power of n. n Let ΔΦ be the power of n, {a0, a1, ..., a m } represents the pixel-by-pixel PZC mapping coefficients to be calibrated, ΔΦ=Φ-Φ0, where the obtained 3D point cloud data of the target and the phase carried by the target points are used to fit these coefficients.

2. The 3D microscopic measurement method based on a tilt-shift projector and a dual telecentric camera according to claim 1, characterized in that, In step 1, the total number of iterations is 9. By projecting these pre-distorted horizontal and vertical phase-shifted fringe patterns, the pinhole shift light field projector projects a distortion-free image, which means that the projector distortion is canceled. Thus, the pinhole shift light field projector is regarded as a linear projection system.

3. The 3D microscopic measurement method based on a tilt-shift projector and a dual telecentric camera according to claim 1, characterized in that, In step 6, the parameterized Z-axis coordinate is calculated as follows: O p O c The origins of the projector coordinate system and the dual telecentric camera coordinate system are respectively. The projector projects two sets of orthogonal fringe patterns onto the target, thus forming a stable phase field. This allows each point on the target to be marked with a phase value. Based on the ray path and imaging model, the relationship between the phase and the Z-axis coordinate is derived. The derivation steps include: (1). Q0 is a point on the reference plane, and H is O. p and its projection point O' p The distance between them, Φ0 is the phase value carried by Q0, and L is the distance between Q0 and O'. p The distances between them, l', l'0, l1', l'2, are from O p The reflected rays l, l0, l1, l2 are given, and q0 is the image point of Q0. A micro-device of height Z is placed at Q0, causing the phase at point q0 to change from Φ0 to Φ. Assuming that when the phase Φ0 increases by 1, the corresponding change in distance on the reference plane is λ0, according to geometric relationships, we have: (2). For image I c For any pixel on the graph, once the measurement system is determined, H, L, Φ0, and λ0 are all constants. Therefore, formula (8) can be rewritten as follows: Where, {c0, c i c m , g} represents the pixel-wise mapping coefficients to be calibrated, M is the degree of the polynomial fitting function, and for computational convenience, F(Φ) is Taylor-expanded at Φ0, c0 represents the coefficients of the constant term in the polynomial fitting, c i c represents the coefficient of the i-th term in the phase-height mapping relationship. m The coefficients of the highest-degree term of the polynomial are represented by g, and g represents the global calibration parameter or system-level parameter. For any pixel F(Φ0), F'(Φ0), F""Φ0",...,F (M) (Φ0) and Φ0 are constants. Formula (10) can be rewritten in the following form: Where F is a polynomial function with independent variable Φ, and Φ is the pixel value (u c ,v c The phase value corresponding to ) is Φ0, which is the pixel point (u c ,v c The phase value at the reference plane corresponding to (Φ-Φ0) n ΔΦ is (Φ-Φ0) raised to the power of n. n Let ΔΦ be the power of n, {a0, a1, ..., a m } represents the pixel-wise PZC mapping coefficients to be calibrated, ΔΦ = Φ - Φ0. Here, the obtained 3D point cloud data of the target and the phase carried by the target points are used to fit these coefficients. (3) The reflected ray intersects the target at position i at point X. i =(X i ,Y i Z i ), whose image point is q(u c ,ν c Let Φ i For q and X i The phase carried is calculated by formulas (6) and (7) for Z. i Similarly, calculate points X0, X1, ..., X n-1 ,X n The Z-axis coordinates Z0, Z1, ..., Z in the universal world coordinate system n-1 Z n This yields n+1 sets of Z-axis coordinates and their corresponding phase values ​​{(Z wi , Φ i ), i = 0, 1, ..., n-1, n}, are used to fit the polynomial parameters of PZC, usually n+1 > M.

Images