Calibration method for telecentric three-dimensional measurement system

Abstract

The invention relates to a telecentric three-dimensional measurement system calibration method, which comprises the following steps that a virtual 3D calibration target composed of a plurality of calibration planes parallel to one another is constructed, and a plurality of calibration feature points are arranged on the calibration planes; using the virtual 3D calibration target to calibrate the telecentric camera; a global homography relation between a camera pixel coordinate system and a projector pixel coordinate system is utilized to optimize projector pixel coordinates of the calibrated feature points; according to the optimized pixel coordinates of the projector, calibration of the telecentric projector is completed; and performing joint calibration on the camera and the projector based on a beam adjustment method. The method can avoid falling into a plane local optimal solution, effectively solves the influence of a phase error, and obtains a more accurate calibration result.

Description

Technical Field

[0001] The invention relates to the technical field of telecentric three-dimensional measurement, and in particular to a telecentric three-dimensional measurement system calibration method. Background Art

[0002] Microscopic Fringe Projection Profilometry (MFPP) is an important microscopic measurement technology with wide applications in precision manufacturing and industrial inspection. It uses the phase information provided by the fringe image to achieve pixel matching between the camera and the projector, and finally uses the triangulation principle and system calibration parameters to calculate the three-dimensional coordinates of the object to be measured. Compared with traditional long working distance lenses, telecentric lenses have constant magnification, greater depth of field (DOF) and near-zero distortion, and have gradually become an important component in MFPP systems.

[0003] The accuracy of MFPP system measurement depends greatly on the accuracy of system calibration, which includes camera calibration and projector calibration. However, due to the unique orthographic projection property of telecentric lenses, the calibration method of pinhole lenses cannot be used directly. In order to solve this problem, many methods model the imaging process of telecentric lenses and use plane calibration plates to calibrate telecentric lenses. However, the insensitivity of telecentric lenses to depth and the lack of depth information of calibration plates lead to problems such as parameter sign ambiguity, solution failure, and falling into local optimal solutions in the plane. On the other hand, since the projector cannot directly image, it is usually regarded as an inverse camera, and the projector is calibrated through a phase-coordinate conversion scheme. However, the phase information is affected by the nonlinear effect of the projector and the reflection characteristics of the object surface, and there is often a large phase error, which directly affects the calibration accuracy of the projector. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a telecentric three-dimensional measurement system calibration method, which can avoid falling into a local optimal solution of a plane and obtain a more accurate calibration result.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a telecentric three-dimensional measurement system calibration method, comprising the following steps:

[0006] Constructing a virtual 3D calibration target consisting of a plurality of mutually parallel calibration planes, wherein a plurality of calibration feature points are arranged on the calibration planes;

[0007] The virtual 3D calibration target is used to calibrate the telecentric three-dimensional measurement system.

[0008] Furthermore, the calibrating the telecentric three-dimensional measurement system using the virtual 3D calibration target includes:

[0009] Detecting and obtaining the camera pixel coordinates of the calibrated feature points;

[0010] According to the conversion relationship between the world coordinate system and the camera coordinate system, a camera pixel coordinate model of any calibration feature point in the virtual 3D calibration target projected onto the camera plane is established;

[0011] Converting the camera pixel coordinate model into a least squares problem;

[0012] Solving the camera pixel coordinate model using the camera pixel coordinates and world coordinates of the calibrated feature points to obtain initial values ​​of camera intrinsic parameters and initial values ​​of camera extrinsic parameters;

[0013] Set the initial values ​​of the camera distortion parameters;

[0014] According to the camera pixel coordinates and world coordinates of the calibration feature points under different postures, the least squares optimization algorithm is used to solve the camera internal parameters, the camera external parameters and the camera distortion parameters that minimize the sum of the square errors of the camera pixel coordinates of all the calibration feature points as the first camera parameters, and the camera pixel coordinate square error is the square of the difference between the camera pixel coordinate of the current calibration feature point and its camera pixel coordinate model.

[0015] Furthermore, solving the camera pixel coordinate model by using the camera pixel coordinates and world coordinates of the calibrated feature points to obtain initial values ​​of camera intrinsic parameters and initial values ​​of camera extrinsic parameters includes:

[0016] Substituting the camera pixel coordinates and world coordinates of the calibrated feature points into the camera pixel coordinate model;

[0017] Solving the projection matrix of the camera pixel coordinate model by using a singular value decomposition method, wherein the projection matrix of the camera pixel coordinate model is the product of its intrinsic parameter matrix and its extrinsic parameter matrix;

[0018] Based on the orthogonality and unit of the rotation matrix, the projection matrix of the camera pixel coordinate model is decomposed to obtain the initial values ​​of the camera internal parameters, and then the initial values ​​of the camera external parameters are calculated.

[0019] Furthermore, the camera pixel coordinates of the calibrated feature points are detected and obtained through a circle center extraction algorithm.

[0020] Furthermore, it also includes:

[0021] According to the homography transformation relationship between the world coordinate system and the camera coordinate system and the projector coordinate system, a transformation relationship model between the camera coordinate system and the projector coordinate system is established;

[0022] Obtaining projector pixel coordinates of a plurality of reference feature points within a set area on the calibration plane, wherein the set area does not include the calibration feature points;

[0023] Solving the conversion relationship model using the camera pixel coordinates and the projector pixel coordinates of the reference feature points;

[0024] Calculating the projector pixel coordinates of the calibrated feature points based on the conversion relationship model;

[0025] According to the conversion relationship between the world coordinate system and the projector coordinate system, a projector pixel coordinate model of any calibration feature point in the virtual 3D calibration target projected onto the projector plane is established;

[0026] The projector pixel coordinate model is solved using the projector pixel coordinates and world coordinates of the calibrated feature points to obtain a first projector parameter that minimizes the sum of the square errors of the projector pixel coordinates of all the calibrated feature points, wherein the projector pixel coordinate square error is the square of the difference between the projector pixel coordinate of the current calibrated feature point and its projector pixel coordinate model.

[0027] Furthermore, the step of obtaining the projector pixel coordinates of a plurality of reference feature points within a set area on the calibration plane includes:

[0028] The absolute phase information of the projector is obtained by using the four-step phase shift method and multi-frequency unfolding technology;

[0029] Extracting a set area on the calibration plane and determining reference feature points by a threshold segmentation method;

[0030] The projector pixel coordinates of the reference feature point are calculated based on the camera pixel coordinates of the reference feature point and the absolute phase information.

[0031] Furthermore, the method of solving the projector pixel coordinate model by using the projector pixel coordinates and world coordinates of the calibrated feature points to obtain the first projector parameter that minimizes the sum of square errors of the projector pixel coordinates of all the calibrated feature points includes:

[0032] Converting the projector pixel coordinate model into a least squares problem;

[0033] Solving the projector pixel coordinate model using the projector pixel coordinates and world coordinates of the calibrated feature points to obtain initial values ​​of projector internal parameters and initial values ​​of projector external parameters;

[0034] Set the initial values ​​of the projector distortion parameters;

[0035] According to the projector pixel coordinates and world coordinates of the calibrated feature points in different postures, a least squares optimization algorithm is used to solve the projector internal parameters, the projector external parameters and the projector distortion parameters that minimize the sum of the square errors of the projector pixel coordinates of all the calibrated feature points as the first projector parameters.

[0036] Furthermore, the method of calculating the initial values ​​of the internal parameters of the projector and the initial values ​​of the external parameters of the projector by using the projector pixel coordinates and the world coordinates of the calibrated feature points includes:

[0037] Substituting the projector pixel coordinates and world coordinates of the calibrated feature points into the projector pixel coordinate model;

[0038] The projection matrix of the projector pixel coordinate model is solved by using a singular value decomposition method, wherein the projection matrix of the projector pixel coordinate model is the product of its internal parameter matrix and its external parameter matrix;

[0039] Based on the orthogonality and unit of the rotation matrix, the projection matrix of the projector pixel coordinate model is decomposed to obtain the initial values ​​of the internal parameters of the projector, and then the initial values ​​of the external parameters of the projector are calculated.

[0040] Furthermore, it also includes:

[0041] Taking the first camera parameters and the first projector parameters as initial values, and according to the camera pixel coordinates, projector pixel coordinates and world coordinates of the calibration feature points in different postures, using a least squares optimization algorithm to solve the camera internal parameters, the camera external parameters, the camera distortion parameters, the projector internal parameters, the projector external parameters and the projector distortion parameters that minimize the sum of the global pixel coordinate square errors of all the calibration feature points, the global pixel coordinate square error being the sum of the camera pixel coordinate square error of the current calibration feature point and the projector pixel coordinate square error thereof.

[0042] Furthermore, the virtual 3D calibration target is obtained by controlling the plane calibration plate to translate multiple times in a direction perpendicular to the plane calibration plate.

[0043] Beneficial Effects

[0044] Due to the adoption of the above-mentioned technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention can solve the global optimal solution of telecentric camera parameters by constructing a virtual 3D calibration target containing depth information; in the calibration of the telecentric projector, the present invention optimizes the projector pixel coordinates corresponding to the feature points through the global homography relationship between the camera coordinate system and the projector coordinate system, thereby improving the calibration accuracy of the projector; the present invention also realizes the joint optimization of the camera and the projector by constructing the reprojection error, thereby further improving the calibration accuracy of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of an embodiment of the present invention;

[0046] Figure 2 is a schematic diagram of a virtual 3D calibration target according to an embodiment of the present invention;

[0047] Figure 3 is a schematic diagram of a calibration image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0049] The embodiments of the present invention relate to a high-precision, noise-robust MFPP system calibration method, such as Figure 1 As shown, specifically including:

[0050] (i) Telecentric camera calibration based on virtual 3D calibration target;

[0051] (ii) Telecentric projector calibration based on global homography;

[0052] (iii) Camera-projector joint calibration.

[0053] The following is a further description of each of the above steps.

[0054] (I) Telecentric camera calibration based on virtual 3D calibration target

[0055] Use a precision translation stage and a flat calibration plate to construct a virtual 3D calibration target. Figure 2 As shown in the figure, the Z axis of the world coordinate system is specified to be perpendicular to the calibration plane, and the precision translation stage is controlled to translate a certain distance along the Z axis multiple times to obtain a virtual calibration target composed of multiple layers of planes [(X w ,Y w,d1),(X w ,Y w ,d2),…,(X w ,Y w ,d N )].

[0056] Since the telecentric camera has a unique orthographic projection property, the image size is not affected by the distance of the object. The projection process is a transformation relationship from the world coordinate system to the camera coordinate system and then to the camera pixel coordinate system. Its mathematical model can be expressed as:

[0057]

[0058] Among them, M c Represents the projection matrix, which is the internal parameter matrix K c and the external parameter matrix [R c ,T c ], α c and β c are the magnifications in the horizontal and vertical directions respectively; γ c is the coefficient related to the tilt angle of the image plane; is the center of the image; and They are the rotation matrix R from the world coordinate system to the camera coordinate system c and the translation vector T c Elements; (X w ,Y w ,Z w ) is the three-dimensional coordinate of the object; (u c ,v c ) are the pixel coordinates in the camera image plane.

[0059] Since the distortion of a telecentric lens is very small and the radial distortion is much greater than the tangential distortion, usually only the radial distortion is considered, and the distortion model is expressed as:

[0060]

[0061] Among them, (u c ,v c ) is the image coordinate under ideal conditions; is the actual distorted image coordinate; is the radial distortion coefficient.

[0062] From formula (1), we can see that for each feature point, its projection process to the camera plane can be expressed as:

[0063]

[0064] Use M cRepresents the projection matrix, and transforms formula (3) into the representation of the least squares problem:

[0065]

[0066] Substitute the camera pixel coordinates (which can be obtained by the circle center extraction algorithm) and the world coordinates of the detected feature points into formula (4), and use singular value decomposition (SVD) to realize the projection matrix M c Solve. Using the rotation matrix R c The orthogonality and unity of Mc are decomposed to obtain the initial values ​​of the internal parameters:

[0067]

[0068] After obtaining the complete initial internal parameter matrix Kc, the corresponding initial external parameter matrix is ​​also restored:

[0069] [Rc,T](K)-1Mc(6)

[0070] Considering the influence of observation noise, the solution process of formula (5) actually concentrates the error caused by noise on the internal parameters to ensure that the rotation matrix Rc can satisfy the unity and orthogonality. For calibration images at different postures, the external parameters of the camera will change, but the internal parameters of the camera will not change. Therefore, the internal parameters can be further optimized through calibration images at different postures to average the influence of observation noise.

[0071] Considering the distortion of the telecentric camera, the initial value of the distortion parameter is set to zero, and the initial values ​​of the internal and external parameters are calculated using the above method. The following optimization problem is defined:

[0072]

[0073] in, represents the camera pixel coordinates of the jth circle center feature point in the i-th pose, Represents the pixel coordinates of the world point Pij projected onto the camera image plane. This optimization problem can be solved by the Levenberg-Marquardt (LM) algorithm to optimize the scalar Finally, all the parameters of the telecentric camera are obtained.

[0074] (II) Telecentric projector calibration based on global homography

[0075] This embodiment uses a four-step phase shift method and multi-frequency expansion technology to obtain the horizontal absolute phase and vertical absolute phase The corresponding projector pixel coordinates can be obtained as follows:

[0076]

[0077] Where nh, nv represent the number of projected horizontal and vertical stripes, respectively, and Wp×Hp is the resolution of the projector image.

[0078] The calibration accuracy of the projector depends largely on the projector pixel coordinates obtained by the phase-coordinate conversion method. However, due to the inherent nonlinear effects of the projector and the difference in the reflectivity of the object surface, it is difficult to obtain a high-quality phase map during the calibration process. Figure 3 As shown, many phase abnormal points can be seen around the black circle, which will cause deviations in coordinate calculation and affect the calibration accuracy of the projector. Therefore, the projector pixel coordinates need to be optimized.

[0079] For points on the plane calibration plate, their world coordinates satisfy the plane constraint:

[0080] AX+BYwCZw+D=0(9)

[0081] Where A, B, C, and D are the parameters of the plane equation. Combining formula (3) and formula (9), we can get the homography transformation relationship from the world coordinates of the feature point to the camera pixel coordinates:

[0082]

[0083] Similarly, the homography transformation relationship from the world coordinates of the feature points to the projector pixel coordinates can be obtained:

[0084]

[0085] Combining formula (10) and formula (11), we can obtain the transformation relationship between camera pixel coordinates and projection pixel coordinates:

[0086]

[0087] Because there is an obvious phase error in the black circle part and the phase of the non-black area is relatively accurate, the phase is not directly converted into the projector pixel coordinates by formula (8). Instead, the matrix H is first solved by using the matching point pairs in the non-black area, and then the obtained H is used to calculate the projector pixel coordinates of the circular feature points. Then, the same calibration method as in step (1) is used to complete the calibration of the projector.

[0088] The specific calibration process of the projector is as follows:

[0089] Step 1: Calculate the absolute phase information from the sinusoidal fringe pattern and calculate the corresponding projector pixel coordinates on the calibration image according to formula (8).

[0090] Step 2: Extract the following by threshold segmentation: Figure 3 The non-black area in the calibration plate image shown is considered as a reliable area.

[0091] Step 3: Calculate the homography matrix H using the least squares method from the projector pixel coordinates of the reliable area.

[0092] Step 4: Calculate the projector pixel coordinates corresponding to the center feature point according to formula 12, and use the telecentric camera calibration algorithm in (I) to calibrate the telecentric projector using the optimized projector pixel coordinates.

[0093] 3. Camera-projector joint calibration

[0094] After obtaining the initial values ​​of the parameters of the telecentric camera and the telecentric projector through step (1) and step (2) respectively, joint calibration is performed by minimizing the reprojection errors of the camera and the projector to further improve the accuracy of the system calibration. Taking into account the manufacturing error of the calibration plate and the translation error of the precision translation stage, the present invention uses the bundle adjustment method to perform global optimization on the camera parameters, the projector parameters and the world coordinates of the center feature point as optimization variables:

[0095]

[0096] in, and They represent the camera pixel coordinates and projector pixel coordinates of the jth circle center feature point in the i-th pose, respectively. and Respectively represent P ij The pixel coordinates projected onto the camera image plane and the projector image plane. This problem is solved using the LM algorithm to obtain the optimal camera parameters, projector parameters, and world coordinates of the center feature point.

[0097] In summary, in the calibration of telecentric cameras, the present invention uses a precision translation stage and a plane calibration plate to construct a virtual 3D calibration target, which can solve the global optimal solution of telecentric camera parameters; in the calibration of telecentric projectors, the present invention optimizes the projector pixel coordinates corresponding to the feature points through the global homography relationship between the camera pixel coordinate system and the projector pixel coordinate system, thereby improving the calibration accuracy of the projector; the present invention realizes the joint optimization of the camera and the projector by constructing a reprojection error, thereby further improving the calibration accuracy of the system.

Claims

1. A telecentric three-dimensional measurement system calibration method, characterized in that: The following steps are involved: Constructing a virtual 3D calibration target consisting of a plurality of mutually parallel calibration planes, wherein a plurality of calibration feature points are arranged on the calibration planes; The virtual 3D calibration target is used to calibrate the telecentric three-dimensional measurement system.

2. The calibration method according to claim 1, characterized in that: The method of calibrating the telecentric three-dimensional measurement system using the virtual 3D calibration target includes: Detecting and obtaining the camera pixel coordinates of the calibrated feature points; According to the conversion relationship between the world coordinate system and the camera coordinate system, a camera pixel coordinate model of any calibration feature point in the virtual 3D calibration target projected onto the camera plane is established; Converting the camera pixel coordinate model into a least squares problem; Solving the camera pixel coordinate model using the camera pixel coordinates and world coordinates of the calibrated feature points to obtain initial values ​​of camera intrinsic parameters and initial values ​​of camera extrinsic parameters; Set the initial values ​​of the camera distortion parameters; According to the camera pixel coordinates and world coordinates of the calibration feature points under different postures, the least squares optimization algorithm is used to solve the camera internal parameters, the camera external parameters and the camera distortion parameters that minimize the sum of the square errors of the camera pixel coordinates of all the calibration feature points as the first camera parameters, and the camera pixel coordinate square error is the square of the difference between the camera pixel coordinate of the current calibration feature point and its camera pixel coordinate model.

3. The calibration method according to claim 2, characterized in that: The step of solving the camera pixel coordinate model by using the camera pixel coordinates and the world coordinates of the calibrated feature points to obtain the initial values ​​of the camera internal parameters and the initial values ​​of the camera external parameters includes: Substituting the camera pixel coordinates and world coordinates of the calibrated feature points into the camera pixel coordinate model; Solving the projection matrix of the camera pixel coordinate model by using a singular value decomposition method, wherein the projection matrix of the camera pixel coordinate model is the product of its intrinsic parameter matrix and its extrinsic parameter matrix; Based on the orthogonality and unit of the rotation matrix, the projection matrix of the camera pixel coordinate model is decomposed to obtain the initial values ​​of the camera internal parameters, and then the initial values ​​of the camera external parameters are calculated.

4. The calibration method according to claim 2, characterized in that: The camera pixel coordinates of the calibrated feature points are detected by a circle center extraction algorithm.

5. The calibration method according to claim 2, characterized in that: Also includes: According to the homography transformation relationship between the world coordinate system and the camera coordinate system and the projector coordinate system, a transformation relationship model between the camera coordinate system and the projector coordinate system is established; Obtaining projector pixel coordinates of a plurality of reference feature points within a set area on the calibration plane, wherein the set area does not include the calibration feature points; Solving the conversion relationship model using the camera pixel coordinates and the projector pixel coordinates of the reference feature points; Calculating the projector pixel coordinates of the calibrated feature points based on the conversion relationship model; According to the conversion relationship between the world coordinate system and the projector coordinate system, a projector pixel coordinate model of any calibration feature point in the virtual 3D calibration target projected onto the projector plane is established; The projector pixel coordinate model is solved using the projector pixel coordinates and world coordinates of the calibrated feature points to obtain a first projector parameter that minimizes the sum of the square errors of the projector pixel coordinates of all the calibrated feature points, wherein the projector pixel coordinate square error is the square of the difference between the projector pixel coordinate of the current calibrated feature point and its projector pixel coordinate model.

6. The calibration method according to claim 5, characterized in that: The step of obtaining the projector pixel coordinates of a plurality of reference feature points within a set area on the calibration plane comprises: The absolute phase information of the projector is obtained by using the four-step phase shift method and multi-frequency unfolding technology; Extracting a set area on the calibration plane and determining reference feature points by a threshold segmentation method; The projector pixel coordinates of the reference feature point are calculated based on the camera pixel coordinates of the reference feature point and the absolute phase information.

7. The calibration method according to claim 5, characterized in that: The method of solving the projector pixel coordinate model by using the projector pixel coordinates and world coordinates of the calibrated feature points to obtain a first projector parameter that minimizes the sum of square errors of the projector pixel coordinates of all the calibrated feature points comprises: Converting the projector pixel coordinate model into a least squares problem; Solving the projector pixel coordinate model using the projector pixel coordinates and world coordinates of the calibrated feature points to obtain initial values ​​of projector internal parameters and initial values ​​of projector external parameters; Set the initial values ​​of the projector distortion parameters; According to the projector pixel coordinates and world coordinates of the calibrated feature points in different postures, a least squares optimization algorithm is used to solve the projector internal parameters, the projector external parameters and the projector distortion parameters that minimize the sum of the square errors of the projector pixel coordinates of all the calibrated feature points as the first projector parameters.

8. The calibration method according to claim 7, characterized in that: The method of calculating the initial values ​​of the internal parameters and the external parameters of the projector by using the projector pixel coordinates and the world coordinates of the calibrated feature points includes: Substituting the projector pixel coordinates and world coordinates of the calibrated feature points into the projector pixel coordinate model; The projection matrix of the projector pixel coordinate model is solved by using a singular value decomposition method, wherein the projection matrix of the projector pixel coordinate model is the product of its internal parameter matrix and its external parameter matrix; Based on the orthogonality and unit of the rotation matrix, the projection matrix of the projector pixel coordinate model is decomposed to obtain the initial values ​​of the internal parameters of the projector, and then the initial values ​​of the external parameters of the projector are calculated.

9. The calibration method according to claim 7, characterized in that: Also includes: Taking the first camera parameters and the first projector parameters as initial values, and according to the camera pixel coordinates, projector pixel coordinates and world coordinates of the calibration feature points in different postures, using a least squares optimization algorithm to solve the camera internal parameters, the camera external parameters, the camera distortion parameters, the projector internal parameters, the projector external parameters and the projector distortion parameters that minimize the sum of the global pixel coordinate square errors of all the calibration feature points, the global pixel coordinate square error being the sum of the camera pixel coordinate square error of the current calibration feature point and the projector pixel coordinate square error thereof.

Images