Method for len distortion calibration and calibrated system therefrom

Abstract

This invention provides a method for lens distortion calibration of a stereo vision system comprising a left camera and a right camera. In one embodiment, said method comprises the steps of: a) Acquiring images of corner points using said stereo vision system, wherein said corner points are formed by intersection of m horizontal lines and n vertical lines arranged on a plane; and actual distance between adjacent comer points is precisely known; b) Extracting image coordinates of said corner points from said set of images and calculating a set of intrinsic, extrinsic and distortion parameters of said left camera and said right camera through homography matrix; c) Obtaining a set of corrected image coordinates (I) and (II) using said set of intrinsic, extrinsic and distortion parameters of step (b) and a lens distortion model.

Description

METHOD FOR LEN DISTORTION CALIBRATION AND CALIBRATED SYSTEM THEREFROMFIELD OF THE INVENTION

[0001] The present invention relates to computer vision measurement technologies, particularly depth-dependent lens distortion model and its calibration method.BACKGROUND OF THE INVENTION

[0002] Stereo vision measurement technology obtains the 3D information of an object by mapping image information and imaging model parameters. Compared with traditional measurement methods, it has the advantages of wide measurement range, great versatility, high precision and being non-contact, and it is widely used in various fields such as 3D measurement. The calibration of stereo vision measurement system means calibrating the intrinsic and extrinsic camera parameters and lens distortion parameters. The lens distortion parameters are closely related to the depth information of the object, but the existing depth-dependent lens distortion models are very complicated, resulting in complicated calibration process and low practicability. In this invention, the simplified depth-dependent lens distortion model overcomes the problem and the calibration method brings high precision and high efficiency. This invention is of great significance for improving the accuracy and practicability of the vision measurement system.

[0003] In 2021, Li Xiao et al. from China University of Petroleum (East China) disclosed the “calibration method of depth of field distortion model based on target loose attitude Constraint” (CN 113781581 A). The depth-of-field distortion model in Li Xiao et al. was very complicated and had more unknown parameters, which results in a complicated calibration process. Moreover, the depth information of cooperative targets is required. Thus, this method has a low practicality.

[0004] In addition, in 2017 Liu Wei et al. from Dalian University of Technology disclosed “A high-precision stereo vision measurement method based on depth of field compensation” (CN 107144241A). In Liu Wei et al., a radial distortion compensation model in the depth-of-field direction was developed. To use the model to obtain distortion parameters at different depths, the optical axis must be perpendicular to thetarget plane. As a result, it is difficult to implement this measurement method and its practicability is low.SUMMARY OF THE INVENTION

[0005] This invention provides a method for lens distortion calibration of a stereo vision system comprising a left camera and a right camera. In one embodiment, said method comprises the steps of: a) Acquiring images of corner points using said stereo vision system, wherein said corner points are formed by intersection of m horizontal lines and n vertical lines arranged on a plane; and actual distance between adjacent comer points is precisely known; b) Extracting image coordinates of said corner points from said set of images and calculating a set of intrinsic, extrinsic and distortion parameters of said left camera and said right camera through homography matrix; c) Obtaining a set of corrected image coordinates (u^, pA ) and (itf, vtf) using said set of intrinsic, extrinsic and distortion parameters of step (b) and a lens distortion model defined by Eqd) Reconstructing a set of 3D information from said set of initial intrinsic, extrinsic anddistortion parameters of step (b) and said set of corrected image coordinates of step (c) using Eqn (8) anwheree) Minimizing RMSE between said actual distances and corresponding reconstructed distances from said set of 3D information to optimize said lens distortion model.

[0006] This invention also provides a stereo vision system comprising a computing device for implementing the method of invention; a method for computer vision measurement using the stereo vision system of this invention; a non-transitory computer-readable medium having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations for lens distortion calibration of a stereo vision system; comprising the method of this invention; a computing device comprising: a) a processor; b) memory; and c) program instructions, stored in the memory, that upon execution by the processor cause the computing device to perform operations for lens distortion calibration of a stereo vision system; comprising the method of this invention.BRIEF DESCRIPTION OF THE FIGURES

[0007] Figure 1 shows the simplified depth-dependent lens distortion model calibration method in this invention employs the most commonly used checkerboard.DETAILED DESCRIPTION OF THE INVENTION

[0008] Existing depth-dependent lens distortion models are complicated with many unknown parameters, which leads to complicated calibration process. The simplified depth-dependent lens distortion model and its calibration method in this invention successfully overcome this problem. The simplified model in this invention is simple in construction, few in parameters and easy in calibration. The calibration method in this invention works in four procedures: (1) stereo vision system is employed to photograph checkerboard of different poses; (2) Zhang’s calibration method is employed to calibrate the intrinsic and extrinsic parameters of the cameras; (3) preliminarily optimize the parameters of the simplified depth-dependent lens distortion model based on the linear constraints on the checkerboard plane; (4) taking the preliminarily optimized parameters of the above procedures as the initial values, and taking the minimum value of mean RMSE between the actual distances and the reconstructed distances of the corner points on the checkerboard plane as the optimization objective, the overall optimization of the intrinsic and extrinsic parameters of the cameras and the distortion parameters is conducted. And the optimized values of all parameters are obtained. The calibration method ensures the calibration accuracy of the simplified depth-dependent lens distortion model and simplifies the calibration process, thus enhancing the practicality.

[0009] This invention provides a simplified depth-dependent lens distortion model and its calibration method which belongs to the field of computer vision measurement technology. Lens distortion is a main cause of image distortion. Most lens distortion models neglect the influence of depth, which leads to a low measurement accuracy in the depth-of-field direction. The lens distortion models, which consider the influence of depth, are very complicated. This makes the calibration process complicated, which in turn leads to a low practicality. In the present invention, there is a simplified depthdependent lens distortion model and a simple method to calibrate its parameters. The calibration method follows four procedures. First, the stereo vision system is employed to photograph checkerboard of different poses; second, Zhang's calibration method is employed to calibrate the intrinsic and extrinsic parameters of the cameras; third,preliminarily optimize the parameters of the simplified depth-dependent lens distortion model based on the linear constraints on the checkerboard plane; fourth, obtain more accurate intrinsic and extrinsic parameters and simplified depth-dependent lens distortion parameters by (1) using the mean RMSE (root-mean-square error) between the actual distances and the reconstructed distances of the corner points on the checkerboard plane as objective function and (2) employing multi-objective optimization.

[0010] There is a simplified depth-dependent lens distortion model and its calibration method in this invention.

[0011] 1. A simplified depth-dependent lens distortion model

[0012] The simplified depth-dependent lens distortion model in this invention is developed on the basis of simplifying the distortion model proposed by Fraser and Shortis in “Variation of distortion within the photographic field” in 1992. The equation of the simplified depth-dependent lens distortion model is as follows:where A2, A3, A4, BltB2, B3, and B4are the radial lens distortion parameters; CltC2, C3, C4, DltD2, D3, and Z)4are the decentering lens distortion parameters; (u, v) is the uncorrected image coordinate; (u, v) is the corrected image coordinate; 8Uand 8Vare the distortion functions of an image point in the u and v direction, respectively; (u0, v0) is the coordinate of the distortion center; r = ^ / (u — u0)2+ (v — v0)2is the distortion radius of the image point; snis the depth information of the image point.

[0013] 2. Calibration of the simplified depth-dependent lens distortion model

[0014] (1) Image acquisition

[0015] Stereo vision systems usually consist of two cameras and two lenses. Thesimplified depth-dependent lens distortion model calibration method in this invention employs the most commonly used checkerboard, as shown in Fig. 1. There are m horizontal lines and n vertical lines arranged on the checkerboard. The two lines intersect to form corner points. The distance between adjacent corner points is precisely known. In the process of image acquisition, the lenses and cameras are first fixed on the optical platform, and the checkerboard is randomly placed in multiple poses within a certain depth range. For each pose, the two cameras captured the images of the checkerboard at the same time. A total of 2 X w images of the checkerboard were collected.

[0016] (2) Calibration of the intrinsic and extrinsic camera parameters

[0017] Zhang’s calibration method, which is proposed in “A flexible new technique for camera calibration” in 2000, is employed to calibrate the intrinsic and extrinsic parameters of two cameras. The image coordinates of corner points in the checkerboard images are extracted, and then the initial values of intrinsic and extrinsic camera parameters are calculated through homography matrix.

[0018] (3) Preliminary optimization of the simplified depth-dependent lens distortion model parameters

[0019] Defineand (u^, v-j) as corrected image coordinates in the i-th row and j -th column in the checkerboard images under the left and right image coordinate systems, respectively. They are obtained by Equation (2):where A[, A2, AR, AR, B{, B2, BRand Bf are the radial lens distortion parameters; Cf , Cf , CR, CR, DR, D2, D? and Of are the decentering lens distortion parameters; (u[p V-j) and (uR-, vR) are corner point image coordinates in the i-th row and j-th column in the checkerboard images under the left and right image coordinate systems, respectively. (UQ, VQ ) and (UQ , VR) are principal point coordinates under the left and right image coordinate systems, respectively; andr-y =316distortion radiuses of the image points underthe left and right image coordinate systems, respectively;are the depth information of the corner points in the i-th row and j -th column in the checkerboard images under the left and right image coordinate systems, respectively.

[0020] As shown in Fig. 1 , when the corner points on the same straight lines in theleft and right checkerboard images are known, the line equations determined by the corner points under the left and right image coordinate systems arewhere aRand pRare line parameters of the i-th line in the left checkerboard image; aRand fiRare line parameters of the i -th line in the right checkerboard image; (ufj, v-j) and (uR, vR) are also defined as the uncorrected image coordinates of the j- th corner point on the i-th line under the left and right image coordinate systems, respectively. Then, Equation (4) is obtained by substituting (ufy, vfy) and (uR, vR) into (uL, vL) and (uR, vR) in Equation (3), respectively,where KR= [AR, Bf ] and KL2= [AL2, B2] are the first and second-order parameters of radial distortion under the left image coordinate system, respectively; PR= [CR, DR] and P2L= [C2\ Df ] are the first and second-order parameters of decentering distortion under the left image coordinate system, respectively; KR= [ftf, Bf] and KR= [AR, BR] are the first and second-order parameters of radial distortion under the right image coordinate system, respectively; PR= [Cf, Df] and PR= [Cf, Of] are the first and second-order parameters of decentering distortion under the right image coordinate system, respectively.

[0021] If there are i lines in an image (left or right image) and j observation points are extracted from each line, we can have i ■ j equations. In these equations, there are t ■ 2 + 8 variables (t ■ 2 line parameters, 8 distortion parameters). If i ■ j > i ■ 2 + 8, the optimal solution of distortion parameters can be obtained.

[0022] Aff = [«f Tf ] is the 4-by-3 projection matrix, expressing the transformation relationship of the left camera coordinate system relative to the right camera coordinate system, where RRand TRare the rotation and translation matrices of left camera coordinate system relative to the right camera coordinate system, respectively; MR= [RRLTLR] is the 4-by-3 projection matrix, expressing the transformation relationship ofthe right camera coordinate system relative to the left camera coordinate system, where RRand TRare the rotation and translation matrices of right camera coordinate system relative to the left camera coordinate system, respectively; is the element in the ith row and jth column of the flff ; t-j is the element in the ith row and jth column of the MR: fLand fRare the focal lengths of the left and right cameras, respectively; andare roughly estimated by substituting ( y ( ) the calibrated intrinsic and extrinsic camera parameters, intothe equations.

[0023] (4) Overall optimization of the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters

[0024] First, 3D reconstructs the corrected points in image coordinates by the equations as shown below.

[0025] Equation (7) is obtained by combining Equation (5) and Equation (6) and eliminating the ZLand ZR:

[0026] In Equation (5), (6) and (7), R™ and T™ are the rotation matrix and the translation matrix, expressing the transformation between the left image coordinate system and the world coordinate system; R™ and T™ are rotation matrix and translation matrix, expressing the transformation between the right image coordinatesystem and the world coordinate system; KLand KRare the intrinsic parameters matrices of the left and right cameras; [Xij, Yij, Zij, 1]Tis the 3D reconstructed information of the corner points in the i-th row and j -th column in the checkerboard image under the world coordinate system; ZLand ZRare scaling factors;and mfy are the values of the i-th row and j -th column in the left and right camera projection matrices, respectively. To solve (Xtj, YQ, Z^). Equation (7) is rewritten as:where ATis the transposition of matrix A: (J4TJ4)1is the inverse of matrix ATA.

[0027] Then, the objective function is proposed as shown in Equation (9):whererepresents the 3D reconstruction equation related to the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters; a and b represent the actual length and width of the rectangle on the checkerboard plane, respectively.

[0028] Finally, Levenberg-Marquardt optimization method is employed to optimize the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters. The optimized initial values are the intrinsic and extrinsic camera parameters obtained by Zhang’s calibration method and the simplified depthdependent lens distortion model parameters obtained by linear constraints. According to the above optimization variables, optimized initial values and optimization objective function, the optimized values of the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters are obtained.

[0029] Examples

[0030] The following example is used to illustrate the calibration method of the simplified depth-dependent lens distortion model in this invention:

[0031] 1. Calibration of simplified depth-dependent lens distortion model

[0032] (1) Image acquisition

[0033] The simplified depth-dependent lens distortion model calibration system consists of two cameras, two 12 mm prime lenses and a 12x9 (150 mmxl05 mm) checkerboard. There are 8 horizontal and 11 vertical lines distributed on the checkerboard, and the lines intersect to form a total of 88 corner points. The distances between adjacent corner points are all 15 mm. The camera resolution is 2464 pixels x 2056 pixels. In the process of image acquisition, the lenses and cameras are first fixed on the optical platform. Then, the checkerboard is randomly placed in multiple poses within a certain depth range. The checkerboard and the corner points on it only need to cover as wide a depth as possible, while ensuring that all corner points on the checkerboard plane can be clearly imaged. For each pose, the two cameras simultaneously take the images of the checkerboard. And finally, 26 images of checkerboard in random poses are collected.

[0034] (2) Calibration of the intrinsic and extrinsic camera parameters

[0035] Zhang’s calibration method is employed to calibrate the intrinsic and extrinsic parameters. First, the image coordinates of corner points in the checkerboard images are extracted, and then the initial values of intrinsic and extrinsic camera parameters are calculated through homography matrix. The parameters are shown in Table 1 :Table 1. intrinsic and extrinsic camera parameters

[0036] (3) Preliminary optimization of the simplified depth-dependent lens distortion model parameters

[0037] First, according to Equation (2), the corrected image coordinates of the corner points in the i-th row and j - th column in the checkerboard images are obtained under the left and right image coordinate systems. Second, with the intrinsic and extrinsic parameters in Table 1 and the image coordinates in the checkerboard images corrected by the simplified depth-dependent lens distortion model, Equation (8) is employed to reconstruct the 3D information. Finally, as shown in Equation (4), the linear constraint relationships of corner points in the same straight lines in the left and right images under left and right image coordinate systems are established, thus achieving the simplified depth-dependent preliminary optimization of lens distortion model parameters, as shown in Table 2:Table 2. Preliminarily optimized values of the simplified depth- dependent lens distortion model parameters

[0038] (4) Overall optimization of the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters

[0039] The optimization objective is to minimize the mean RMSE between the actual distances and the reconstructed distances of the corner points on the checkerboard plane, and the objective function is shown in Equation (9).

[0040] Levenberg-Marquardt optimization method is employed to optimize the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters. The intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters in Table 1 and Table 2, which are obtained by preliminary optimization, are used as optimized initial values. According to the above optimization variables, optimized initial values and optimization objective function, the optimized values of the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters are obtained, as shown in Table 3 and Table 4.Table 3. optimized intrinsic and extrinsic camera parametersTable 4. optimized simplified depth-dependent lens distortion model parameters

[0041] To verify the validity of the calibration method, firstly, the positions of the left and right cameras are kept unchanged, and 5 checkerboard images in different poses and different depths are collected continuously. Then, the image coordinates of corner points in the checkerboard images are extracted. The optimized values in Table 3 and Table 4 are substituted into the simplified depth-dependent lens distortion model in this invention. Then, the image coordinates considering lens distortion are obtained. Finally, the 3D information of the corner points on the same straight line in the checkerboard images is obtained by Equation (6). And the mean RMSE between the actual distancesand the different geometric distances of the corner points on a line on the checkerboard plane is used as the error evaluation standard.

[0042] Select different corner points on the same line on the checkerboard plane with actual geometric distances of 15 mm, 75 mm and 150 mm. Calibrate the lens distortion parameters by using Zhang’s calibration method and the calibration method in this invention, respectively. 3D reconstruct the corner points in the checkerboard images by Equation (6). Calculate the 3D reconstructed geometric distance between the two corner points, and the results are as follows:

[0043] When the geometric distance between two corner points is 15 mm and the average depth change is 8.75 mm, the difference between the geometric distance optimized by the calibration method in this invention and the actual distance between the two corner points is 0.0057 mm; the difference between the geometric distance optimized by Zhang’s calibration method and the actual distance between the two corner points is 0.0170 mm.

[0044] When the geometric distance between two corner points is 75 mm and the average depth change is 43.77 mm, the difference between the geometric distance optimized by the calibration method in this invention and the actual distance between the two corner points is 0.0255 mm; the difference between the geometric distance optimized by Zhang’s calibration method and the actual distance between the two corner points is 0.0315 mm.

[0045] When the geometric distance between two corner points is 150 mm and the average depth change is 87.51 mm, the difference between the geometric distance optimized by the calibration method in this invention and the actual distance between the two corner points is 0.0315 mm; the difference between the geometric distance optimized by Zhang’s calibration method and the actual distance between the two corner points is 0.0625 mm.

[0046] The above results show that the simplified depth-dependent lens distortion model can ensure the accuracy of camera calibration, simplify the calibration process, and improve the practicability.

[0047] This invention provides a simplified depth-dependent radial lens distortion model. In one embodiment, said radial lens distortion model is shown below, a simplified depth-dependent radial lens distortion model parameters are in a first-order linear relationship with depth information:In Equation (10), (u, v) is the coordinate of uncorrected image point; 8urand 8vrare the radial distortion functions in the direction of u and v, respectively; (u0, v0) is the coordinate of the distortion center; r =is the distortion radius of the image point; snis the depth information of the image point; A4, A2, A3, A4, B4, B2, B3, and B4are the radial lens distortion parameters of the simplified depth-dependent lens distortion model.

[0048] This invention also provides a simplified depth-dependent decentering lens distortion model. In one embodiment, as is shown below, a simplified depth-dependent decentering lens distortion model parameters are in a nonlinear relationship with depth information:In Equation (11), (u, v) is the coordinate of uncorrected image point; 8ud> 8vdare the decentering distortion functions in the direction of u and v, respectively; (u0, v0) is the coordinate of the distortion center; is thedistortion radius of the image point; snis the depth information of the image point; C1;C2, C3, C4, DltD2, D3, and Z)4are the decentering lens distortion parameters of the simplified depth-dependent lens distortion model.

[0049] This invention also provides a calibration method of a simplified depthdependent lens distortion model. In one embodiment, said calibration method comprises the step of: Step 1: Image acquisition. Stereo vision systems usually consist of two cameras and two lenses. The simplified depth-dependent lens distortion modelcalibration method in this invention employs the most commonly used checkerboard, as shown in Fig. 1. There are m horizontal lines and n vertical lines arranged on the checkerboard. The two lines intersect to form corner points. The distance between adjacent corner points is precisely known. In the process of image acquisition, the lenses and cameras are first fixed on the optical platform, and the checkerboard is randomly placed in multiple poses within a certain depth range. For each pose, the two cameras captured the images of the checkerboard at the same time. A total of 2 X w images of the checkerboard were collected. Step 2: Calibration of the intrinsic and extrinsic camera parameters. Zhang’s calibration method, which is proposed in “A flexible new technique for camera calibration” in 2000, is employed to calibrate the intrinsic and extrinsic parameters of two cameras. The image coordinates of corner points in the checkerboard images are extracted, and then the initial values of intrinsic and extrinsic camera parameters are calculated through homography matrix. Step 3: Preliminary optimization of the simplified depth-dependent lens distortion model parameters. Define (Ujj, v-j) and (u^, v-j) as corrected image coordinates in the i th row and j th column in the checkerboard images under the left and right image coordinate systems, respectively. They are obtained by Equation (12):where A[, A2, A± , A2, B[, B2, B^ and B2are the radial lens distortion parameters;Cf , C2, C^, C2, D± , D2, Dh and D2are the decentering lens distortion parameters;(U[j, v-j) and (u^, v-j) are corner point image coordinates in the i th row and j th column in the checkerboard images under the left and right image coordinate systems, respectively. (UQ, VQ ) and (UQ , VQ ) are principal point coordinates under the left and right image coordinate systems, respectively; andr-y =316distortion radiuses of the image points underthe left and right image coordinate systems, respectively;are the depth information of the corner points in the ith row and j th column in the checkerboard images under the left and right image coordinate systems, respectively.

[0050] As shown in Fig. 1 , when the corner points on the same straight lines in theleft and right checkerboard images are known, the line equations determined by the corner points under the left and right image coordinate systems arewhereare line parameters of the ith line in the left checkerboard image; aRand fiRare line parameters of the ith line in the right checkerboard image; (u-j, v-j) and (uR, vR) are also defined as the uncorrected image coordinates of the j th corner point on the ith line under the left and right image coordinate systems, respectively. Then, Equation (14) is obtained by substituting (uRj, vRj) and (uR, vR) into (uL, vL) and (uR, vR) in Equation (13), respectively.where KR= [AR, BR] andB2] are the first and second-order parameters of radial distortion under the left image coordinate system, respectively; PR= [CR, DR] and P2L= [C2\ Df ] are the first and second-order parameters of decentering distortion under the left image coordinate system, respectively;and KR=[AR, BR] are the first and second-order parameters of radial distortion under the right image coordinate system, respectively; PR= [Cf, Df] and PR= [Cf, Df] are the first and second-order parameters of decentering distortion under the right image coordinate system, respectively.

[0051] If there are i lines in an image (left or right image) and j observation points are extracted from each line, we can have i ■ j equations. In these equations, there are i ■ 2 + 8 variables (t ■ 2 line parameters, 8 distortion parameters). If i ■ j > i ■ 2 + 8, the optimal solution of distortion parameters can be obtained.

[0052] Aff = [Pf Tf ] is the 4-by-3 projection matrix, expressing the transformation relationship of the left camera coordinate system relative to the right camera coordinate system, where RRand TRare the rotation and translation matrices of left camera coordinate system relative to the right camera coordinate system, respectively; MR=[«« TLR] is the 4-by-3 projection matrix, expressing the transformation relationship of the right camera coordinate system relative to the left camera coordinate system, where RRand TRare the rotation and translation matrices of right camera coordinate system relative to the left camera coordinate system, respectively; t-j is the element in the ith row and jth column of the flff ; t-j is the element in the ith row and jth column of the MR, ' fLand fRare the focal lengths of the left and right cameras, respectively; and = roughly estimated by substitutingthe calibrated intrinsic and extrinsic camera parameters, (u^, v-j) and (ufj, v-j) into the equations.

[0053] Step 4: Overall optimization of the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters. First, 3D reconstructs the corrected points in image coordinates by the equations as shown below.Equation (17) is obtained by combining Equation (15) and Equation (16) and eliminating the ZLand ZR: _

[0054] In Equation (15), (16) and (17), R^ and are the rotation matrix and the translation matrix, expressing the transformation between the left image coordinate system and the world coordinate system; R™ and T™ are rotation matrix and translation matrix, expressing the transformation between the right image coordinatesystem and the world coordinate system; KLand KRare the intrinsic parameters matrices of the left and right cameras; [Xq, Yq, Zq, 1]Tis the 3D reconstructed information of the corner points in the ith row and j th column in the checkerboard image under the world coordinate system; ZLand ZRare scaling factors;and mfy are the values of the ith row and j th column in the left and right camera projection matrices, respectively. To solve (Xtj, YQ, Z^). Equation (17) is rewritten as:where ATis the transposition of matrix A. (J4TJ4)1is the inverse of matrix A? A.

[0055] Then, the objective function is proposed as shown in Equation (19):wherePMHi (KL, KR, R^, T^, RR , T%, K[, K2L, P[, Pi Ki Ki PiP$) =Yu koi represents the 3D reconstruction equation related to the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters; a and b represent the actual length and width of the rectangle on the checkerboard plane, respectively.

[0056] Finally, Levenberg-Marquardt optimization method is employed to optimize the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters. The optimized initial values are the intrinsic and extrinsic camera parameters obtained by Zhang’s calibration method and the simplified depth-dependent lens distortion model parameters obtained by linear constraints. According to the above optimization variables, optimized initial values and optimization objective function, the optimized values of the intrinsic and extrinsic camera parameters and the simplified depth-dependent lens distortion model parameters are obtained.

[0057] This invention provides a method for lens distortion calibration of a stereo vision system comprising a left camera and a right camera. In one embodiment, said method comprises the steps of: a) Acquiring images of corner points using said stereo vision system, wherein said corner points are formed by intersection of m horizontal lines and n vertical lines arranged on a plane; and actual distance between adjacent comer points is precisely known; b) Extracting image coordinates of said corner points from said set of images and calculating a set of intrinsic, extrinsic and distortion parameters of said left camera and said right camera through homography matrix; c) Obtaining a set of corrected image coordinates (uE, pE) and (uE, pE) using said set of intrinsic, extrinsic and distortion parameters of step (b) and a lens distortion model defined by Eqd) Reconstructing a set of 3D information from said set of initial intrinsic, extrinsic and distortion parameters of step (b) and said set of corrected image coordinates of step (c) using Eqn (8) awheree) Minimizing RMSE between said actual distances and corresponding reconstructed distances from said set of 3D information to optimize said lens distortion model.

[0058] In one embodiment, said step (b) is achieved using Zhang’s method.

[0059] In one embodiment, said step (e) is achieved using Levenberg-Marquardt optimization method.

[0060] In one embodiment, said images comprise images of corner points at different poses and / or depths.

[0061] In one embodiment, said images comprise images of corner points randomly placed in multiple poses within a certain depth range.

[0062] In one embodiment, said corner points are located on a checkerboard.

[0063] In one embodiment, said left and right cameras are fixed on an optical platform.

[0064] In one embodiment, said distortion parameters comprise decentering distortion parameters and radial distortion parameters.

[0065] This invention also provides a stereo vision system comprising a computing device for implementing the method of invention and a method for computer visionmeasurement using the stereo vision system of this invention.

[0066] In one embodiment, said computer vision measurement comprises depth dependent measurements.

[0067] This invention further provides a non-transitory computer-readable medium having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations for lens distortion calibration of a stereo vision system; comprising the method of this invention.

[0068] This invention also provides a computing device comprising: a) a processor; b) memory; and c) program instructions, stored in the memory, that upon execution by the processor cause the computing device to perform operations for lens distortion calibration of a stereo vision system; comprising the method of this invention.

[0069] In one embodiment, said computing device is connected to a stereo vision system.

[0070] In one embodiment, said computing device comprises a data input for transferring images of corner points.

Claims

What is claimed is:

1. A method for lens distortion calibration of a stereo vision system comprising a left camera and a right camera, said method comprises the steps of: a. Acquiring images of corner points using said stereo vision system, wherein said corner points are formed by intersection of m horizontal lines and n vertical lines arranged on a plane; and actual distance between adjacent comer points is precisely known; b. Extracting image coordinates of said corner points from said set of images and calculating a set of intrinsic, extrinsic and distortion parameters of said left camera and said right camera through homography matrix; c. Obtaining a set of corrected image coordinates (Ujj, v-j) and(Uij, v-j) using said set of intrinsic, extrinsic and distortion parameters of step (b) and a lens distortion model defined by Eqn (2):d. Reconstructing a set of 3D information from said set of initial intrinsic, extrinsic and distortion parameters of step (b) and said set of corrected image coordinates of step (c) using Eqn (8) and Eqn (9):e. Minimizing RMSE between said actual distances and corresponding reconstructed distances from said set of 3D information to optimize said lens distortion model.

2. The method of claim 1, wherein said step (b) is achieved using Zhang’s method.

3. The method of claim 1, wherein said step (e) is achieved using Levenberg- Marquardt optimization method.

4. The method of claim 1, wherein said images comprise images of corner points at different poses and / or depths.

5. The method of claim 4, wherein said images comprise images of corner points randomly placed in multiple poses within a certain depth range.

6. The method of claim 1, wherein said corner points are located on a checkerboard.

7. The method of claim 1, wherein said left and right cameras are fixed on an optical platform.

8. The method of claim 1, wherein said distortion parameters comprise decentering distortion parameters and radial distortion parameters.

9. A stereo vision system comprising a computing device for implementing the method of any one of claims 1 to 8.

10. A method for computer vision measurement using the stereo vision system of claim 9.

11. The method of claim 10, wherein said computer vision measurement comprises depth dependent measurements.

12. A non-transitory computer-readable medium having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations for lens distortion calibration of a stereo vision system; comprising the method of any one of claims 1 to 8.

13. A computing device comprising: a. a processor; b. memory; and c. program instructions, stored in the memory, that upon execution by the processor cause the computing device to perform operations for lens distortion calibration of a stereo vision system; comprising the method of any one of claims 1 to 8.

14. The computing device of claim 13, wherein said computing device is connected to a stereo vision system.

15. The computing device of claim 13, wherein said computing device comprises a data input for transferring images of corner points.

Images