Variable-object-distance large-view-field camera calibration method considering lens imaging model

Abstract

A variable-object-distance large-view-field camera calibration method considering a lens imaging model comprises the steps that after an imaging system comprising a prime lens is adopted to shoot a close-shot image and a long-shot image respectively, based on a fixed principal point coordinate and a distortion coefficient, an internal reference calibration result at the close-shot position is used as an initial value; calibration of the variable-object-distance large-field-of-view camera is realized by means of cyclically calculating a mode of solving an external parameter according to an internal parameter and a homography matrix, solving an image distance according to a lens imaging model and solving an equivalent focal length (internal parameter) according to the image distance. According to the invention, the camera calibration process in a variable object distance large view field scene is significantly simplified, and the requirement for the checkerboard view field ratio is greatly reduced.

Description

Technical Field

[0001] The present invention relates to a technology in the field of image measurement, in particular to a calibration method for a camera with variable object distance and large field of view taking into account a lens imaging model. Background Art

[0002] Camera calibration technology based on calibration objects with known coordinates is based on linear or nonlinear imaging theory, and the correspondence between spatial points with known coordinates on the calibration object and their image points is established. The internal and external parameters in the camera imaging model are obtained by using linear or nonlinear optimization algorithms. However, in the scene of variable object distance and large field of view, the existing camera calibration technology has the following problems: 1) The calibration method of constructing a "large" calibration object by using a "small" calibration object is complicated and inefficient, and requires high precision in the motion control of the mechanical device that changes the position of the "small" calibration object. 2) The self-calibration method without calibration objects with known coordinates requires imaging from multiple perspectives, which requires a large number of cameras and is not suitable for calibration of single cameras, dual cameras, etc. In addition, this method requires a large number of feature points on the calibration object for calibration, which needs to reach thousands, greatly increasing the complexity of the calibration process. 3) None of them can efficiently solve the camera calibration problem under variable object distance. Therefore, for the problem of camera calibration with variable object distance and large field of view, it is urgent to propose a camera calibration method with simple operation, low cost, high efficiency and high calibration accuracy. Summary of the invention

[0003] In view of the problems in the prior art that the camera calibration process in variable object distance scenes is complicated and inefficient, and the calibration accuracy is low due to the high production cost of calibration objects and the limited size of calibration objects in large field of view scenes, a variable object distance and large field of view camera calibration method considering the lens imaging model is proposed.

[0004] The present invention is achieved through the following technical solutions:

[0005] The present invention relates to a calibration method for a variable object distance and large field of view camera taking into account a lens imaging model, comprising:

[0006] S1: Build an imaging system (including a digital camera, a fixed-focus optical lens, etc.) in a variable object distance and large field of view measurement scenario, and determine the pixel sizes dx and dy of the digital camera imaging sensor and the focal length f of the fixed-focus optical lens. The imaging system may be a single camera, a dual camera or a multi-camera;

[0007] The imaging sensor pixel sizes dx and dy may be the same;

[0008] The fixed-focus optical lens has a focus ring, which can be focused manually or automatically. Through the focusing operation, objects at different object distances can be clearly imaged;

[0009] S2: placing a two-dimensional checkerboard calibration plate with known coordinates at the near-view position, adjusting the focus ring of the fixed-focus optical lens, so that the imaging system in step S1 can clearly image the two-dimensional checkerboard calibration plate at the near-view position, and collecting images of the checkerboard calibration plate in different positions;

[0010] The area of ​​the two-dimensional checkerboard calibration plate in the field of view accounts for greater than or equal to 1 / 9;

[0011] S3: Based on the two-dimensional chessboard calibration plate image at the near view acquired in step S2, the Zhang Zhengyou calibration method is used to complete the calibration of the imaging system internal parameters, distortion coefficients and external parameters;

[0012] The internal parameter matrix Including equivalent focal length f x and f y , principal point coordinates (c x ,c y );

[0013] The distortion coefficients are radial distortion coefficients k1 and k2;

[0014] The external parameters include the translation vector Where: T x 、T y 、T z are the coordinate components in three directions, which can be used to solve the object distance And the rotation matrix R = [r1r2 r3], where: r1, r2, r3 represent the first, second, and third column vectors of R.

[0015] S4: placing a two-dimensional checkerboard calibration plate with known coordinates at different object distances / distant positions, adjusting the focus ring of the fixed-focus optical lens, so that the imaging system in step S1 can clearly image the two-dimensional checkerboard calibration plate at different object distances / distant positions, and collecting an image of the checkerboard calibration plate in a single posture;

[0016] The area of ​​the two-dimensional checkerboard calibration plate in the field of view accounts for greater than or equal to 1 / 64;

[0017] S5: constructing a homography matrix H according to the world coordinates of the corner points of the two-dimensional chessboard calibration plate in step S4 and their pixel coordinates in the image;

[0018] The homography matrix H is a transformation matrix H=[h1 h2 h3] between the world coordinates of the corner points of the two-dimensional chessboard calibration plate and their image pixel coordinates, wherein h1, h2, h3 are the first, second, and third column vectors of the homography matrix.

[0019] S6: Estimate the translation vector T′ and rotation matrix R′ at different object distances / distant scenes according to the homography matrix H and the intrinsic parameter matrix K;

[0020] The translation vector in:

[0021] The rotation matrix Among them: r1′, r2′, r3′ are the first, second and third column vectors of the rotation matrix R′ respectively.

[0022] S7: Calculate the object distance u′ according to the translation vector T′ estimated in S6;

[0023] The object distance

[0024] S8: Based on the object distance u′ estimated in S7, using the lens imaging model Get the estimated value v′ of the image distance at different object distances / distant scenes;

[0025] S9: Update the equivalent focal length based on the estimated image distance v′ and the camera imaging sensor pixel sizes dx and dy Construct a new internal parameter matrix

[0026] S10: Determine whether |uu′| is less than the threshold ε. If it is less than the threshold, output the current intrinsic parameter matrix, translation vector and rotation matrix; if it is greater than the threshold, set K = K′, T = T′, iterate S6 to S9, update the translation vector, rotation matrix and intrinsic parameter matrix until |uu′| is less than the threshold. Technical Effects

[0027] The present invention relates the equivalent focal length calibration results of the imaging system at the near view (small field of view) and the equivalent focal length calibration results of the imaging system at different object distances / long view (large field of view) through a lens imaging model that characterizes the relationship between image distance, object distance and focal length. The calibration results of the internal and external parameters at the near view are used as initial values. The homography matrix and lens imaging model constructed by combining a single chessboard image shot at different object distances / long view are used to iteratively solve the object distance and the internal parameter matrix to achieve the calibration of the internal and external parameters of the camera at different object distances / long view (large field of view). Compared with the prior art, the present invention has a simple operation process, low cost, high efficiency and high precision. The requirements for the size of the two-dimensional calibration object in the variable object distance and large field of view calibration are reduced (the area ratio of the calibration object in the field of view can be reduced to 1 / 64). At the same time, only one two-dimensional calibration object image needs to be shot at each measured object distance to obtain the internal and external parameters of the camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of variable object distance and large field of view calibration in one embodiment of the present invention

[0029] Figure 2 Flowchart for iterative solution of imaging parameters;

[0030] Figure 3 This is an example effect diagram. DETAILED DESCRIPTION

[0031] like Figure 1 As shown, this embodiment involves a variable object distance large field of view camera calibration method considering the lens imaging model. The imaging system (including a single digital camera and a fixed focus lens) images the chessboard at different object distances respectively. The object distance is divided into three areas, respectively recorded as areas A, B, and C. When shooting at different object distances, it is necessary to adjust the focus ring of the lens to make the image clear. In area A, the imaging system is calibrated with a chessboard calibration plate ① with a field of view area accounting for approximately 1 / 9 and a chessboard calibration plate ④ with a field of view area accounting for approximately 1 / 64; in area B, the imaging system is calibrated with a chessboard calibration plate ② with a field of view area accounting for approximately 1 / 9 and a chessboard calibration plate ⑤ with a field of view area accounting for approximately 1 / 64; in area C, the imaging system is calibrated with a chessboard calibration plate ③ with a field of view area accounting for approximately 1 / 9 and a chessboard calibration plate ⑥ with a field of view area accounting for approximately 1 / 64. Among them: chessboard calibration plates ①, ②, ③ are photographed with images of different postures at different object distances, and Zhang Zhengyou calibration method is used to obtain reference calibration results at different object distances; chessboard calibration plates ④, ⑤, ⑥ are photographed with images of single postures at different object distances, and the present invention is used to obtain calibration results at different object distances. A chessboard calibration plate ⑦ with a field of view of about 1 / 9 is placed in the near field, and images of different postures are taken, and Zhang Zhengyou calibration method is used to complete the initial calibration of the internal and external parameters of the imaging system. The specific implementation steps are as follows:

[0032] S1: Build a variable object distance and large field of view camera calibration experimental system to determine the pixel size dx and dy of the digital camera imaging sensor and the focal length f of the fixed focus lens used;

[0033] S2: Adjust the focus ring of the imaging system to complete the acquisition of different posture images of the checkerboard calibration plate ⑦; in the acquisition of images, The field of view area of ​​the checkerboard calibration plate ⑦ accounts for about 1 / 9;

[0034] S3: Based on the chessboard calibration plate ⑦ image at the near view collected in step S2, the Zhang Zhengyou calibration method is used to complete the calibration of the imaging system's internal parameter matrix, distortion coefficient, translation vector and rotation matrix;

[0035] S4: In Figure 1Place checkerboard calibration plates ① to ⑥ in the areas A / B / C shown, adjust the focus ring of the imaging system, and clearly image the checkerboard calibration plates at different object distances; collect images in multiple postures for checkerboard calibration plates ① to ③; collect images in one posture for checkerboard calibration plates ④ to ⑥. In the collected images, the field of view of checkerboard calibration plates ① to ③ accounts for about 1 / 9, and the field of view of checkerboard calibration plates ④ to ⑥ accounts for about 1 / 64;

[0036] S5: Select the single images of the checkerboard calibration plates ④ to ⑥ taken in S4, and construct the homography matrix H respectively;

[0037] Afterwards, follow Figure 2 The solution block diagram shown in FIG. 1 is implemented in S6 to S10 to determine the calibration results of the chessboard calibration plates ④ to ⑥ of the present invention. According to the Zhang Zhengyou calibration method, the calibration results of the chessboard calibration plates ① to ③ are determined.

[0038] S6: Estimate the translation vector T′ and the rotation matrix R′ according to the homography matrix H in S5 and the intrinsic parameter matrix K calibrated in the near view;

[0039] S7: Calculate the object distance u′ according to the translation vector T′ estimated in S6;

[0040] S8: Based on the object distance u′ estimated in S7, using the lens imaging model Estimate the image distance v′;

[0041] S9: Update the equivalent focal length according to the image distance v′ and the camera imaging sensor pixel size dx and dy Construct a new internal parameter matrix

[0042] S10: Determine whether |uu′| is less than the threshold ε. If it is less than the threshold, output the current intrinsic parameter matrix, translation vector and rotation matrix; if it is greater than the threshold, set K = K′, T = T′, iterate S6 to S9, update the translation vector, rotation matrix and intrinsic parameter matrix until |uu′| is less than the threshold.

[0043] After the above specific actual experiments, the calibration results of using the checkerboard calibration plates ①~③ and the calibration results of using the checkerboard calibration plates ④~⑥ at different object distances are as follows Figure 3 As shown. The results show that the present invention only requires that the chessboard field of view accounts for 1 / 64, which can achieve the calibration effect of the conventional Zhang Zhengyou calibration method when the chessboard field of view accounts for 1 / 9, greatly reducing the requirements of the calibration method on the field of view of the calibration object. In the case of calibration objects of the same size, it is suitable for calibration with a larger field of view. At the same time, the present invention only needs to shoot a single chessboard image at different object distances, which greatly simplifies the camera calibration in variable object distance scenes.

[0044] Compared with the prior art, the performance indicators of the present device / method are improved in that: through the lens imaging model and the homography matrix, a quantitative relationship between the intrinsic parameter matrix and the translation vector of the imaging system is established, and through the iterative solution of steps S6 to S10, a chessboard with a smaller field of view can be used to achieve high-precision imaging parameter calibration.

[0045] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims

1. A method for calibrating a camera with variable object distance and large field of view considering a lens imaging model, characterized in that: include: S1: Build an imaging system in a variable object distance and large field of view measurement scenario, determine the pixel sizes dx and dy of the digital camera imaging sensor and the focal length f of the fixed-focus optical lens; S2: placing a two-dimensional checkerboard calibration plate with known coordinates at the near-view position, adjusting the focus ring of the fixed-focus optical lens, so that the imaging system in step S1 can clearly image the two-dimensional checkerboard calibration plate at the near-view position, and collecting images of the checkerboard calibration plate in different positions; S3: Based on the two-dimensional chessboard calibration plate image at the near view acquired in step S2, the Zhang Zhengyou calibration method is used to complete the calibration of the imaging system internal parameters, distortion coefficients and external parameters; S4: placing a two-dimensional checkerboard calibration plate with known coordinates at different object distances / distant positions, adjusting the focus ring of the fixed-focus optical lens, so that the imaging system in step S1 can clearly image the two-dimensional checkerboard calibration plate at different object distances / distant positions, and collecting an image of the checkerboard calibration plate in a single posture; S5: constructing a homography matrix H according to the world coordinates of the corner points of the two-dimensional chessboard calibration plate in step S4 and their pixel coordinates in the image; S6: Estimate the translation vector T′ and rotation matrix R′ at different object distances / distant scenes according to the homography matrix H and the intrinsic parameter matrix K; S7: Calculate the object distance according to the translation vector T′ estimated in S6 S8: Based on the object distance u′ estimated in S7, using the lens imaging model Get the estimated value v′ of the image distance at different object distances / distant scenes; S9: Update the equivalent focal length based on the estimated image distance v′ and the camera imaging sensor pixel sizes dx and dy Construct a new internal parameter matrix S10: Determine whether |uu′| is less than the threshold ε; if it is less than the threshold, output the current intrinsic parameter matrix, translation vector and rotation matrix; if it is greater than the threshold, set K=K′, T=T′, iterate S6~S9, update the translation vector, rotation matrix and intrinsic parameter matrix until |uu′| is less than the threshold.

2. The variable object distance and large field of view camera calibration method considering the lens imaging model according to claim 1 is characterized in that: The area of ​​the two-dimensional chessboard calibration plate in the field of view accounts for greater than or equal to 1 / 9.

3. The variable object distance and large field of view camera calibration method considering the lens imaging model according to claim 1, characterized in that: The internal parameter matrix Including equivalent focal length f x and f y , principal point coordinates (c x ,c y ); The distortion coefficients are radial distortion coefficients k1 and k2; The external parameters include the translation vector Where: T x , T y , T z are the coordinate components in three directions, which can be used to solve the object distance And the rotation matrix R = [r1r2 r3], where: r1, r2, r3 represent the first, second, and third column vectors of R.

4. The variable object distance and large field of view camera calibration method considering the lens imaging model according to claim 1, characterized in that: The area of ​​the two-dimensional chessboard calibration plate in the field of view accounts for greater than or equal to 1 / 64.

5. The variable object distance and large field of view camera calibration method considering the lens imaging model according to claim 1, characterized in that: The homography matrix H is a transformation matrix H=[h1 h2 h3] between the world coordinates of the corner points of the two-dimensional chessboard calibration plate and their image pixel coordinates, wherein h1, h2, h3 are the first, second, and third column vectors of the homography matrix.

6. The variable object distance and large field of view camera calibration method considering the lens imaging model according to claim 1, characterized in that: The translation vector in: The rotation matrix Among them: r1′, r2′, r3′ are the first, second and third column vectors of the rotation matrix R′ respectively.

Images