Calibration method for large field-of-view cameras with variable object distance considering lens imaging model

Abstract

A method for calibrating a variable-distance, large-field-of-view camera considering a lens imaging model is proposed. This method employs an imaging system including a fixed-focus lens to capture near-field and far-field images. Based on fixed principal point coordinates and distortion coefficients, and using the intrinsic parameter calibration results at the near-field location as initial values, the method iteratively calculates "extrinsic parameters based on intrinsic parameters and homography matrix – image distance based on lens imaging model – equivalent focal length (intrinsic parameters) based on image distance" to calibrate the variable-distance, large-field-of-view camera. This invention significantly simplifies the camera calibration process in variable-distance, large-field-of-view scenarios and greatly reduces the requirement for a checkerboard field of view.

Description

Technical Field

[0001] This invention relates to a technique in the field of image measurement, specifically a calibration method for a large field-of-view camera with variable object distance that takes into account a lens imaging model. Background Technology

[0002] Camera calibration techniques based on known-coordinate calibration objects, grounded in linear or nonlinear imaging theory, establish the correspondence between spatial points with known coordinates on the calibration object and their image points. Linear or nonlinear optimization algorithms are then used to obtain the intrinsic and extrinsic parameters of the camera imaging model. However, existing camera calibration techniques suffer from the following problems in scenarios with varying object distances and large fields of view: 1) Calibration methods that construct large calibration objects from small calibration objects are complex, inefficient, and require high precision in the motion control of the mechanical devices used to change the position of the small calibration object. 2) Self-calibration methods that do not require known-coordinate calibration objects need to be imaged from multiple perspectives, requiring a large number of cameras and are unsuitable for single-camera or dual-camera calibration. Furthermore, these methods require a large number of feature points on the calibration object, reaching thousands, significantly increasing the complexity of the calibration process. 3) None of these methods efficiently solve the camera calibration problem under varying object distances. Therefore, a simple, low-cost, efficient, and highly accurate camera calibration method is urgently needed to address the problem of camera calibration with varying object distances and large fields of view. Summary of the Invention

[0003] This invention addresses the problems of complex and inefficient camera calibration processes in variable object distance scenarios, and low calibration accuracy caused by high manufacturing costs and limited size of calibration objects in large field-of-view scenarios. It proposes a variable object distance large field-of-view camera calibration method that considers the lens imaging model.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a calibration method for a variable object distance, large field-of-view camera considering a lens imaging model, comprising:

[0006] S1: In a variable object distance and large field of view measurement scenario, build an imaging system (including a digital camera, a fixed-focus optical lens, etc.), and determine the pixel size dx and dy of the digital camera imaging sensor and the focal length f of the fixed-focus optical lens.

[0007] The imaging system can be a single camera, a dual camera, or multiple cameras;

[0008] The pixel sizes dx and dy of the imaging sensor can be the same;

[0009] The fixed-focus optical lens has a focus ring that can be manually or automatically focused. Through focusing, it can clearly image objects at different object distances.

[0010] S2: Place a two-dimensional checkerboard calibration plate with known coordinates at a close-up position, adjust the focus ring of the fixed-focus optical lens so that the imaging system in step S1 can clearly image the two-dimensional checkerboard calibration plate at the close-up position, and acquire checkerboard calibration plate images in different poses.

[0011] The area of ​​the two-dimensional checkerboard calibration plate in the field of view is greater than or equal to 1 / 9;

[0012] S3: Based on the two-dimensional checkerboard calibration board image of the near field acquired in step S2, use the Zhang Zhengyou calibration method to complete the calibration of the imaging system's intrinsic parameters, distortion coefficients, and extrinsic parameters;

[0013] The intrinsic parameter matrix Including equivalent focal length f x and f y Principal point coordinates (c x ,c y );

[0014] The distortion coefficients mentioned are radial distortion coefficients k1 and k2;

[0015] The extrinsic parameters include the translation vector. Wherein: T x T y T z These are coordinate components in three directions, which can be used to solve for the object distance. And the rotation matrix R = [r1r2 r3], where r1, r2, and r3 represent the first, second, and third column vectors of R.

[0016] S4: Place a two-dimensional checkerboard calibration plate with known coordinates at different object distances / distant locations, adjust the focus ring of the fixed-focus optical lens, so that the imaging system in step S1 can clearly image the two-dimensional checkerboard calibration plate at different object distances / distant locations, and acquire checkerboard calibration plate images in a single pose.

[0017] The area of ​​the two-dimensional checkerboard calibration plate in the field of view is greater than or equal to 1 / 64;

[0018] S5: Construct the homography matrix H based on the world coordinates of the corner points of the two-dimensional chessboard calibration board in step S4 and their pixel coordinates in the image;

[0019] The homography matrix H is a transformation matrix H = [h1 h2 h3] between the world coordinates of the corner points of the two-dimensional chessboard calibration board and their image pixel coordinates, where h1, h2, and h3 are the first, second, and third column vectors of the homography matrix.

[0020] S6: Estimate the translation vector T′ and rotation matrix R′ at different object distances / distant locations based on the homography matrix H and the intrinsic parameter matrix K;

[0021] The translation vector in:

[0022] The rotation matrix Where r1′, r2′, and r3′ are the first, second, and third column vectors of the rotation matrix R′, respectively.

[0023] S7: Calculate the object distance u′ based on the translation vector T′ estimated in S6;

[0024] The object distance

[0025] S8: Based on the object distance u′ estimated in S7, use the lens imaging model. Obtain estimated values ​​v′ for different object distances / image distances at distant locations;

[0026] S9: Update the equivalent focal length based on the estimated image distance v′ and the camera imaging sensor pixel sizes dx and dy. Construct a new intrinsic parameter matrix

[0027] S10: Determine if |uu′| is less than the threshold ε. If it is less than the threshold, output the current intrinsic parameter matrix, translation vector, and rotation matrix; if it is greater than the threshold, let K = K′, T = T′, and iterate through S6 to S9, updating the translation vector, rotation matrix, and intrinsic parameter matrix until |uu′| is less than the threshold.

[0028] Technical effect

[0029] This invention uses a lens imaging model that characterizes the relationship between image distance, object distance, and focal length to correlate the equivalent focal length calibration results of the imaging system at near-field (small field of view) and at different object distances / far-field (large field of view). Using the intrinsic and extrinsic parameter calibration results at near-field as initial values, and combining a homography matrix constructed from single checkerboard images taken at different object distances / far-fields with the lens imaging model, the invention achieves the calibration of camera intrinsic and extrinsic parameters at different object distances / far-field (large field of view) by iteratively solving for the object distance and intrinsic parameter matrices. Compared with existing technologies, this invention has a simpler operation process, lower cost, higher efficiency, and higher precision. In variable object distance and large field-of-view calibration, the size requirement for the two-dimensional calibration object is reduced (the area ratio of the calibration object in the field of view can be reduced to 1 / 64). Furthermore, only one two-dimensional calibration object image needs to be captured for each measured object distance to obtain the camera's intrinsic and extrinsic parameters. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of variable object distance large field of view calibration in one embodiment of the present invention.

[0031] Figure 2Flowchart for iterative solution of imaging parameters;

[0032] Figure 3 The image shown is a rendering of an example. Detailed Implementation

[0033] like Figure 1 The diagram illustrates a method for calibrating a large field-of-view camera with variable object distance, considering a lens imaging model, as described in this embodiment. The imaging system (including a single digital camera and a fixed-focus lens) images a checkerboard pattern at different object distances. The object distance is divided into three regions, denoted as regions A, B, and C. For each shot taken at a different object distance, the lens's focus ring needs adjustment to ensure sharpness. In region A, the imaging system is calibrated using a checkerboard calibration board ① (approximately 1 / 9 of the field of view) and a checkerboard calibration board ④ (approximately 1 / 64 of the field of view); in region B, the system is calibrated using a checkerboard calibration board ② (approximately 1 / 9 of the field of view) and a checkerboard calibration board ⑤ (approximately 1 / 64 of the field of view); and in region C, the system is calibrated using a checkerboard calibration board ③ (approximately 1 / 9 of the field of view) and a checkerboard calibration board ⑥ (approximately 1 / 64 of the field of view). In this process, checkerboard calibration boards ①, ②, and ③ are used to capture images of different poses at different object distances. The Zhang Zhengyou calibration method is used to obtain reference calibration results at different object distances. Checkerboard calibration boards ④, ⑤, and ⑥ are used to capture images of individual poses at different object distances. This invention is used to obtain calibration results at different object distances. A checkerboard calibration board ⑦, with a field of view of approximately 1 / 9, is placed in the foreground, and images of different poses are captured. The Zhang Zhengyou calibration method is used to complete the initial calibration of the internal and external parameters of the imaging system. The specific implementation steps are as follows:

[0034] S1: Build a calibration experimental system for a variable object distance large field of view camera, and determine the pixel size dx and dy of the imaging sensor of the digital camera used and the focal length f of the fixed-focus lens used.

[0035] S2: Adjust the focusing ring of the imaging system to complete the acquisition of images of the checkerboard calibration board ⑦ in different poses; in the acquired images

[0036] The field of view of the checkerboard calibration board ⑦ accounts for approximately 1 / 9;

[0037] S3: Based on the chessboard calibration board image ⑦ acquired in step S2, use Zhang Zhengyou calibration method to complete the calibration of the imaging system's intrinsic parameter matrix, distortion coefficients, translation vector, and rotation matrix.

[0038] S4: In such Figure 1In areas A / B / C shown, checkerboard calibration plates ① to ⑥ were placed respectively. The focusing ring of the imaging system was adjusted to achieve clear imaging of the checkerboard calibration plates at different object distances. Images of checkerboard calibration plates ① to ③ were acquired in multiple poses; images of checkerboard calibration plates ④ to ⑥ were acquired in one pose. In the acquired images, the field of view of checkerboard calibration plates ① to ③ occupies approximately 1 / 9, and the field of view of checkerboard calibration plates ④ to ⑥ occupies approximately 1 / 64.

[0039] S5: Select the single images of the checkerboard calibration board ④ to ⑥ taken in S4, and construct the homography matrix H for each;

[0040] After that, according to Figure 2 The solution flowchart shown is executed in steps S6 to S10 to determine the calibration results using checkerboard calibration plates ④ to ⑥. Following Zhang Zhengyou's calibration method, the calibration results using checkerboard calibration plates ① to ③ are determined.

[0041] S6: Based on the homography matrix H and the intrinsic parameter matrix K of the near-field calibration in S5, estimate the translation vector T′ and the rotation matrix R′;

[0042] S7: Calculate the object distance u′ based on the translation vector T′ estimated in S6;

[0043] S8: Based on the object distance u′ estimated in S7, use the lens imaging model. Estimate the image distance v′;

[0044] S9: Update the equivalent focal length based on the image distance v′ and the camera imaging sensor pixel sizes dx and dy. Construct a new intrinsic parameter matrix

[0045] S10: Determine if |uu′| is less than the threshold ε. If it is less than the threshold, output the current intrinsic parameter matrix, translation vector, and rotation matrix; if it is greater than the threshold, let K = K′, T = T′, and iterate through S6 to S9, updating the translation vector, rotation matrix, and intrinsic parameter matrix until |uu′| is less than the threshold.

[0046] Based on the above specific practical experiments, the calibration results using checkerboard calibration plates ①~③ and checkerboard calibration plates ④~⑥ at different object distances are as follows: Figure 3 As shown. The results indicate that this invention only requires the checkerboard area to occupy 1 / 64 of the field of view to achieve the calibration effect of the conventional Zhang Zhengyou calibration method when the checkerboard field of view occupies 1 / 9, greatly reducing the requirements of the calibration method on the field of view occupancy of the calibration object. With calibration objects of the same size, it is applicable to calibration of larger fields of view. Furthermore, this invention only requires capturing a single checkerboard image at different object distances, greatly simplifying camera calibration in scenarios with varying object distances.

[0047] Compared with existing technologies, the performance improvement of this device / method lies in the following: by using the lens imaging model and homography matrix, a quantitative relationship between the intrinsic parameter matrix and translation vector of the imaging system is established. Through iterative solutions in steps S6 to S10, high-precision imaging parameter calibration can be achieved using a checkerboard grid with a smaller field of view.

[0048] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A calibration method for a variable object distance, large field-of-view camera considering a lens imaging model, characterized in that, include: S1: In a large field-of-view measurement scenario with variable object distance, build an imaging system and determine the pixel size dx and dy of the digital camera imaging sensor and the focal length f of the fixed-focus optical lens. S2: Place a two-dimensional checkerboard calibration board with known coordinates at a close-up position, adjust the focus ring of the fixed-focus optical lens so that the imaging system in step S1 can clearly image the two-dimensional checkerboard calibration board at the close-up position, and acquire checkerboard calibration board images in different poses. S3: Based on the two-dimensional checkerboard calibration board image at the near view acquired in step S2, use the Zhang Zhengyou calibration method to complete the calibration of the imaging system's intrinsic parameters, distortion coefficients, and extrinsic parameters; S4: Place a two-dimensional checkerboard calibration board with known coordinates at different object distances / distant locations, adjust the focus ring of the fixed-focus optical lens, so that the imaging system in step S1 can clearly image the two-dimensional checkerboard calibration board at different object distances / distant locations, and acquire checkerboard calibration board images in a single pose. S5: Construct the homography matrix H based on the world coordinates of the corner points of the two-dimensional chessboard calibration board in step S4 and their pixel coordinates in the image; S6: Estimate the translation vector at different object distances / distant locations based on the homography matrix H and the intrinsic parameter matrix K. and rotation matrix ; S7: Based on the translation vector estimated in S6 Calculate object distance ; S8: Object distance calculated based on S7 Using lens imaging model Estimated values ​​of image distance at different object distances / distant locations were obtained. ; S9: Based on the estimated image distance Update the equivalent focal length using the camera imaging sensor pixel sizes dx and dy. , This forms a new intrinsic parameter matrix. ; S10: Judgment Is it less than the threshold? If the result is less than the threshold, output the current intrinsic parameter matrix, translation vector, and rotation matrix. If it is greater than the threshold, then let , Iterate through S6 to S9, updating the translation vector, rotation matrix, and intrinsic parameter matrix, until... Less than the threshold.

2. The method for calibrating a large field-of-view camera with variable object distance considering a lens imaging model according to claim 1, characterized in that, The area of ​​the two-dimensional checkerboard calibration plate in the field of view is greater than or equal to 1 / 9.

3. The method for calibrating a large field-of-view camera with variable object distance considering a lens imaging model according to claim 1, characterized in that, S6 The intrinsic parameter matrix described in Including equivalent focal length and Principal point coordinates ( , ); The distortion coefficient mentioned is the radial distortion coefficient. , ; The extrinsic parameters include the translation vector. and rotation matrix ,in: , , These are coordinate components in three directions, which can be used to solve for the object distance. ; , , Represents the first, second, and third column vectors of R.

4. The method for calibrating a large field-of-view camera with variable object distance considering a lens imaging model according to claim 1, characterized in that, The area of ​​the two-dimensional checkerboard calibration plate in the field of view is greater than or equal to 1 / 64.

5. The method for calibrating a large field-of-view camera with variable object distance considering a lens imaging model according to claim 1, characterized in that, S6 The homography matrix H mentioned above is the transformation matrix between the world coordinates of the corner points of the two-dimensional checkerboard calibration board and their image pixel coordinates. ,in: , , These are the first, second, and third column vectors of the homography matrix.

6. The method for calibrating a large field-of-view camera with variable object distance considering a lens imaging model according to claim 5, characterized in that, The translation vector ,in: ; The rotation matrix ,in: , , Rotation matrices The first, second, and third column vectors, and the rotation matrix. , , , Let R represent the first, second, and third column vectors of R.

Images