Full-automatic calibration method and device for large-range zoom imaging system

Abstract

The invention provides a full-automatic calibration method and device for a large-range zoom imaging system. The calibration device comprises a remote control intelligent trolley, a two-degree-of-freedom rotating holder, a calibration plate, an auxiliary camera, a set of trolley advancing guide device and a main control computer. The calibration plate is mounted on a holder carried on a trolley, and the trolley is guided to advance along the optical axis direction of the zoom imaging system and stays at a series of specified positions. And during stay, the main control computer drives the zoom lens to magnify the calibration plate image to a proper size, controls the holder to rotate to provide calibration plate images with different postures for a calibration algorithm, and controls the zoom imaging system and the auxiliary camera carried on the trolley to synchronously record the calibration plate image. And after a series of calibration results corresponding to the zooming positions are obtained through a calibration algorithm, the internal reference of the imaging system corresponding to any zooming position is obtained through fitting. The high-precision and high-efficiency calibration problem of the large-range zoom imaging system is solved, and the application prospect is wide.

Description

Technical Field

[0001] This invention relates to an intrinsic parameter calibration method for a zoom imaging system, and particularly to a fully automatic calibration method and apparatus for a wide-range zoom imaging system. Background Technology

[0002] Existing vision systems mainly include three types: monocular, binocular, and multi-view vision systems, consisting of one, two, and multiple vision imaging systems, respectively. In applications involving vision system measurement, calibration is necessary. For monocular vision systems, the intrinsic parameters of the imaging system need to be calibrated, namely the principal point coordinates, lens focal length, and lens distortion coefficient. For binocular and multi-view vision systems, in addition to calibrating the intrinsic parameters of each imaging system, the extrinsic parameters of each imaging system also need to be calibrated, namely the position and orientation parameters of each imaging system in a unified coordinate system. Therefore, intrinsic parameter calibration of the imaging system is essential in all types of vision systems.

[0003] Existing imaging system calibration methods primarily target fixed-focus imaging systems with fixed lens focal lengths. However, many applications involving visual measurements over large spatial areas often require variable focal lengths in the imaging system to ensure accurate measurement of targets at different locations within the measurement space. Changes in focal length will alter the intrinsic parameters of the imaging system, thus affecting the measurement results. Therefore, calibration of zoom imaging systems is necessary.

[0004] In an electric zoom imaging system, the rotational position of the zoom drive motor can be read. Calibration of a zoom imaging system essentially requires obtaining the intrinsic parameters of the imaging system at each rotational position of the zoom drive motor. Theoretically, intrinsic parameters can be calibrated at several zoom positions using existing imaging system calibration methods, and then the intrinsic parameters for each zoom position can be calculated using a fitting algorithm. However, in practical applications, when the focal length of the imaging system is large, widely used imaging system calibration methods, such as the well-known Zhang checkerboard calibration method (Zhang Zhengyou's 1998 paper "A Flexible New Technique for Camera Calibration"), become ineffective. Currently, there is no mature and universally applicable method for calibrating large-range zoom imaging systems. Summary of the Invention

[0005] To overcome the problems in existing technologies, such as the inability of traditional calibration methods to obtain the intrinsic parameters of the imaging system in real time when the focal length changes during the operation of a wide-range zoom imaging system, this invention provides a fully automatic calibration method and device for a wide-range zoom imaging system. Even when the focal length becomes large, the calibrated zoom imaging system can still calculate the accurate intrinsic parameters of the imaging system using the zoom position information obtained by the electric zoom device.

[0006] The fully automatic calibration method and apparatus for a wide-range zoom imaging system proposed in this invention calculates the relationship between the intrinsic parameters of the zoom imaging system and the rotational position of the motor by acquiring a series of image information and the rotational position information of the zoom motor. Then, the corresponding intrinsic parameters of the zoom imaging system can be calculated in real time using the rotational position information of the zoom motor. This invention is achieved through the following technical solution:

[0007] This invention discloses a fully automatic calibration device for a wide-range zoom imaging system, comprising:

[0008] Calibration board: Used as a target with known features to provide calibration image information to the zoom imaging system to be calibrated;

[0009] Two-degree-of-freedom rotary gimbal: used to rotate the calibration plate and provide a set of calibration plate images in different poses for the zoom imaging system to be calibrated;

[0010] Auxiliary camera: used to synchronously acquire images of the calibration board with the zoom imaging system to be calibrated;

[0011] Programmable trolley: Used to move the calibration board along the optical axis of the zoom imaging system to be calibrated, and to adjust the distance between the zoom imaging system and the calibration board;

[0012] The travel direction guidance device includes one forward-looking guidance camera, one rear-looking guidance camera, one front guidance light, and one rear guidance light, which are used to guide the programmable trolley to move in the optical axis direction of the zoom imaging system to be calibrated.

[0013] Lower-level computer: Used to receive control commands from the upper-level computer, acquire images from the forward-looking guidance camera and the rear-looking guidance camera, calculate the forward direction control parameters of the programmable trolley, control the movement of the programmable trolley, control the auxiliary camera to acquire images of the calibration board, and upload the acquired images to the upper-level computer;

[0014] The host computer is used to control the zoom and focus of the zoom imaging system to be calibrated, acquire the calibration board image obtained by the zoom imaging system to be calibrated, send control commands to the lower computer, receive the calibration board image acquired by the auxiliary camera sent by the lower computer, calculate the intrinsic parameters of the auxiliary camera, calculate the intrinsic parameters of the zoom imaging system to be calibrated at different zoom positions, and fit the relationship between the intrinsic parameters of the zoom imaging system to be calibrated and the zoom position.

[0015] The host computer is connected to the zoom imaging system to be calibrated and to the slave computer via a wireless network. The slave computer is connected to the programmable trolley, the two-degree-of-freedom rotating pan-tilt head, the auxiliary camera, the forward-looking guidance camera, and the rear-looking guidance camera. The two-degree-of-freedom rotating pan-tilt head, the slave computer, the forward-looking guidance camera, the rear-looking guidance camera, and the auxiliary camera are fixed on the programmable trolley.

[0016] As a further improvement, the intersection of the horizontal rotation axis and the pitch rotation axis of the two-degree-of-freedom rotating gimbal described in this invention coincides with the center of the calibration plate. When the calibration plate is in its initial pose, its calibration pattern surface faces the zoom imaging system to be calibrated. Throughout the calibration process, the optical axis of the zoom imaging system to be calibrated always points to the center of the calibration plate.

[0017] As a further improvement, the auxiliary camera of the present invention is installed at the same height as the center of the calibration board, with its optical axis pointing to the center of the calibration board and having an incident angle of approximately 30° with the pattern surface of the calibration board in the initial pose. The distance from the center of the calibration board is 300-500mm, ensuring that the auxiliary camera can acquire complete images of the calibration board in each pose throughout the calibration process, without obstructing the complete images of the calibration board acquired by the zoom imaging system to be calibrated. The forward-looking guidance camera and the rear-looking guidance camera are located at the middle of the front and rear of the programmable trolley, respectively, with their optical axes pointing directly in front of and directly behind the trolley, respectively.

[0018] This invention also discloses a fully automatic calibration method for a wide-range zoom imaging system, comprising:

[0019] S1 Auxiliary Camera Calibration: The lower-level computer controls the two-degree-of-freedom rotating gimbal to rotate the calibration plate and controls the auxiliary camera to acquire multiple images of the calibration plate in different poses. The intrinsic parameters of the auxiliary camera are accurately calibrated according to Zhang Zhengyou's calibration method to ensure that the three-dimensional coordinates of the actual object are accurately mapped to the image coordinates captured by the auxiliary camera.

[0020] S2 Guide Light Setting: Place the front guide light and the rear guide light in the direction of the optical axis of the zoom imaging system under test. The front guide light should be placed far in front of the zoom imaging system, and the rear guide light should be placed near the zoom imaging system. Ensure that the front-view guide camera and the rear-view guide camera mounted on the programmable trolley can observe the front guide light and the rear guide light respectively during the entire calibration process.

[0021] S3 Initial pose setting of the programmable trolley: Place the programmable trolley at the minimum object distance of the zoom imaging system under test, with the forward direction pointing towards the front guide light. The forward-looking guide camera on the programmable trolley captures the front guide light set in step S2 at the left and right center position in its image, and the rear-looking guide camera on the programmable trolley captures the rear guide light set in step S2 at the left and right center position in its image.

[0022] S4 Initial pose setting of calibration board: After the programmable trolley is set to the initial pose described in step S3, the lower computer controls the two-degree-of-freedom rotating gimbal to drive the calibration board pattern to face the zoom imaging system under test and the line direction is parallel to the optical axis direction of the zoom imaging system under test. At this time, the horizontal angle θ and the pitch angle φ of the two-degree-of-freedom rotating gimbal are both zero.

[0023] S5. Calibration Image Data Acquisition: After setting the calibration board to the initial pose described in step S4, control the zoom imaging system under test to focus on and adaptively zoom the calibration board, record the initial zoom position of the zoom imaging system, and then control the two-degree-of-freedom gimbal to rotate via the lower-level computer. Select at equal intervals within the range A horizontal angle, in Select at equal intervals within the range A pitch angle, controlling the calibration board at... When the device is stationary at a certain pose, the calibration board image acquired by the zoom imaging system under test is collected, and the calibration board image acquired by the auxiliary camera is acquired by the lower computer. The binocular stereo vision calibration image data is obtained by the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera at the initial zoom position.

[0024] S6 Calibration image data acquisition at multiple zoom positions: The lower-level computer controls the programmable trolley to move forward a distance d along the optical axis of the zoom imaging system under test and then stops. The calibration data acquisition steps of S5 are repeated. After multiple "forward-stop-S5 calibration data acquisition" steps, the calibration data acquisition stops when the zoom position of the zoom imaging system under test reaches the maximum value. The result is the binocular stereo vision calibration image data captured by the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera at multiple zoom positions.

[0025] S7. Calibration of the zoom imaging system at multiple zoom positions: Using the binocular stereo vision calibration image data captured by the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera at the initial zoom position of the zoom imaging system acquired in step S5 and the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera acquired in step S6 at multiple zoom positions, the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera corresponding to each zoom position is calibrated. During the calibration process, the intrinsic parameters of the auxiliary camera obtained by the precise calibration in step S1 are fixed, and the intrinsic parameters of the zoom imaging system are obtained from them.

[0026] S8 Intrinsic parameters of zoom imaging system and zoom position relationship fitting: Using the intrinsic parameters of zoom imaging system at multiple zoom positions obtained in S7, fit the relationship between each intrinsic parameter component of zoom imaging system and zoom position.

[0027] As a further improvement, in step S5 of the present invention, the specific implementation steps of adaptive zoom include: controlling the focal length of the zoom imaging system to increase slowly, detecting the feature points of the calibration plate in real time, calculating the area ratio of the calibration plate in the image, and stopping zooming when the area ratio is greater than a set threshold, so as to ensure that the area ratio of the calibration plate in the image is large enough at each zoom position where calibration data is collected.

[0028] As a further improvement, in step S7 of this invention, the calibration of the binocular stereo vision system, consisting of the zoom imaging system and the auxiliary camera, employs an optimization algorithm based on the Levenberg-Marquardt nonlinear least squares problem to solve for the intrinsic and extrinsic parameters of each binocular camera, and obtains the intrinsic parameters of the zoom imaging system from this algorithm. During binocular stereo reconstruction, utilizing the principle that the object point, optical center, and image point are collinear, after finding the matching image point in the binocular image, two straight lines can be formed in three-dimensional space. The corresponding object point is the intersection of the two straight lines. Due to deviations, the two straight lines usually do not intersect at a single point; here, the intersection point refers to the least squares intersection point. These are the world coordinates of the object's point. and yes The coordinates are transformed to their respective camera coordinate systems by the extrinsic parameter matrices of the zoom imaging system and the auxiliary camera, i.e.:

[0029] (1)

[0030] (2)

[0031] in, and Let the extrinsic parameter matrices of the zoom imaging system and the auxiliary camera be respectively. Rearranging formulas (1) and (2), we get:

[0032] (3)

[0033] Rearranging in matrix form, we get:

[0034] (4)

[0035] in addition, and The image coordinates are projected onto the intrinsic parameter matrices of the zoom imaging system and the auxiliary camera, respectively. and ,Right now:

[0036] (5)

[0037] (6)

[0038] From (4), (5), and (6), it can be seen that the input matching image coordinates and Given the initial values ​​of the intrinsic and extrinsic parameters of the binocular system, the coefficient matrix of formula (4) is decomposed by SVD to calculate the corresponding three-dimensional world coordinates. Based on this, the following maximum likelihood estimation is proposed, and the intrinsic parameters of the zoom imaging system are solved by minimizing the function using the Levenberg-Marquardt nonlinear least squares optimization algorithm:

[0039] (7)

[0040] in, These are the world coordinates of the j-th feature point on the calibration board. It is by The coordinates of the j-th feature point projected onto the i-th calibration plate image captured by the zoom imaging system The coordinates of the j-th feature point projected onto the i-th calibration board image captured by the auxiliary camera. The world coordinates are calculated using formula (4). These are the intrinsic parameter matrix, extrinsic parameter rotation matrix, and extrinsic parameter translation vector of the zoom imaging system. These are the intrinsic parameter matrix, extrinsic parameter rotation matrix, and extrinsic parameter translation vector of the auxiliary camera. During the solution process... The internal parameters of the auxiliary camera calibrated in step S1 are always fixed.

[0041] The beneficial effects of this invention are as follows:

[0042] 1. It solves the problem of high-precision calibration of large-range zoom imaging systems, especially when the imaging system zooms to telephoto, the problem that currently widely used calibration methods, such as Zhang Zhengyou's checkerboard calibration method, cannot correctly calibrate the imaging system;

[0043] 2. Solved the problem of high-efficiency calibration of large-range zoom imaging systems, especially the problem of high-efficiency calibration image acquisition. The entire calibration process, from changing the distance between the calibration board and the zoom imaging system, to zooming and focusing of the zoom imaging system, to adjusting the pose of the calibration board, to acquiring calibration images at various poses of the calibration board, to calibrating the zoom imaging system at each zoom position, and finally to fitting the calibration results of the zoom imaging system, can be completed without manual intervention and with one click.

[0044] 3. The proposed method for guiding the forward direction of the programmable trolley utilizes a forward-looking guidance camera and a rear-looking guidance camera to simultaneously observe the front and rear guidance lights, ensuring that the programmable trolley moves strictly in the direction of the line connecting the front and rear guidance lights, and enabling the calibration board carried by the programmable trolley to move accurately on the optical axis of the zoom imaging system.

[0045] 4. In order to accurately calibrate the intrinsic parameters of the zoom imaging system at the telephoto position, an auxiliary camera was added to form a binocular stereo vision imaging system with the zoom imaging system under test. Using the accurately calibrated intrinsic parameters of the auxiliary camera, the intrinsic parameters of the auxiliary camera were fixed and the intrinsic parameters of the zoom imaging system were optimized during the calibration process of the binocular stereo vision imaging system, so as to accurately calibrate the intrinsic parameters of the zoom imaging system. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the fully automatic calibration device for the wide-range zoom imaging system of the present invention;

[0047] Figure 2 This is the checkerboard calibration template diagram used in the calibration algorithm employed in this invention;

[0048] Figure 3 This is a data acquisition flowchart of the fully automatic calibration method for the wide-range zoom imaging system of the present invention;

[0049] Figure 4 This is a data processing flowchart of the fully automatic calibration method for the wide-range zoom imaging system of the present invention.

[0050] In the diagram, 1 is the calibration plate, 2 is the two-degree-of-freedom rotating gimbal, 3 is the auxiliary camera, 4 is the programmable trolley, 5 is the forward-looking guidance camera, 6 is the rear-looking guidance camera, 7 is the front guide light, 8 is the rear guide light, 9 is the lower-level computer, 10 is the upper-level computer, and 11 is the zoom imaging system. Detailed Implementation

[0051] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings. For the sake of brevity, some content known in the art has been omitted when describing the processes, conditions, experimental methods, etc. of the embodiments of the present invention, and the present invention does not have any particular limitations on such content.

[0052] The following will combine Figure 1-4 The fully automatic calibration method and apparatus for the wide-range zoom imaging system of the present invention will be described in further detail.

[0053] The fully automatic calibration device for the wide-range zoom imaging system mentioned in this invention mainly includes:

[0054] Calibration board: Used as a target with known features to provide calibration image information to the zoom imaging system to be calibrated;

[0055] Two-degree-of-freedom rotary gimbal: used to rotate the calibration plate and provide a set of calibration plate images in different poses for the zoom imaging system to be calibrated;

[0056] Auxiliary camera: used to synchronously acquire images of the calibration board with the zoom imaging system to be calibrated;

[0057] Programmable trolley: Used to move the calibration board along the optical axis of the zoom imaging system to be calibrated, and to adjust the distance between the zoom imaging system and the calibration board;

[0058] The travel direction guidance device includes one forward-looking guidance camera, one rear-looking guidance camera, one front guidance light, and one rear guidance light, which are used to guide the programmable trolley to move in the optical axis direction of the zoom imaging system to be calibrated.

[0059] Lower-level computer: Used to receive control commands from the upper-level computer, acquire images from the forward-looking guidance camera and the rear-looking guidance camera, calculate the forward direction control parameters of the programmable trolley, control the movement of the programmable trolley, control the auxiliary camera to acquire images of the calibration board, and upload the acquired images to the upper-level computer;

[0060] The host computer is used to control the zoom and focus of the zoom imaging system to be calibrated, acquire the calibration board image obtained by the zoom imaging system to be calibrated, send control commands to the lower computer, receive the calibration board image acquired by the auxiliary camera sent by the lower computer, calculate the intrinsic parameters of the auxiliary camera, calculate the intrinsic parameters of the zoom imaging system to be calibrated at different zoom positions, and fit the relationship between the intrinsic parameters of the zoom imaging system to be calibrated and the zoom position.

[0061] like Figure 1 As shown, the host computer is connected to the zoom imaging system to be calibrated and to the slave computer via a wireless network. The slave computer is connected to the programmable trolley, the two-degree-of-freedom rotating pan-tilt head, the auxiliary camera, the forward-looking guidance camera, and the rear-looking guidance camera. The two-degree-of-freedom rotating pan-tilt head, the slave computer, the forward-looking guidance camera, the rear-looking guidance camera, and the auxiliary camera are fixed on the programmable trolley.

[0062] The intersection of the horizontal rotation axis and the pitch rotation axis of the two-degree-of-freedom gimbal coincides with the center of the calibration plate. When the calibration plate is in its initial pose, its calibration pattern surface faces the zoom imaging system to be calibrated. Throughout the calibration process, the optical axis of the zoom imaging system to be calibrated always points to the center of the calibration plate.

[0063] The auxiliary camera is installed at the same height as the center of the calibration board, with its optical axis pointing to the center of the calibration board and forming an incident angle of approximately 30° with the pattern surface of the calibration board in the initial pose. It is 300-500mm away from the center of the calibration board, ensuring that the auxiliary camera can acquire complete images of the calibration board in each pose throughout the calibration process, without obstructing the complete images of the calibration board acquired by the zoom imaging system to be calibrated. The forward-looking guidance camera and the rear-looking guidance camera are located at the middle of the front and rear of the programmable trolley, respectively, with their optical axes pointing directly in front of and directly behind the trolley, respectively.

[0064] This invention proposes a fully automatic calibration method for a wide-range zoom imaging system, comprising:

[0065] S1 Auxiliary Camera Calibration: A lower-level computer controls a two-degree-of-freedom gimbal to rotate the calibration plate and controls the auxiliary camera to acquire multiple images of the calibration plate in different poses. Figure 2 A planar calibration board is created from the shown checkerboard image. The intrinsic parameters of the auxiliary camera are precisely calibrated using Zhang Zhengyou's calibration method to ensure that the 3D coordinates of the actual object are accurately mapped to the image coordinates captured by the auxiliary camera. Generally, 20 images are sufficient. During the calibration process, maximum likelihood estimation is used to calculate the intrinsic parameters of the auxiliary camera. n images of the calibration board are acquired, and the calibration board has m feature points. Based on the known spatial coordinates of the feature points and the image coordinates of the feature points extracted from the calibration board images, the intrinsic parameters of the auxiliary camera and the extrinsic parameters for each image captured on the calibration board are calculated using maximum likelihood estimation. Maximum likelihood estimation can be obtained by minimizing the following function:

[0066] (1)

[0067] in These are the image coordinates of the j-th feature point in the i-th calibration plate image. It is the j-th feature point on the calibration board. The world coordinates are the projected coordinates on the i-th calibration plate image. It is the intrinsic parameter matrix of the auxiliary camera. The extrinsic parameter matrix of the auxiliary camera is used when capturing the i-th calibration board image, and the intrinsic parameter matrix of the auxiliary camera represents the rotation and translation transformations from the world coordinate system to the auxiliary camera coordinate system. It can be represented as:

[0068] (2)

[0069] in and It is the focal length value of the auxiliary camera, measured in units of a single pixel size of the image sensor. and These are the coordinates of the principal point. Minimizing function (1) is a nonlinear minimization problem, which can be solved using the Levenberg-Marquardt algorithm.

[0070] S2 Guide Light Setting: Place the front guide light and the rear guide light in the direction of the optical axis of the zoom imaging system under test. The front guide light should be placed far in front of the zoom imaging system, and the rear guide light should be placed near the zoom imaging system. Ensure that the front-view guide camera and the rear-view guide camera mounted on the programmable trolley can observe the front guide light and the rear guide light respectively during the entire calibration process.

[0071] S3 Initial pose setting of the programmable trolley: Place the programmable trolley at the minimum object distance of the zoom imaging system under test, with the forward direction pointing towards the front guide light. The forward-looking guide camera on the programmable trolley captures the front guide light set in step S2 at the left and right center position in its image, and the rear-looking guide camera on the programmable trolley captures the rear guide light set in step S2 at the left and right center position in its image.

[0072] S4 Initial pose setting of calibration board: After the programmable trolley is set to the initial pose described in step S3, the lower computer controls the two-degree-of-freedom rotating gimbal to drive the calibration board pattern to face the zoom imaging system under test and the line direction is parallel to the optical axis direction of the zoom imaging system under test. At this time, the horizontal angle θ and the pitch angle φ of the two-degree-of-freedom rotating gimbal are both zero.

[0073] S5. Calibration Image Data Acquisition: After setting the calibration board to the initial pose described in step S4, control the zoom imaging system under test to focus on and adaptively zoom the calibration board, record the initial zoom position of the zoom imaging system, and then control the two-degree-of-freedom gimbal to rotate via the lower-level computer. Select at equal intervals within the range A horizontal angle, in Select at equal intervals within the range A pitch angle, controlling the calibration board at... When the device is stationary at a certain pose, the calibration board image acquired by the zoom imaging system under test is collected, and the calibration board image acquired by the auxiliary camera is acquired by the lower computer. The binocular stereo vision calibration image data is obtained by the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera at the initial zoom position.

[0074] S6 Calibration image data acquisition at multiple zoom positions: The lower-level computer controls the programmable trolley to move forward a distance d along the optical axis of the zoom imaging system under test and then stops. The calibration data acquisition steps of S5 are repeated. After multiple "forward-stop-S5 calibration data acquisition" steps, the calibration data acquisition stops when the zoom position of the zoom imaging system under test reaches the maximum value. The result is the binocular stereo vision calibration image data captured by the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera at multiple zoom positions.

[0075] S7. Calibration of the zoom imaging system at multiple zoom positions: Using the binocular stereo vision calibration image data captured by the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera at the initial zoom position of the zoom imaging system acquired in step S5 and the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera acquired in step S6 at multiple zoom positions, the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera corresponding to each zoom position is calibrated. During the calibration process, the intrinsic parameters of the auxiliary camera obtained by the precise calibration in step S1 are fixed, and the intrinsic parameters of the zoom imaging system are obtained from them.

[0076] S8 Intrinsic parameters of zoom imaging system and zoom position relationship fitting: Using the intrinsic parameters of zoom imaging system at multiple zoom positions obtained in S7, fit the relationship between each intrinsic parameter component of zoom imaging system and zoom position.

[0077] In step S5, the specific implementation steps of adaptive zoom include: controlling the focal length of the zoom imaging system to increase slowly, detecting the feature points of the calibration plate in real time, calculating the area ratio of the calibration plate in the image, and stopping zooming when the area ratio is greater than a set threshold, so as to ensure that the area ratio of the calibration plate in the image is large enough at each zoom position where calibration data is collected.

[0078] In step S7, the calibration of the binocular stereo vision system, consisting of the zoom imaging system and the auxiliary camera, employs an optimization algorithm based on the Levenberg-Marquardt nonlinear least squares problem to solve for the intrinsic and extrinsic parameters of each binocular camera, thereby obtaining the intrinsic parameters of the zoom imaging system. During binocular stereo reconstruction, utilizing the principle that the object point, optical center, and image point are collinear, after finding the matching image point in the binocular image, two straight lines can be formed in three-dimensional space. The corresponding object point is the intersection of these two lines. Due to deviations, the two lines typically do not intersect at a single point; here, the intersection point refers to the least squares intersection point. These are the world coordinates of the object's point. and yes The coordinates are transformed to their respective camera coordinate systems by the extrinsic parameter matrices of the zoom imaging system and the auxiliary camera, i.e.:

[0079] (1)

[0080] (2)

[0081] in, and Let the extrinsic parameter matrices of the zoom imaging system and the auxiliary camera be respectively. Rearranging formulas (1) and (2), we get:

[0082] (3)

[0083] Rearranging in matrix form, we get:

[0084] (4)

[0085] in addition, and The image coordinates are projected onto the intrinsic parameter matrices of the zoom imaging system and the auxiliary camera, respectively. and ,Right now:

[0086] (5)

[0087] (6)

[0088] From (4), (5), and (6), it can be seen that the input matching image coordinates and Given the initial values ​​of the intrinsic and extrinsic parameters of the binocular system, the coefficient matrix of formula (4) is decomposed by SVD to calculate the corresponding three-dimensional world coordinates. Based on this, the following maximum likelihood estimation is proposed, and the intrinsic parameters of the zoom imaging system are solved by minimizing the function using the Levenberg-Marquardt nonlinear least squares optimization algorithm:

[0089] (7)

[0090] in, These are the world coordinates of the j-th feature point on the calibration board. It is by The coordinates of the j-th feature point projected onto the i-th calibration plate image captured by the zoom imaging system The coordinates of the j-th feature point projected onto the i-th calibration board image captured by the auxiliary camera. The world coordinates are calculated using formula (4). These are the intrinsic parameter matrix, extrinsic parameter rotation matrix, and extrinsic parameter translation vector of the zoom imaging system. These are the intrinsic parameter matrix, extrinsic parameter rotation matrix, and extrinsic parameter translation vector of the auxiliary camera. During the solution process... The internal parameters of the auxiliary camera calibrated in step S1 are always fixed.

[0091] The fully automatic calibration method for the wide-range zoom imaging system mentioned in the invention mainly includes two parts: a fully automatic calibration data acquisition method and a calibration data processing method.

[0092] Fully automated calibration data acquisition methods, such as Figure 3 As shown, the main steps include the following:

[0093] S101. The programmable trolley is placed at the minimum object distance of the zoom imaging system. The forward direction of the programmable trolley is parallel to its optical axis, and its optical axis is perpendicular to the initial position of the calibration plate and passes through the center of the calibration plate.

[0094] S102. The host computer sends an auxiliary camera calibration data acquisition command to the programmable trolley;

[0095] S103. A programmable trolley drives a two-degree-of-freedom rotating gimbal to rotate, assisting the camera in capturing a set of calibration plate images in different orientations and uploading them to the host computer;

[0096] S104. The host computer sends a forward command to the programmable trolley;

[0097] S105. The programmable trolley moves a certain distance along the optical axis of the zoom imaging system under the guidance of the front and rear guide lights and then stops, and reports the stop status to the host computer.

[0098] S106. The host computer drives the zoom imaging system to focus on the calibration board, making the image of the calibration board clear;

[0099] S107. The host computer drives the zoom imaging system to zoom, stops zooming after the calibration plate occupies the main part of the image, and records the zoom position.

[0100] S108. The host computer sends an auxiliary camera calibration data acquisition command to the programmable trolley;

[0101] S109. The programmable trolley drives the two-degree-of-freedom rotating gimbal to rotate. After the rotation stops, the auxiliary camera takes a picture of the calibration board and uploads it to the host computer. At the same time, the host computer is notified to control the zoom imaging system to take a picture of the calibration board in this state.

[0102] S110. Repeat step S109 until the number of calibration board images taken in different orientations is sufficient to complete the zoom imaging system calibration;

[0103] S111. Determine whether the zoom position has reached the maximum zoom range. If not, repeat steps S104 to S110. If it has reached the maximum zoom range, end the calibration data acquisition process.

[0104] Data processing of the fully automatic calibration method for the wide-range zoom imaging system mentioned in the invention, such as... Figure 4 As shown, the main steps include the following:

[0105] S201. The host computer uses a set of calibration board images captured by the auxiliary camera in step S103 to calibrate the intrinsic parameters of the auxiliary camera;

[0106] S202. The host computer reads a set of calibration board image pairs with different orientations that are synchronously captured by the zoom imaging system and the auxiliary camera in steps S109 and S110 corresponding to a zoom position recorded in step S107. The zoom imaging system and the auxiliary camera are used as a binocular stereo vision system for binocular stereo vision calibration. During the calibration process, the intrinsic parameters of the auxiliary camera are fixed to the intrinsic parameters that were calibrated in step S201. After the calibration is completed, the intrinsic parameters of the zoom imaging system at that zoom position are recorded.

[0107] S203. The host computer traverses each zoom position recorded in step S107, repeats step S202, calibrates and records the intrinsic parameters of the zoom imaging system at each zoom position;

[0108] S204. Fit the intrinsic parameters of the zoom imaging system obtained from calibration at different zoom positions to obtain the fitting formula, and complete the intrinsic parameter calibration of the zoom imaging system.

[0109] The fully automatic calibration method and apparatus for a wide-range zoom imaging system mentioned in this invention are not limited to the embodiments described above. Various modifications and improvements can be made without departing from the principle of this invention.

[0110] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of this invention are included in this invention and are protected by the appended claims.

Claims

1. A full-automatic calibration device for a large zoom imaging system, characterized in that, The application relates to a zoom imaging system calibration device. A calibration board is used as a target with known features to provide calibration image information to a zoom imaging system to be calibrated; A two-degree-of-freedom rotating holder is used to rotate the calibration board to provide a set of calibration board images with different poses for the zoom imaging system to be calibrated; An auxiliary camera is used to synchronously acquire the calibration board images with the zoom imaging system to be calibrated; A program-controlled trolley is used to carry the calibration board to move along the optical axis of the zoom imaging system to be calibrated to adjust the distance between the zoom imaging system and the calibration board; A forward-looking direction guiding device and a rear-looking direction guiding device are used to guide the program-controlled trolley to move along the optical axis of the zoom imaging system to be calibrated; A lower computer is used to receive control instructions from an upper computer, acquire images of the forward-looking direction guiding device and the rear-looking direction guiding device, calculate the forward direction control parameters of the program-controlled trolley, control the movement of the program-controlled trolley, control the auxiliary camera to acquire the images of the calibration board, and upload the acquired images to the upper computer; The upper computer is used to control the zooming and focusing of the zoom imaging system to be calibrated, acquire the calibration board images acquired by the zoom imaging system to be calibrated, send control instructions to the lower computer, receive the calibration board images acquired by the auxiliary camera and sent by the lower computer, calculate the intrinsic parameters of the auxiliary camera, calculate the intrinsic parameters of the zoom imaging system to be calibrated at different zooming positions, and fit the relationship between the intrinsic parameters of the zoom imaging system to be calibrated and the zooming positions. The upper computer is connected with the zoom imaging system to be calibrated, connected with the lower computer through a wireless network, the lower computer is connected with the program-controlled trolley, the two-degree-of-freedom rotating holder, the auxiliary camera, the forward-looking direction guiding device and the rear-looking direction guiding device, the two-degree-of-freedom rotating holder, the lower computer, the forward-looking direction guiding device, the rear-looking direction guiding device and the auxiliary camera are fixed on the program-controlled trolley.

2. The apparatus according to claim 1, wherein The intersection of the horizontal rotating shaft and the pitching rotating shaft of the two-degree-of-freedom rotating holder coincides with the center of the calibration board, the calibration pattern surface of the calibration board faces the zoom imaging system to be calibrated at the initial pose, and the optical axis of the zoom imaging system to be calibrated always points to the center of the calibration board during the whole calibration process.

3. The apparatus according to claim 1, wherein The auxiliary camera is installed at the same height as the center of the calibration board, the optical axis of the auxiliary camera points to the center of the calibration board and forms an about 30-degree incident angle with the calibration pattern surface of the calibration board at the initial pose, the distance between the optical axis of the auxiliary camera and the center of the calibration board is 300-500 mm, the auxiliary camera can acquire the complete calibration board images at different poses during the whole calibration process, the forward-looking direction guiding device and the rear-looking direction guiding device are respectively arranged at the front and rear intermediate positions of the program-controlled trolley, and the optical axes of the forward-looking direction guiding device and the rear-looking direction guiding device respectively point to the front and rear of the trolley.

4. A full-automatic calibration method for a large zoom imaging system, characterized in that, S1 auxiliary camera calibration: the two-degree-of-freedom rotating holder is controlled by the lower computer to rotate the calibration board, and the auxiliary camera is controlled to acquire a plurality of calibration board images with different poses, the intrinsic parameters of the auxiliary camera are accurately calibrated according to Zhang Zhengyou's calibration method, and the three-dimensional coordinate calibration of the actual object is accurately mapped to the image coordinates captured by the auxiliary camera. ​ S2 guiding light setting: place the front guiding light and the rear guiding light in the optical axis direction of the zoom imaging system to be measured, place the front guiding light far away from the front of the zoom imaging system, and place the rear guiding light near the zoom imaging system, so as to ensure that the front-view guiding camera and the rear-view guiding camera carried on the program-controlled trolley can observe the front guiding light and the rear guiding light, respectively, during the whole calibration process; S3 initial pose setting of the program-controlled trolley: place the program-controlled trolley at the minimum object distance of the zoom imaging system to be measured, and the forward direction points to the front guiding light; the front-view guiding camera carried on the program-controlled trolley photographs the front guiding light set in step S2 to be in the left and right center positions of the image; and the rear-view guiding camera carried on the program-controlled trolley photographs the rear guiding light set in step S2 to be in the left and right center positions of the image; S4 initial pose setting of the calibration board: after the program-controlled trolley is set to the initial pose described in step S3, the pattern of the calibration board is driven by the two-degree-of-freedom rotating holder to face the zoom imaging system to be measured, and the line direction is parallel to the optical axis direction of the zoom imaging system to be measured; at this time, the horizontal angle θ and the pitch angle φ of the two-degree-of-freedom rotating holder are both zero; S5. Calibration Image Data Acquisition: After setting the calibration board to the initial pose described in step S4, control the zoom imaging system under test to focus on and adaptively zoom the calibration board, record the initial zoom position of the zoom imaging system, and then control the two-degree-of-freedom gimbal to rotate via the lower-level computer. Select at equal intervals within the range A horizontal angle, in Select at equal intervals within the range A pitch angle, controlling the calibration board at... When the device is stationary at a certain pose, the calibration board image acquired by the zoom imaging system under test is collected, and the calibration board image acquired by the auxiliary camera is acquired by the lower computer. The binocular stereo vision calibration image data is obtained by the binocular stereo vision system consisting of the zoom imaging system and the auxiliary camera at the initial zoom position. S6 calibration image data acquisition at multiple zoom positions: stop the program-controlled trolley after it advances a distance d along the optical axis direction of the zoom imaging system to be measured; repeat the calibration data acquisition step S5; after multiple "advancing-stopping-S5 calibration data acquisition", stop the calibration data acquisition after the zoom position of the zoom imaging system to be measured reaches the maximum value; and obtain the binocular stereo vision calibration image data photographed by the binocular stereo vision system composed of the zoom imaging system and the auxiliary camera at multiple zoom positions; S7 zoom imaging system calibration at multiple zoom positions: use the binocular stereo vision calibration image data photographed by the binocular stereo vision system composed of the zoom imaging system and the auxiliary camera at the initial zoom position of the zoom imaging system acquired in step S5 and at multiple zoom positions of the zoom imaging system acquired in step S6 to calibrate the binocular stereo vision system composed of the zoom imaging system and the auxiliary camera corresponding to each zoom position; fix the auxiliary camera intrinsic parameter obtained in the accurate calibration in step S1 during the calibration process; and obtain the intrinsic parameter of the zoom imaging system therefrom; S8 fitting of the relationship between the intrinsic parameter of the zoom imaging system and the zoom position: use the intrinsic parameters of the zoom imaging system at multiple zoom positions obtained in S7 to fit the relationship between each intrinsic parameter component of the zoom imaging system and the zoom position.

5. The method of claim 4, wherein, In the step S5, the specific implementation steps of the adaptive zooming include: controlling the focal length of the zoom imaging system to slowly increase, detecting the feature points of the calibration board in real time, calculating the area proportion of the calibration board in the image, and stopping zooming when the area proportion is greater than a set threshold, so as to ensure that the area proportion of the calibration board in the image is large enough at each zoom position where the calibration data is collected.

6. The method of claim 4, wherein, In the step S7, the calibration of the binocular stereo vision system composed of the zoom imaging system and the auxiliary camera is solved by using the optimization algorithm of the Levenberg-Marquardt nonlinear least square problem to obtain the internal and external parameters of the binoculars, and the internal parameters of the zoom imaging system are obtained. In the binocular stereo reconstruction, the principle of collinearity of the object point, the optical center and the image point is used, and after the matched image points in the binocular images are found, two straight lines can be formed in the three-dimensional space. The corresponding object point is the intersection point of the two straight lines. The two straight lines usually do not intersect at a point due to deviation. Here, the intersection point refers to the least square intersection point. If is the world coordinate of the object point, and are the external parameter matrix of the zoom imaging system and the auxiliary camera is transformed to the coordinates in the respective camera coordinate system, that is: (1) (2) wherein, and are extrinsic matrixes of the zoom imaging system and the auxiliary camera, respectively, and the formulas (1) and (2) are arranged as follows: (3) The arrangement in the matrix form is as follows: (4) In addition, and are projected to the image coordinates of both by the intrinsic matrix of the zoom imaging system and the auxiliary camera, respectively and i.e.: (5) (6) From (4), (5), (6), it can be seen that the input matching image coordinates and , given the initial value of the intrinsic and extrinsic parameters of the binocular, the SVD decomposition is performed on the coefficient matrix of formula (4) to calculate the corresponding three-dimensional world coordinates Based on this, the following maximum likelihood estimation is proposed to minimize the function by Levenberg-Marquardt nonlinear least squares optimization algorithm to solve the intrinsic parameters of the zoom imaging system: (7) in, These are the world coordinates of the j-th feature point on the calibration board. It is by The coordinates of the j-th feature point projected onto the i-th calibration plate image captured by the zoom imaging system The coordinates of the j-th feature point projected onto the i-th calibration board image captured by the auxiliary camera. The world coordinates are calculated using formula (4). These are the intrinsic parameter matrix, extrinsic parameter rotation matrix, and extrinsic parameter translation vector of the zoom imaging system. These are the intrinsic parameter matrix, extrinsic parameter rotation matrix, and extrinsic parameter translation vector of the auxiliary camera. During the solution process... The internal parameters of the auxiliary camera calibrated in step S1 are always fixed.

Images