A laser tracker-based camera extrinsic parameter calibration method and system without common field of view

Abstract

This invention relates to a method and system for calibrating the extrinsic parameters of cameras without a common field of view based on a laser tracker. The method calibrates the extrinsic parameters of cameras without a common field of view using a laser tracker. First, a laser tracker is set up and a global coordinate system is defined. For each individual camera, if it does not share a common field of view with the tracker, the tracker is moved to a location with a common field of view, a temporary calibration board is set up, and the pose relationship between the temporary calibration board and the global coordinate system is calculated. When there is a common field of view, a calibration board with a target mount is set up in the common field of view. The target sphere coordinates are measured using the laser tracker, and the transformed sphere coordinates are measured using the camera. The target sphere coordinate system and the transformed sphere coordinate system are established and unified, thereby calculating the pose relationships between the camera and the calibration board, and between the calibration board and the global coordinate system. Finally, the extrinsic parameters of the camera without a common field of view are obtained by combining all pose relationships. Compared with existing technologies, this invention has the advantages of accurately and efficiently acquiring camera extrinsic parameters when processing camera systems without a common field of view, strong anti-interference ability, and high efficiency.

Description

Technical Field

[0001] This invention relates to the field of camera extrinsic parameter calibration technology, and in particular to a method and system for calibrating extrinsic parameters of a camera without a common field of view based on a laser tracker. Background Technology

[0002] Photogrammetry, with its advantages of non-contact operation, high precision, and dynamic capabilities, is widely used. Multi-camera systems, compared to single-camera systems, have attracted more attention due to their ability to acquire geometric information of large-volume, complex structures by expanding the field of view. However, in practical applications, multi-camera systems often lack a common field of view or have an extremely small common field of view, necessitating global calibration.

[0003] Global calibration is fundamental to the fusion of measurement data in multi-camera systems, aiming to transform measurement data from each camera into a unified global coordinate system. The result of global calibration is the camera's extrinsic parameters, which are independent of the camera's intrinsic parameters but closely related to the spatial layout and have a significant impact on the overall measurement accuracy of the system.

[0004] Currently, global calibration methods for multi-camera systems are mainly divided into two categories:

[0005] One type involves adding additional fixed constraints, including the fixed-connection camera method and the fixed-connection calibration plate method. The fixed-connection camera method uses a fixed-connection camera group to photograph a calibration plate with a fixed pose, and then uses the fixed connections to solve for the pose relationships between the cameras. The fixed-connection calibration plate method uses a large calibration plate to cover the fields of view of different cameras, and then uses the positional relationships of feature points on the target to solve for the pose relationships between the cameras. Both methods are flexible and convenient, but they suffer from problems such as a small range of combined target pose variations and unstable calibration results.

[0006] Another approach involves using additional equipment to provide extra information, such as the mirror method and the auxiliary camera method. The mirror method uses mirror reflection to expand the camera's field of view and determines the relative position and attitude of the cameras by calculating the relationship between the cameras and the target. However, in complex environments, it is difficult to guarantee that all cameras will observe the target. The auxiliary camera method uses additional cameras to provide pose information and solves the pose relationship between the cameras through coordinate transformation, but it limits the flexibility of camera placement.

[0007] Existing technologies such as CN104457569A and CN102506758A provide methods based on visual measurement systems, laser tracker systems, and positioning systems.

[0008] For example, the invention patent with publication number CN104457569A discloses a visual measurement method for the geometric parameters of large composite panels, which achieves rapid on-site 3D reconstruction and measurement through a binocular vision measurement system and a laser tracker system. However, this method faces the challenge of coordinate system unity when dealing with camera systems without a common field of view, especially when the laser tracker's field of view is obstructed. It cannot accurately and efficiently obtain the camera's extrinsic parameters, ensuring the accuracy of coordinate system unity and 3D data among multiple cameras.

[0009] The invention patent with publication number CN102506758A proposes a flexible dynamic vision measurement system and method for the three-dimensional shape of an object surface using multiple three-dimensional optical probes and a planar target. This system measures the three-dimensional shape of the object's surface by using multiple three-dimensional optical probes and a planar target, solving the problem of rigid connections between vision sensors in a multi-vision sensor measurement system and improving dynamic measurement accuracy. However, this method suffers from complexity and low efficiency when dealing with timing issues related to cameras without a common field of view, especially when the laser tracker in a multi-camera system needs to frequently change locations to adapt to the field environment.

[0010] In summary, the current problems include: when dealing with camera systems without a common field of view, especially when the laser tracker's field of view is obstructed, there is a challenge of coordinate system uniformity, making it impossible to accurately and efficiently obtain camera extrinsic parameters. Furthermore, most camera extrinsic parameter calibration methods still suffer from complex operation and low efficiency. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a method and system for extrinsic parameter calibration of cameras without a common field of view based on a laser tracker. It aims to solve the problem of extrinsic parameter calibration of cameras in multi-camera systems when there is no common field of view and the field of view of the laser tracker is limited, improve calibration efficiency and accuracy, and realize the effective application of multi-camera systems in complex environments.

[0012] The objective of this invention can be achieved through the following technical solutions:

[0013] According to one aspect of the present invention, a method for extrinsic parameter calibration of a camera without a common field of view based on a laser tracker is provided, the method comprising the following steps:

[0014] S1. Deploy the laser tracker and construct and define a global coordinate system based on the laser tracker's measurement reference.

[0015] S2. Select a single camera without a common field of view, and determine whether the current camera and the laser tracker have a common measurement field of view. If they do, proceed to step S5; otherwise, proceed to step S3.

[0016] S3. Move the laser tracker to the preset station position so that it has a common field of view with the current measuring camera, and set up a temporary calibration plate; place the target ball on the target seat in the temporary calibration plate, and use the laser tracker before and after the transfer station to measure the spatial coordinates of the target ball in the temporary calibration plate, and establish the first temporary coordinate system and the second temporary coordinate system.

[0017] S4. Set the first temporary coordinate system and the second temporary coordinate system to the same coordinate system, and use spatial coordinates to calculate the pose transformation relationship between the temporary calibration plate and the global coordinate system.

[0018] S5. Set the calibration plate equipped with the target holder within the common measurement field of view of the camera and the laser tracker; place the target ball on the target holder of the calibration plate, measure the spatial coordinates of the target ball using the laser tracker, and establish the target ball coordinate system; replace the target ball with the photogrammetry-tracker conversion ball, place it on the corresponding target holder of the calibration plate, measure the spatial coordinates of the photogrammetry-tracker conversion ball using the current camera, and establish the conversion ball coordinate system.

[0019] S6. Set the target ball coordinate system and the transformation ball coordinate system to the same coordinate system, and use spatial coordinates to calculate the pose transformation relationship between the current camera and the calibration board and the pose transformation relationship between the calibration board and the global coordinate system.

[0020] S7. Using all the pose transformation relationships obtained from the calculation, calculate the current camera extrinsic parameters;

[0021] S8. Repeat steps S2-S7 until the extrinsic parameter calibration of all single cameras without a common field of view is completed.

[0022] As a preferred technical solution, both the calibration board and the temporary calibration board are equipped with three target holders. The three target holders are non-collinearly distributed, with permanent magnets at the bottom of the target holders and concentric ring-shaped coded gratings engraved on the surface of the target holders. When the camera takes a picture of the conversion ball, the phase information of the grating is extracted by Fourier transform, and the eccentricity of the center of the conversion ball relative to the center of the target holder is calculated. Then, the correction amount is calculated based on the eccentricity, thereby correcting the coordinates of the conversion ball in the camera coordinate system in real time.

[0023] As a preferred technical solution, the correction amount consists of a correction term and an integral term. The correction term is obtained by nonlinearly correcting the eccentricity, and the specific calculation formula for the correction amount is as follows:

[0024] ;

[0025] in, This is a correction amount; It is an eccentric quantity; This is the proportionality coefficient; is the integral coefficient.

[0026] As a preferred technical solution, in S1, multiple reference targets are set within a preset range centered on the laser tracker. Using these reference targets, the origin of the global coordinate system is corrected in real time according to changes in temperature, humidity, and time. The specific formula is as follows:

[0027] ;

[0028] in, The updated origin coordinates; The initial coordinates of the origin of the global coordinate system; The baseline target weight is assigned based on the stability of the target material; Let be the temperature sensitivity coefficient of the i-th reference target; The temperature difference between time t and the initial time; Let be the coordinates of the i-th reference target in the initial global coordinate system; Let be the humidity sensitivity coefficient of the i-th reference target; Let t be the humidity difference between time t and the initial time. This is the mechanical stress relief coefficient; The attenuation factor describes the rate at which the residual mechanical stress of the benchmark target is released. The error compensation term based on wavelet denoising achieves dynamic smoothing by decomposing high-frequency noise.

[0029] As a preferred technical solution, in step S2, when determining whether the camera and the laser tracker share a common measurement field of view, a virtual ray model is constructed. Using the camera's intrinsic parameter matrix and the laser tracker's measurement range parameters, the intersection of the camera's imaging ray and the laser tracker's measured cone is calculated. Let the camera's optical center be... O c The laser tracker measurement center is O t The spatial ray direction vector corresponding to a point on the camera's imaging plane is d p The laser tracker measures the half-angle of the cone as... The existence of a common field of vision is determined by checking whether the following inequality holds:

[0030] ;

[0031] If multiple points on the camera's imaging plane satisfy the above inequality, then it is determined that there is a common measurement field of view.

[0032] As a preferred technical solution, in S4, when calculating the pose transformation relationship between the temporary calibration plate and the global coordinate system, if there is a camera with calibrated extrinsic parameters, the coordinates of the transformation ball in the calibrated camera coordinate system are transformed to the global coordinate system to obtain the first set of pose parameters of the temporary calibration plate in the global coordinate system. Based on the measurement data of the laser tracker after the station transfer, the second set of pose parameters of the temporary calibration plate in the global coordinate system is calculated. The deviation between the first set of pose parameters and the second set of pose parameters is calculated, which is the redundant constraint residual. The Gauss-Helmert adjustment model is used to iteratively optimize the residual.

[0033] As a preferred technical solution, the calculation of the current camera extrinsic parameters in S7 includes two cases:

[0034] When the current camera and laser tracker share a common measurement field of view, the pose transformation relationship includes the pose transformation relationship between the current camera and the calibration board and the pose transformation relationship between the calibration board and the global coordinate system;

[0035] When the camera and laser tracker do not have a common measurement field of view, the pose transformation relationship includes the pose transformation relationship between the temporary calibration board and the global coordinate system, the pose transformation relationship between the current camera and the calibration board, and the pose transformation relationship between the calibration board and the global coordinate system.

[0036] As a preferred technical solution, when the current camera and the laser tracker share a common measurement field of view, the specific formula for calculating the current camera's extrinsic parameters is as follows:

[0037] ;

[0038] ;

[0039] ;

[0040] in, For a single camera and calibration plate The serial number; , and Represent homogeneous pose transformation matrix, rotation matrix and Translation vector; , and Subscript variables ci , ti and w These respectively represent the definition symbols for each individual camera, calibration plate, and laser tracker, i.e., the global coordinate system; such as , and They represent single cameras Relative to global coordinate system The homogeneous pose transformation matrix, rotation matrix, and translation vector.

[0041] As a preferred technical solution, when the camera and laser tracker do not share a common measurement field of view, the specific formula for calculating the extrinsic parameters of the current camera is as follows:

[0042] ;

[0043] ;

[0044] ;

[0045] in, For a single camera and calibration plate The serial number, , and Represent homogeneous pose transformation matrix, rotation matrix and The translation vector, , and Subscript variables cj , tj , temp , w and wt These represent the definition symbols for each individual camera, calibration board, temporary calibration board, laser tracker, and laser tracker after relocation, such as... , , They represent single cameras Relative to global coordinate system The homogeneous pose transformation matrix, rotation matrix, and translation vector.

[0046] According to another aspect of the present invention, a laser tracker-based extrinsic parameter calibration system for cameras without a common field of view is provided. The system includes a laser tracker, a group of cameras to be calibrated, a calibration plate group, a ball group, and a data processing unit.

[0047] Laser trackers are used as global coordinate system reference devices;

[0048] The camera group to be calibrated consists of multiple single cameras without a common field of view;

[0049] The calibration board set includes a calibration board and a temporary calibration board. The calibration board is used for the pose association between the camera and the laser tracker in a common field of view, while the temporary calibration board is used for the coordinate system connection of the laser tracker before and after the station change.

[0050] The ball set includes a target ball and a transition ball;

[0051] The data processing unit is equipped with a calibration algorithm and an error compensation model. This unit is used to complete the pose transformation relationship calculation and camera extrinsic parameter calculation, and output the calibration results.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. In this invention, a global coordinate system is defined based on a laser tracker, and a calibration board is used as a medium. Leveraging the high precision, non-contact, and large-range measurement advantages of the laser tracker, extrinsic parameter calibration between cameras without a common field of view is achieved without relying on a shared field of view, nor requiring the addition of auxiliary cameras or low-precision calibration methods. Furthermore, when the laser tracker's field of view is obstructed, a mechanism of switching stations and using a temporary calibration board is employed to overcome the field-of-view obstruction limitation, ensuring uninterrupted calibration in obstructed scenarios. Multi-camera calibration can be completed simply by iterating through this method, and its standardized process significantly improves efficiency. When dealing with camera systems without a shared field of view, this method can accurately and efficiently acquire camera extrinsic parameters, exhibits strong anti-interference capabilities, and is highly efficient.

[0054] 2. In this invention, both the calibration plate and the temporary calibration plate are equipped with three target holders. The bottom of the target holder is equipped with a permanent magnet, which improves the placement efficiency of the target ball and the conversion ball. Furthermore, the surface of the target holder is engraved with a concentric ring-shaped coded grating. When the camera takes a picture of the conversion ball, the phase information of the grating is extracted by means of Fourier transform, and the eccentricity of the center of the conversion ball relative to the center of the target holder is further calculated. Then, the correction amount is calculated based on the eccentricity, thereby correcting the coordinates of the conversion ball in the camera coordinate system in real time. This provides higher precision basic data for the subsequent calculation of pose relationship, resulting in a smaller overall camera extrinsic parameter calibration error.

[0055] 3. In this invention, multiple reference targets are set within a preset range centered on the laser tracker. A multi-parameter coupled model is established using the reference targets, taking into account the effects of temperature, humidity and stress release, and incorporating wavelet denoising. The origin of the global coordinate system is corrected in real time with changes in temperature, humidity and time, ensuring the long-term consistency of the external parameter calibration of multiple cameras and solving the problem of "large calibration deviation in multiple batches" caused by reference drift in traditional methods.

[0056] 4. In this invention, when calculating the pose transformation relationship between the temporary calibration board and the global coordinate system, if there is a camera with calibrated extrinsic parameters, the coordinates of the transformation ball in the coordinate system of the calibrated camera are transformed to the global coordinate system to obtain the first set of pose parameters of the temporary calibration board in the global coordinate system. Based on the measurement data of the laser tracker after the transfer station, the second set of pose parameters of the temporary calibration board in the global coordinate system is calculated. The deviation between the first set of pose parameters and the second set of pose parameters is calculated, which is the redundant constraint residual. The Gauss-Helmert adjustment model is used to iteratively optimize the residual. The redundant constraint residual is constructed through the dual data sources of the calibrated camera and the tracker after the transfer station, and then iteratively optimized by Gauss-Helmert adjustment to improve the pose calculation accuracy of the temporary calibration board. This provides a high-precision coordinate system connection for the transfer calibration of cameras without a common field of view and ensures the global consistency of multi-station calibration.

[0057] 5. In this invention, a virtual ray model and a cone intersection judgment method are used to quantitatively determine whether the current camera and laser tracker have a common measurement field of view. This improves the level of automation and reduces misjudgments, enabling automated differentiation between two working conditions: "calculating camera extrinsic parameters when there is a common measurement field of view" and "calculating camera extrinsic parameters when there is no common measurement field of view". The method clearly defines the corresponding pose relationships required for the two working conditions, as well as the specific methods for calculating camera extrinsic parameters based on pose relationships for the two working conditions. This makes the calibration method adaptable to all scenarios. Through the common pose relationship chain, the working conditions can be automatically identified and the corresponding calculation logic can be called. It is suitable for complex scenarios, improves the automation rate of the overall calibration process, and reduces errors caused by manual intervention.

[0058] 6. In this invention, the system integrates a laser tracker, a camera group to be calibrated, a calibration board group, a sphere group, and a data processing unit, realizing full automation of the "measurement, calculation, calibration, and output" process. Compared with traditional solutions, it reduces manual data transmission and processing time, shortens the calibration cycle, and improves calibration efficiency. Furthermore, the system has a built-in error compensation model, which improves the accuracy of the calibration results, making it suitable for large-scale camera extrinsic parameter calibration needs. Attached Figure Description

[0059] Figure 1 This is a schematic diagram illustrating the steps of a laser tracker-based method for calibrating the extrinsic parameters of a camera without a common field of view in this invention.

[0060] Figure 2 This is a flowchart of the external parameter calibration process for a camera without a common field of view based on a laser tracker in this invention;

[0061] Figure 3 This is a schematic diagram of the external parameter calibration of a camera without a common field of view based on a laser tracker in this invention;

[0062] Figure 4This is a schematic diagram illustrating the camera extrinsic parameter calibration achieved by the laser tracker relocation station in this invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0064] Photogrammetry, with its advantages of non-contact operation, high precision, and dynamic capabilities, is widely used. Multi-camera systems, compared to single-camera systems, have attracted more attention due to their ability to acquire geometric information of large-volume, complex structures by expanding the field of view. However, in practical applications, multi-camera systems often lack a common field of view or have an extremely small common field of view, necessitating global calibration.

[0065] Therefore, this paper proposes a method and system for extrinsic parameter calibration of cameras without a common field of view based on a laser tracker. It flexibly and accurately realizes the global calibration of multi-camera systems, reduces the hardware costs of auxiliary cameras, and helps multi-camera systems achieve full-field, high-precision, and dynamic monitoring tasks. It has important theoretical research significance and broad application prospects.

[0066] Example 1

[0067] In this embodiment, a laser tracker-based method for extrinsic parameter calibration of cameras without a common field of view is adopted. The basic principle of this method for extrinsic parameter calibration of cameras without a common field of view is as follows:

[0068] First, the number and location of individual cameras are selected by considering factors such as cost and site environment. If a multi-camera system can integrate the coordinate systems of multiple cameras into the global coordinate system through the common field of view between the cameras, then the intrinsic and extrinsic parameters of the cameras can be directly calibrated using a reference ruler. Otherwise, laser trackers should be placed in appropriate locations. , has Single camera: And a calibration plate was placed in sequence Within the measurement field of view of each camera and laser tracker, that is: The calibration board consists of three target mounts and is equipped with a target ball and a photogrammetry-tracker conversion ball. If each calibration board is within the measurement field of view of the camera and tracker, the target ball is placed on the target mount of the calibration board, and the spatial coordinates of the target ball are measured using the tracker to establish... Target sphere coordinate system ( Remove the target ball, place the photogrammetry-tracker conversion ball on the target mount of the calibration plate, and use the camera to measure the spatial coordinates of the conversion ball to establish... Transform spherical coordinate system ( If the target sphere coordinate system and the transformation sphere coordinate system are the same coordinate system, then the single camera coordinates can be calculated. coordinate system to global coordinate system Rotation matrix and translation vector:

[0069] ;

[0070] ;

[0071] ;

[0072] in, For a single camera and calibration plate The serial number, , and Represent homogeneous pose transformation matrix, rotation matrix and The translation vector, , and The subscript variables represent the definition symbols for individual cameras, calibration boards, and laser trackers, such as... , , They represent single cameras Relative to global coordinate system The homogeneous pose transformation matrix, rotation matrix, and translation vector.

[0073] Then, when using a laser tracker to calibrate the extrinsic parameters of a camera without a common field of view, if the tracker's field of view is obstructed by obstacles, a relocation operation is required to calibrate the camera's extrinsic parameters. First, the laser tracker must be moved from its initial position to the selected relocation site, and the temporary calibration board required for the relocation must be set up. The target ball is placed on the target mount in the calibration plate, and the spatial coordinates of the target ball are measured using a tracker at different positions to establish the target ball coordinate system. and Since these two coordinate systems are the same, the single-camera position when the tracker's field of view is obstructed can be calculated. To the global coordinate system Rotation matrix and translation vector:

[0074] ;

[0075] ;

[0076] ;

[0077] in, For a single camera and calibration plate The serial number, , and Represent homogeneous pose transformation matrix, rotation matrix and The translation vector, , and The subscript variables represent the definition symbols for individual cameras, calibration boards, and laser trackers, such as... , , They represent single cameras Relative to global coordinate system The homogeneous pose transformation matrix, rotation matrix, and translation vector.

[0078] After completing the extrinsic parameter calibration between cameras without a common field of view, the extrinsic parameters of the cameras with a common field of view and the intrinsic parameters of each camera are calibrated using a reference ruler. This allows for the calibration of individual cameras in a multi-camera system. ( The measurement results are unified to the global coordinate system. middle.

[0079] In this embodiment, the method is applied to calibrate the extrinsic parameters of a large wind turbine tower using multiple cameras. The specific implementation process is as follows:

[0080] The specific requirements of the scenario are as follows: Six cameras need to be deployed around the wind turbine tower to monitor the tower deformation. Some cameras have no common field of view due to obstruction, so a laser tracker is needed to calibrate the external parameters.

[0081] First, establish and initialize the global coordinate system according to this method. The steps are as follows: Figure 1 As shown, it specifically includes:

[0082] S1. Deploy the laser tracker and construct and define a global coordinate system based on the laser tracker's measurement reference.

[0083] S2. Select a single camera without a common field of view, and determine whether the current camera and the laser tracker have a common measurement field of view. If they do, proceed to step S5; otherwise, proceed to step S3.

[0084] S3. Move the laser tracker to the preset station position so that it has a common field of view with the current measuring camera, and set up a temporary calibration plate; place the target ball on the target seat in the temporary calibration plate, and use the laser tracker before and after the transfer station to measure the spatial coordinates of the target ball in the temporary calibration plate, and establish the first temporary coordinate system and the second temporary coordinate system.

[0085] S4. Set the first temporary coordinate system and the second temporary coordinate system to the same coordinate system, and use spatial coordinates to calculate the pose transformation relationship between the temporary calibration plate and the global coordinate system.

[0086] S5. Set the calibration plate equipped with the target holder within the common measurement field of view of the camera and the laser tracker; place the target ball on the target holder of the calibration plate, measure the spatial coordinates of the target ball using the laser tracker, and establish the target ball coordinate system; replace the target ball with the photogrammetry-tracker conversion ball, place it on the corresponding target holder of the calibration plate, measure the spatial coordinates of the photogrammetry-tracker conversion ball using the current camera, and establish the conversion ball coordinate system.

[0087] S6. Set the target ball coordinate system and the transformation ball coordinate system to the same coordinate system, and use spatial coordinates to calculate the pose transformation relationship between the current camera and the calibration board and the pose transformation relationship between the calibration board and the global coordinate system.

[0088] S7. Using all the pose transformation relationships obtained from the calculation, calculate the current camera extrinsic parameters;

[0089] S8. Repeat steps S2-S7 until the extrinsic parameter calibration of all single cameras without a common field of view is completed.

[0090] First, the equipment will be arranged:

[0091] A laser tracker is installed at the bottom of the tower. A global coordinate system G is defined with the tracker's measurement center as the reference. Within a 3-meter radius of the tracker, three ceramic reference targets and two metal reference targets are arranged to correct the global origin drift in real time. Using these reference targets, the origin of the global coordinate system is corrected in real time according to changes in temperature, humidity, and time.

[0092] Next, determine the common field of view and group the cameras:

[0093] Six cameras were deployed at different heights (20m, 40m, 60m) on the tower. Some cameras lacked a common field of view due to obstruction by the tower flange. A virtual ray model was constructed to calculate the intersection of the camera's imaging ray with the cone measured by the laser tracker (half an angle). To determine whether the current camera and laser tracker share a common field of view, the intersection of the camera's imaging ray with the cone measured by the laser tracker was calculated using the camera's intrinsic parameter matrix and the laser tracker's measurement range parameters, based on the constructed virtual ray model. The optical center of the camera was set to... Oc The laser tracker measurement center is O t The spatial ray direction vector corresponding to a point on the camera's imaging plane is d p The laser tracker measures the half-angle of the cone as... The existence of a common field of vision is determined by checking whether the following inequality holds:

[0094] ;

[0095] If multiple points on the camera's imaging plane satisfy the above inequality, then it is determined that there is a common measurement field of view.

[0096] Then, camera calibration without a common field of view is performed:

[0097] Both the calibration board and the temporary calibration board are equipped with three target mounts, which are non-collinearly distributed. A permanent magnet is located at the bottom of each target mount, and concentric ring-shaped coded gratings are engraved on the surface of each target mount. When the camera captures the conversion sphere, Fourier transform is used to extract the grating phase information, and the eccentricity of the conversion sphere's center relative to the target mount center is further calculated. A correction is then calculated based on this eccentricity, thereby correcting the conversion sphere's coordinates in the camera coordinate system in real time. The correction consists of a correction term and an integral term. The correction term is obtained by nonlinearly correcting the eccentricity. The specific formula for calculating the correction is as follows:

[0098] ;

[0099] in, This is a correction amount; It is an eccentric quantity; This is the proportionality coefficient; is the integral coefficient.

[0100] During the relocation, the laser tracker was moved to the 20m platform on the tower (pre-set position), and a temporary calibration plate was set up. The target sphere coordinates were measured by the tracker before the relocation to establish a first temporary coordinate system; the target sphere coordinates were measured after the relocation to establish a second temporary coordinate system. The first and second temporary coordinate systems were unified by Gauss-Helmert adjustment (residual ≤ 0.5μm), and the pose of the laser tracker relative to the global coordinate system after the relocation was calculated.

[0101] For cameras without a common field of view, a calibration board is set up in the common field of view. The target sphere is measured to establish the target sphere coordinate system, and the transformation sphere is measured to establish the transformation sphere coordinate system. The eccentricity is corrected by coded grating, and the pose of the camera without a common field of view and the target sphere coordinate system is calculated. By combining the pose of the target sphere coordinate system and the global coordinate system, the calibration of the current camera without a common field of view is finally completed.

[0102] In S4, when calculating the pose transformation relationship between the temporary calibration plate and the global coordinate system, if there is a camera with calibrated extrinsic parameters, the coordinates of the transformation ball in the calibrated camera coordinate system are transformed to the global coordinate system to obtain the first set of pose parameters of the temporary calibration plate in the global coordinate system. Based on the measurement data of the laser tracker after the station transfer, the second set of pose parameters of the temporary calibration plate in the global coordinate system is calculated. The deviation between the first set of pose parameters and the second set of pose parameters is calculated, which is the redundant constraint residual. The Gauss-Helmert adjustment model is used to iteratively optimize the residual.

[0103] The current camera extrinsic parameters calculated in S7 include two cases:

[0104] When the camera and laser tracker share a common measurement field of view, the calibration diagram is as follows: Figure 3 As shown, the pose transformation relationship at this time includes the pose transformation relationship between the current camera and the calibration board and the pose transformation relationship between the calibration board and the global coordinate system;

[0105] When the camera and laser tracker do not share a common measurement field of view, the calibration diagram is as follows: Figure 4 As shown, at this time, the pose transformation relationship includes the pose transformation relationship between the temporary calibration board and the global coordinate system, the pose transformation relationship between the current camera and the calibration board, and the pose transformation relationship between the calibration board and the global coordinate system.

[0106] When the current camera and laser tracker share a common measurement field of view, the specific formula for calculating the current camera's extrinsic parameters is as follows:

[0107] ;

[0108] ;

[0109] ;

[0110] in, For a single camera and calibration plate The serial number; , and Represent homogeneous pose transformation matrix, rotation matrix and Translation vector; , and Subscript variables ci , ti and w These respectively represent the definition symbols for each individual camera, calibration plate, and laser tracker, i.e., the global coordinate system; such as , and They represent single cameras Relative to global coordinate system The homogeneous pose transformation matrix, rotation matrix, and translation vector.

[0111] When the current camera and laser tracker do not share a common field of view, the specific formula for calculating the current camera's extrinsic parameters is as follows:

[0112] ;

[0113] ;

[0114] ;

[0115] in, For a single camera and calibration plate The serial number, , and Represent homogeneous pose transformation matrix, rotation matrix and The translation vector, , and Subscript variables cj , tj , temp , w and wt These represent the definition symbols for each individual camera, calibration board, temporary calibration board, laser tracker, and laser tracker after relocation, such as... , , They represent single cameras Relative to global coordinate system The homogeneous pose transformation matrix, rotation matrix, and translation vector.

[0116] Finally, perform full system calibration and verification:

[0117] For calibration with a camera sharing a common field of view, the camera's intrinsic and extrinsic parameters are directly calibrated using a reference ruler. For calibration without a camera sharing a common field of view, after completing the extrinsic parameter calibration based on the above steps, the camera's intrinsic parameters are calibrated using a reference ruler, thus completing the full system calibration. Finally, a global verification is performed, which involves measuring the tower's characteristic points using a reference ruler to verify the consistency of the extrinsic parameters of the multiple cameras (coordinate deviation ≤ 2μm), thus completing the camera parameter calibration.

[0118] In summary, this method leverages the high precision, non-contact, and wide-range measurement advantages of laser trackers. It eliminates the need for a shared field of view, auxiliary cameras, or low-precision calibration methods to achieve extrinsic parameter calibration between cameras without a shared field of view. The laser tracker can accurately acquire the spatial location information of measurement points over a large area. Through a specific calibration process and algorithm, the coordinate systems of different cameras are correlated with the coordinate system of the laser tracker. This avoids the dependence on a shared field of view and the need for additional hardware or low-precision calibration methods, enabling flexible and high-precision extrinsic parameter calibration. This, in turn, achieves global calibration of multi-camera systems, reduces hardware costs such as cameras, and helps multi-camera systems achieve full-field, high-precision, and dynamic monitoring tasks.

[0119] Example 2

[0120] In this embodiment, a laser tracker-based camera extrinsic parameter calibration system without a common field of view is adopted. The system includes a laser tracker, a camera group to be calibrated, a calibration board group, a ball group, and a data processing unit.

[0121] The laser tracker serves as a global coordinate system reference device; the camera group to be calibrated consists of multiple single cameras without a common field of view; the calibration board group includes a calibration board and a temporary calibration board, wherein the calibration board is used for the pose association between the camera and the laser tracker under a common field of view, and the temporary calibration board is used for the coordinate system connection before and after the laser tracker changes stations; the sphere group includes a target sphere and a transformation sphere; the data processing unit is equipped with a calibration algorithm to complete the pose transformation relationship calculation and camera extrinsic parameter calculation, and output the calibration results.

[0122] The specific process for calibrating and monitoring the parameters of a multi-camera system for synchronous lifting of large structures using this system is as follows: Figure 2 As shown, the details are as follows:

[0123] (1) Use finite element analysis and other techniques to perform dynamic stress analysis on large structures and simulate their stress conditions under actual operating conditions. Accurately identify stress concentration areas, areas with drastic stress changes, and areas that have been subjected to large stresses for a long time, and determine these areas as key areas to be monitored;

[0124] (2) Taking into account cost, on-site construction environment and distribution of the area to be monitored, select an appropriate number of cameras and determine the camera positions;

[0125] (3) Locate cameras in the multi-camera system that have no common field of view or a small common field of view with other cameras, and select the location of the laser tracker based on the location of these cameras. Then, place the calibration plate within the measurement range of these cameras and the laser tracker;

[0126] (4) Using a camera and laser tracker at the selected location, measure the spatial coordinates of the photogrammetry-tracker conversion ball and the target ball on the calibration board respectively;

[0127] (5) Based on the acquired spatial coordinate data, calculate the rotation matrix and translation vector from the camera coordinate system to the global coordinate system, and determine the position and attitude of the camera in the global coordinate system;

[0128] (6) If there are obstacles blocking the field of view of the laser tracker, the tracker needs to be moved and a common calibration board needs to be set up for tracker relocation operation, and the rotation matrix and translation vector between trackers should be calibrated at the same time.

[0129] (7) Repeat the two steps of spatial coordinate measurement (similar to step 4) and camera position and attitude calculation (similar to step 5) to further calculate the rotation matrix and translation vector from the camera coordinate system to the global coordinate system in order to improve accuracy or adapt to environmental changes and other requirements.

[0130] (8) After completing the extrinsic calibration of the cameras without a common field of view in the multi-camera system, use the reference ruler to calibrate the extrinsic parameters of other cameras and the intrinsic parameters of all cameras, and finally achieve the global calibration of the multi-camera system.

[0131] (9) The multi-camera system performs full-field, high-precision, and dynamic measurement of the dynamic data of all areas to be measured in the global coordinate system during the synchronous lifting of large structures.

[0132] In this practical example, the calibration system is applied to calibrate the parameters of a multi-camera system in a smart factory. The specific implementation process is as follows: In this embodiment, 12 cameras are deployed on the smart factory production line to monitor robot movement. Some cameras lack a common field of view due to obstruction by production line equipment, requiring real-time online calibration of extrinsic parameters. A laser tracker is deployed in the center of the production line as a global coordinate system reference, supporting rapid station relocation. The camera group to be calibrated consists of 12 industrial cameras distributed around the production line, with some cameras lacking a common field of view. The sphere group comprises a ceramic target sphere and a conversion sphere with dual-band surface features. The data processing unit uses an industrial control computer equipped with a calibration algorithm, supporting real-time calculation.

[0133] The system workflow is as follows: First, the global coordinate system is initialized. The global origin is corrected in real time using the tracker and five reference targets to ensure long-term stability. Next, a common field of view is dynamically determined. The intersection of the camera's and tracker's fields of view is calculated in real time, automatically grouping cameras with and without a common field of view. For cameras with a common field of view, extrinsic parameters are calculated in real time using a calibration board. For cameras without a common field of view, the tracker is relocated, and the coordinate system is connected using a temporary calibration board, extrinsic parameters are calculated and updated synchronously. Finally, accuracy verification and feedback are performed using the end-effector feature points of the production line robot to verify the consistency of multi-camera extrinsic parameters, achieving closed-loop calibration.

[0134] This system has strong dynamic adaptability, good real-time performance, and strong robustness. It supports automatic re-judgment of the common field of view and triggering calibration when the "field of view changes" caused by the movement of production line equipment (such as robot handling). Moreover, the pose calculation delay of the data processing unit is ≤20ms, which meets the requirements of online monitoring. By integrating coded grating eccentricity correction and Gauss-Helmert adjustment, the calibration accuracy can still be maintained at ≤5μm under the vibration environment of the production line.

[0135] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for extrinsic parameter calibration of a camera without a common field of view based on a laser tracker, characterized in that, The method steps include: S1. Deploy the laser tracker and construct and define a global coordinate system based on the laser tracker's measurement reference. S2. Select a single camera without a common field of view, and determine whether the current camera and the laser tracker have a common measurement field of view. If they do, proceed to step S5; otherwise, proceed to step S3. S3. Move the laser tracker to the preset station position so that it has a common field of view with the current measuring camera, and set up a temporary calibration plate; place the target ball on the target seat in the temporary calibration plate, and use the laser tracker before and after the transfer station to measure the spatial coordinates of the target ball in the temporary calibration plate, and establish the first temporary coordinate system and the second temporary coordinate system. S4. Set the first temporary coordinate system and the second temporary coordinate system to the same coordinate system, and use spatial coordinates to calculate the pose transformation relationship between the temporary calibration plate and the global coordinate system. S5. Set the calibration plate equipped with the target holder within the common measurement field of view of the camera and the laser tracker; place the target ball on the target holder of the calibration plate, measure the spatial coordinates of the target ball using the laser tracker, and establish the target ball coordinate system; replace the target ball with the photogrammetry-tracker conversion ball, place it on the corresponding target holder of the calibration plate, measure the spatial coordinates of the photogrammetry-tracker conversion ball using the current camera, and establish the conversion ball coordinate system. S6. Set the target ball coordinate system and the transformation ball coordinate system to the same coordinate system, and use spatial coordinates to calculate the pose transformation relationship between the current camera and the calibration board and the pose transformation relationship between the calibration board and the global coordinate system. S7. Using all the pose transformation relationships obtained from the calculation, calculate the current camera extrinsic parameters; S8. Repeat steps S2-S7 until the extrinsic parameter calibration of all single cameras without a common field of view is completed.

2. The method for calibrating extrinsic parameters of a camera without a common field of view based on a laser tracker according to claim 1, characterized in that, Both the calibration plate and the temporary calibration plate are equipped with three target holders, which are non-collinearly distributed. The bottom of each target holder is provided with a permanent magnet, and the surface of each target holder is engraved with a concentric ring-shaped coded grating. When the camera captures the conversion sphere, the phase information of the grating is extracted by means of Fourier transform, and the eccentricity of the center of the conversion sphere relative to the center of the target holder is further calculated. Then, the correction amount is calculated based on the eccentricity, thereby correcting the coordinates of the conversion sphere in the camera coordinate system in real time.

3. The method for extrinsic parameter calibration of a camera without a common field of view based on a laser tracker according to claim 2, characterized in that, The correction amount consists of a correction term and an integral term, wherein the correction term is obtained by nonlinear correction of the eccentricity, and the specific calculation formula for the correction amount is as follows: ； in, This is a correction amount; It is an eccentric quantity; This is the proportionality coefficient; is the integral coefficient.

4. The method for extrinsic parameter calibration of a camera without a common field of view based on a laser tracker according to claim 1, characterized in that, In step S1, multiple reference targets are set within a preset range centered on the laser tracker. Using these reference targets, the origin of the global coordinate system is corrected in real time according to changes in temperature, humidity, and time. The specific formula is as follows: ； in, The updated origin coordinates; The initial coordinates of the origin of the global coordinate system; The baseline target weight is assigned based on the stability of the target material; Let be the temperature sensitivity coefficient of the i-th reference target; The temperature difference between time t and the initial time; Let be the coordinates of the i-th reference target in the initial global coordinate system; Let be the humidity sensitivity coefficient of the i-th reference target; Let t be the humidity difference between time t and the initial time. This is the mechanical stress relief coefficient; The attenuation factor describes the rate at which the residual mechanical stress of the benchmark target is released. The error compensation term based on wavelet denoising achieves dynamic smoothing by decomposing high-frequency noise.

5. The method for extrinsic parameter calibration of a camera without a common field of view based on a laser tracker according to claim 1, characterized in that, In step S2, when determining whether the current camera and laser tracker share a common measurement field of view, a virtual ray model is constructed. Using the camera's intrinsic parameter matrix and the laser tracker's measurement range parameters, the intersection of the camera's imaging ray and the laser tracker's measured cone is calculated. Let the camera's optical center be... O c The laser tracker measurement center is O t The spatial ray direction vector corresponding to a point on the camera's imaging plane is d p The laser tracker measures the half-angle of the cone as... The existence of a common field of vision is determined by checking whether the following inequality holds: ； If multiple points on the camera's imaging plane satisfy the above inequality, then it is determined that there is a common measurement field of view.

6. The method for extrinsic parameter calibration of a camera without a common field of view based on a laser tracker according to claim 1, characterized in that, In step S4, when calculating the pose transformation relationship between the temporary calibration plate and the global coordinate system, if there is a camera with calibrated extrinsic parameters, the coordinates of the transformation ball in the calibrated camera coordinate system are transformed to the global coordinate system to obtain the first set of pose parameters of the temporary calibration plate in the global coordinate system. Based on the measurement data of the laser tracker after the station transfer, the second set of pose parameters of the temporary calibration plate in the global coordinate system is calculated. The deviation between the first set of pose parameters and the second set of pose parameters is calculated, which is the redundant constraint residual. The Gauss-Helmert adjustment model is used to iteratively optimize the residual.

7. The method for extrinsic parameter calibration of a camera without a common field of view based on a laser tracker according to claim 1, characterized in that, The calculation of the current camera extrinsic parameters in S7 includes two cases: When the current camera and laser tracker share a common measurement field of view, the pose transformation relationship includes the pose transformation relationship between the current camera and the calibration board and the pose transformation relationship between the calibration board and the global coordinate system; When the camera and laser tracker do not have a common measurement field of view, the pose transformation relationship includes the pose transformation relationship between the temporary calibration board and the global coordinate system, the pose transformation relationship between the current camera and the calibration board, and the pose transformation relationship between the calibration board and the global coordinate system.

8. The method for extrinsic parameter calibration of a camera without a common field of view based on a laser tracker according to claim 7, characterized in that, When the current camera and the laser tracker share a common measurement field of view, the specific formula for calculating the current camera's extrinsic parameters is as follows: ； ； ； in, For a single camera and calibration plate The serial number; , and Represent homogeneous pose transformation matrix, rotation matrix and Translation vector; , and Subscript variables ci , ti and w These respectively represent the definition symbols for each individual camera, calibration plate, and laser tracker, i.e., the global coordinate system; such as , and They represent single cameras Relative to global coordinate system The homogeneous pose transformation matrix, rotation matrix, and translation vector.

9. A method for calibrating extrinsic parameters of a camera without a common field of view based on a laser tracker according to claim 7, characterized in that, When the current camera and the laser tracker do not share a common measurement field of view, the specific formula for calculating the current camera's extrinsic parameters is as follows: ； ； ； in, For a single camera and calibration plate The serial number, , and Represent homogeneous pose transformation matrix, rotation matrix and The translation vector, , and Subscript variables cj , tj , temp , w and wt These represent the definition symbols for each individual camera, calibration board, temporary calibration board, laser tracker, and laser tracker after relocation, such as... , , They represent single cameras Relative to global coordinate system The homogeneous pose transformation matrix, rotation matrix, and translation vector.

10. A laser tracker-based extrinsic parameter calibration system for cameras without a common field of view, characterized in that, The system operates using a laser tracker-based extrinsic parameter calibration method for cameras without a common field of view, as described in any one of claims 1-9. The system includes a laser tracker, a group of cameras to be calibrated, a calibration board group, a ball group, and a data processing unit. The laser tracker is used as a global coordinate system reference device; The camera group to be calibrated consists of multiple single cameras without a common field of view; The calibration board set includes a calibration board and a temporary calibration board. The calibration board is used for the pose association between the camera and the laser tracker in a common field of view, and the temporary calibration board is used for the coordinate system connection of the laser tracker before and after the station change. The ball set includes a target ball and a transition ball; The data processing unit is equipped with a calibration algorithm and an error compensation model. This unit is used to complete the pose transformation relationship calculation and camera extrinsic parameter calculation, and output the calibration results.

Images