Non-overlapping-area multi-camera attitude calibration method and device
By vertically suspending a checkerboard pattern in front of the camera and utilizing the direction of gravity, images are independently acquired and gravity projection vectors are constructed, solving the calibration problem of multi-camera systems with non-overlapping fields of view. This achieves low-cost, high-precision extrinsic parameter calibration, suitable for industrial sites and robotic systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WUXI A CARRIER INTELLIGENT EQUIP
- Filing Date
- 2026-01-20
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies for multi-camera system calibration rely on additional hardware or mechanical devices under non-overlapping field of view conditions, resulting in high costs, complex deployment, and poor versatility. They cannot achieve fast, low-cost, and highly versatile extrinsic parameter calibration in scenarios without inertial assistance.
By suspending a checkerboard pattern vertically in front of the camera, using the direction of gravity as a global physical prior, images are independently acquired and gravity projection vectors are constructed. Combining singular value decomposition and error minimization models, the relative rotation relationship between cameras is solved, achieving calibration without common view, synchronization, or auxiliary sensors.
It enables multi-camera extrinsic parameter calibration without the need for shared viewing area or synchronization signals, reducing system costs, simplifying hardware deployment, and improving calibration accuracy and robustness, making it suitable for rapid deployment of distributed camera systems.
Smart Images
Figure CN122066784A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a method and apparatus for multi-camera pose calibration in non-overlapping regions. Background Technology
[0002] In scenarios such as large-scale industrial measurement, large component docking and orientation adjustment, intelligent robot navigation, and vehicle surround view, monocular cameras are limited by their field of view and cannot independently complete visual perception tasks in complex spaces. Therefore, multi-camera systems are widely used. To achieve the fusion of multi-source image data and unified coordinate representation, it is necessary to calibrate the external parameters of each camera, that is, to accurately solve the relative pose transformation relationship between any two cameras. However, when there is no overlap in the field of view between cameras, traditional calibration methods based on common-view feature points fail. Currently, the mainstream non-overlapping calibration schemes mainly include three categories: first, using optical devices such as mirrors and prisms to expand the effective field of view to construct a virtual common-view region; second, introducing additional auxiliary cameras to simultaneously observe multiple target cameras or calibration plates to establish cross-view correlation; and third, using rigid fasteners to fix two cameras or two calibration plates together to form a known pose constraint, which is then transformed into the classical AX=XB equation for solution. However, the above methods have significant limitations in practical applications: in large assembly sites, such as aircraft segment docking and wind turbine blade inspection, it is often impossible to install reflectors or auxiliary cameras, and it is also difficult to rigidly fix the camera or calibration object; in addition, these solutions usually rely on precision mechanical devices or additional hardware, resulting in high system costs, complex deployment, and poor versatility.
[0003] To address these challenges, existing research has attempted to introduce external physical references or optimize acquisition strategies. For example, CN114842090B discloses a visual-inertial calibration system based on a precise angular reference. This system uses a high-precision turntable to provide a known rotation angle and combines it with an IMU to measure the direction of gravity, achieving joint calibration of the camera and inertial unit. While this scheme utilizes gravity information, it is highly dependent on expensive turntable equipment and IMU sensors, making it unsuitable for purely visual scenarios without inertial assistance. CN115205399A proposes a non-co-view multi-view camera calibration method. This method moves a single calibration board at multiple locations and records its pose, then uses the pose chain transitivity to solve for the extrinsic parameters between cameras. However, this method requires that the pose of the calibration board at different locations be accurately solved using PnP with controllable error. Essentially, it still relies on the spatial motion consistency of the calibration board itself and does not utilize global physical priors in the environment. CN115248093A discloses a multi-view camera-based method for measuring the center of mass using both static and dynamic perspectives. This method calibrates the object by analyzing the consistency of its center of mass projection across multiple viewpoints. However, it is only applicable to specific targets with known center of mass characteristics, limiting its versatility. Furthermore, it does not address the implicit dependence on time synchronization or common-view calibration during the calibration process. In summary, existing technologies have failed to achieve fully autonomous, low-cost, and highly versatile non-overlapping multi-camera attitude calibration without introducing additional sensors, relying on mechanical constraints, or assuming a motion model for the calibration object, solely utilizing the geometric relationship between a checkerboard pattern and the direction of gravity in a static environment. Therefore, there is an urgent need for a non-overlapping multi-camera attitude calibration method that requires no common-view, no auxiliary equipment, no rigid constraints, and can be rapidly deployed. Summary of the Invention
[0004] In view of this, the present invention proposes a multi-camera attitude calibration method and device for non-overlapping regions. By utilizing the physical prior of the vertically suspended checkerboard grid and the direction of gravity, gravity projection vectors are extracted from images independently acquired by each camera, constructing a multi-camera relative attitude calibration method that does not require shared view, synchronization, or auxiliary sensors, thus achieving fast, low-cost, and highly versatile extrinsic parameter calibration.
[0005] This invention provides a method for multi-camera pose calibration in non-overlapping regions, comprising the following steps: S1 suspends a planar checkerboard grid vertically in front of each camera to be calibrated; under static conditions, image data of each camera is collected at at least two different positions. S2 calculates the pose transformation matrix of each camera coordinate system relative to the corresponding checkerboard coordinate system based on the image data; Based on the characteristic that the chessboard is always vertically suspended, S3 determines the unit vector representing the direction of gravity in its coordinate system, and uses the attitude transformation matrix to project the unit vector of gravity direction onto the coordinate system of each camera to obtain the gravity direction projection coordinates of each camera at multiple positions. S4 constructs an error minimization model and a cross-correlation matrix for any two cameras to be calibrated, based on their respective gravity direction projection coordinates. Then, it performs singular value decomposition on the cross-correlation matrix to solve for a rotation matrix that satisfies the orthogonality and determinant constraints of +1, thereby obtaining the relative rotation relationship between the two cameras.
[0006] Furthermore, the error minimization model is as follows:
[0007] And satisfy the constraints:
[0008] in, For the i-th observation position, and These are the gravity direction projection coordinates in the coordinate systems of the two cameras at the i-th position, respectively; Indicates that the camera Vector transformation in coordinate system to camera Rotation matrix in coordinate system; N is the number of observation positions, and N≥2; , The coordinate systems representing the individual squares of the chessboard. , The coordinate system for the camera corresponding to the chessboard grid; It is a 3×3 identity matrix. The determinant of a matrix is used to ensure... It is a true rotation matrix.
[0009] Furthermore, the cross-correlation matrix is constructed as follows:
[0010] Perform singular value decomposition on the cross-correlation matrix to obtain ,in, and It is an orthogonal matrix. It is a singular value diagonal matrix; Construct the corrected diagonal matrix:
[0011] Calculate the rotation matrix:
[0012] Where H is the cross-correlation matrix, U is the left singular matrix, and V is the right singular matrix. This is a constructor for diagonal matrices.
[0013] Furthermore, the unit vector of the gravitational direction is defined in the chessboard coordinate system as... The corresponding chessboard grid's X-axis points downwards along the direction of gravity.
[0014] Furthermore, the attitude transformation matrix is obtained by solving the correspondence between the image coordinates and three-dimensional coordinates of the chessboard corner points using the PnP algorithm.
[0015] Furthermore, the gravity direction projection coordinates Calculated using the following formula:
[0016] in, For the i-th observation position, the camera Coordinate system at this location to checkerboard coordinate system The rotation part of the attitude transformation matrix; chessboard coordinate system The unit vector representing the direction of gravity; For the direction of gravity at the camera Projected coordinates in the coordinate system.
[0017] Furthermore, each camera operates independently during image acquisition, without synchronous signal transmission between them, and the image acquisition time of each camera is independently set according to its own triggering mechanism. The sequences of images acquired by each camera are not temporally correlated.
[0018] Furthermore, if a candidate solution of the attitude matrix obtained through singular value decomposition causes an abrupt or discontinuous change in the direction of gravity in the reference coordinate system, the candidate solution is excluded.
[0019] Furthermore, the present invention also proposes a multi-camera pose calibration device for non-overlapping regions, comprising: Multiple cameras are used to capture images of a planar checkerboard pattern that is vertically suspended in front of each camera from different positions. Memory, which stores computer programs; The processor is configured to execute the above method to calculate the relative rotation relationship between any two cameras.
[0020] The present invention has the following advantages over the prior art: This invention utilizes gravity as a global physical prior by suspending the checkerboard grid vertically and aligning its plane normal vector with the direction of gravity. This allows each camera to establish a unified orientation reference even without shared view, thereby eliminating the dependence on shared view areas or external calibration devices and significantly increasing the applicable scenarios of the calibration method.
[0021] Since each camera only needs to independently acquire images at different locations in a static environment, without the need for time synchronization or communication connections between them, it avoids the dependence on high-precision triggering systems or synchronization signals in traditional multi-camera calibration, reduces the complexity of hardware deployment, and is suitable for the rapid deployment of distributed or heterogeneous camera systems.
[0022] By extracting the projection vector of the gravity direction in each camera coordinate system from the PnP solution results and constructing a cross-camera attitude constraint model based on this vector, the external parameter calibration process is completely driven by visual information, without the need to introduce an inertial measurement unit (IMU), lidar or other auxiliary sensors, which effectively saves system costs and simplifies the calibration process.
[0023] The error minimization model adopted combines orthogonality and the constraint of determinant being 1, and solves the rotation matrix analytically through singular value decomposition (SVD). This not only ensures the mathematical rationality of the solution (true rotation), but also improves the numerical stability and calibration accuracy of attitude estimation.
[0024] The entire calibration process requires only a standard checkerboard grid that can be vertically suspended and a conventional camera. It can be completed in any location without the need for precision mechanical structures, fixed supports, or motion platforms, which greatly shortens the calibration preparation and execution time. It is particularly suitable for the rapid external parameter calibration needs in industrial sites where cameras are frequently changed or in mass production lines. Attached Figure Description
[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0026] Figure 1 This is a schematic diagram of a multi-camera pose calibration method for non-overlapping regions according to an embodiment of the present invention. Detailed Implementation
[0027] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0028] like Figure 1 The figure shows a schematic diagram illustrating the principle of the non-overlapping field-of-view multi-camera pose calibration method of the present invention. As shown, two cameras... and Located at different positions, their fields of view do not overlap, making it impossible to establish a geometric relationship through shared feature points. A checkerboard calibration board is placed in front of each camera, marked as follows: and The chessboard grid is suspended vertically in a static manner, with its surface normal vector aligned with the direction of gravity. The figure clearly shows this in the chessboard coordinate system. and In the diagram, the X-axis points vertically downwards and coincides with the labeled gravity direction arrow, indicating that this scheme defines the X-axis of the checkerboard coordinate system as the gravity direction; the X and Y axes lie within the checkerboard plane, forming a right-handed coordinate system. (Camera) and Static images of the chessboard are acquired at their respective independent locations (e.g., location 1, location 2). The rotation relationship from the chessboard coordinate system to the camera coordinate system at each location is calculated using the PnP algorithm, and the gravity unit vector in the chessboard coordinate system is then calculated. Projecting the data onto the coordinate systems of each camera, and then constructing a cross-camera attitude constraint model based on the consistency of the gravity projection vectors, finally solves for the relative rotation matrix between the two cameras. This figure visually illustrates the core idea of this invention: multi-camera extrinsic parameter calibration can be achieved without the need for shared viewing areas or time synchronization, relying solely on prior physical knowledge of the gravity direction.
[0029] In one embodiment, the present invention provides a method and apparatus for multi-camera pose calibration in non-overlapping regions, comprising the following steps: S1 suspends a planar checkerboard grid vertically in front of each camera to be calibrated; under static conditions, image data of each camera is collected at at least two different positions. Each camera operates independently during image acquisition, without synchronous signal transmission between them. Furthermore, the image acquisition time of each camera is set independently according to its own triggering mechanism, and the sequences of images acquired by each camera are not temporally correlated.
[0030] Specifically, this embodiment of the invention uses two cameras as an example. The planar checkerboard pattern is a standard checkerboard pattern, vertically suspended in the air by thin ropes or a hanger, keeping its surface vertical. Its coordinate system is defined as follows: the X-axis points vertically downwards along the direction of gravity, the Y-axis lies in the checkerboard plane and points horizontally to the right, and the Z-axis is perpendicular to the checkerboard plane and points towards the camera, forming a right-handed coordinate system. Since the checkerboard is always vertically suspended, its normal vector (Z-axis) naturally points horizontally, ensuring its stability without additional attitude feedback or external physical constraints. In a static environment, the checkerboard is moved to at least two different spatial positions, such as position 1 and position 2. After each stationary period, each camera independently acquires image data. The acquisition process does not require time synchronization or simultaneous shooting; it only needs to ensure that each image contains sufficiently clear checkerboard corner points for subsequent processing.
[0031] Based on the image data, S2 calculates the attitude transformation matrix of each camera coordinate system relative to the corresponding chessboard coordinate system; the attitude transformation matrix is obtained by solving the correspondence between the image coordinates and three-dimensional coordinates of the chessboard corner points using the PnP algorithm.
[0032] Specifically, during the data acquisition process, the calibration plate remains stationary relative to the direction of the gravitational field, and for each observation position... (i=1,2,…,N, and N≥2), using the PnP algorithm (such as solvePnP) in computer vision libraries such as OpenCV, input the 3D corner coordinates of the chessboard and their 2D projection coordinates in the image, and calculate the coordinates from the chessboard coordinate system. To the camera coordinate system The attitude transformation relationship is denoted as , where j = 1, 2, ... represents different cameras. This transformation includes a rotation matrix. Translation vector The rotating part Used for subsequent gravity direction projection calculations.
[0033] Based on the characteristic that the chessboard is always vertically suspended, S3 determines the unit vector representing the direction of gravity in its coordinate system, and uses the attitude transformation matrix to project the unit vector of gravity direction onto the coordinate system of each camera to obtain the gravity direction projection coordinates of each camera at multiple positions. The unit vector of the direction of gravity is defined in the chessboard coordinate system as follows: The corresponding chessboard grid's X-axis points downwards along the direction of gravity.
[0034] Specifically, since the chessboard grid is always suspended vertically, its X-axis direction is consistent with the direction of gravity. Therefore, at any position, the unit vector of the direction of gravity in the chessboard grid coordinate system is always [value missing]. Combining the rotation part of the attitude transformation matrix obtained in step S2 This unit vector can be projected onto the camera coordinate system to obtain the projected coordinates of the gravity direction in the camera coordinate system:
[0035] in, For the i-th observation position, the camera The projection vector of the gravity direction in the coordinate system has a length of 1, and its direction represents the observed direction of gravity from the camera's perspective.
[0036] S4 constructs an error minimization model and a cross-correlation matrix for any two cameras to be calibrated, based on their respective gravity direction projection coordinates. Then, it performs singular value decomposition on the cross-correlation matrix to solve for a rotation matrix that satisfies the orthogonality and determinant constraints of +1, thereby obtaining the relative rotation relationship between the two cameras.
[0037] The error minimization model:
[0038] And satisfy the constraints:
[0039] in, For the i-th observation position, and These are the gravity direction projection coordinates in the coordinate systems of the two cameras at the i-th position, respectively; Indicates that the camera Vector transformation in coordinate system to camera Rotation matrix in coordinate system; N is the number of observation positions, and N≥2; , The coordinate systems representing the individual squares of the chessboard. , The coordinate system for the camera corresponding to the chessboard grid; It is a 3×3 identity matrix. The determinant of a matrix is used to ensure... It is a true rotation matrix.
[0040] To avoid numerical instability caused by direct optimization, a method of constructing a cross-correlation matrix is used to solve for the optimal rotation. First, based on the gravity projection vectors at all observation locations, the original cross-correlation matrix is constructed:
[0041] The original cross-correlation matrix It reflects the consistency of gravity direction observations in two camera coordinate systems, and its elements embody the degree of matching between different coordinate axis directions.
[0042] Specifically, to improve calibration robustness and avoid result deviations caused by single-point noise or measurement errors, this invention introduces a dynamic weighting mechanism based on changes in the direction of gravity.
[0043] First, for two adjacent observation positions and The corresponding unit vector of the direction of gravity and The included angle is defined as the "attitude excitation":
[0044] This included angle reflects the consistency of gravity direction between consecutive observations; the smaller the angle, the more stable the observation.
[0045] Next, we define the normalized angle scale. (Values) ), and set upper and lower limits for the weights. =0.1、 =1, construct the weight function:
[0046] in, The limiting function is defined as follows:
[0047] This function ensures that the weights remain within the range of [0.1, 1], preventing abnormally large angles from causing the weights to be too low and resulting in the loss of effective information. The weights reflect the stability of the corresponding observation location: the smaller the angle, the higher the weight, indicating that the data at that location is more reliable.
[0048] Then, the original cross-correlation matrix H is changed to a weighted cross-correlation matrix:
[0049] in, The weight for the i-th observation location is calculated from the aforementioned excitation amount. This weighting mechanism assigns higher weights to samples with stable observations, effectively suppressing the influence of single-point noise or outliers.
[0050] The weighted cross-correlation matrix H is used to characterize the spatial correlation of point sets in two coordinate systems, and its elements reflect the matching consistency between different coordinate axis directions. By accumulating multiple corresponding projection points, the influence of single-point noise on the attitude matrix solution can be effectively suppressed.
[0051] Furthermore, by adjusting the weighted cross-correlation matrix... Perform singular value decomposition:
[0052] in, and It is an orthogonal matrix. It is a singular value diagonal matrix; To ensure that the final rotation matrix satisfies the constraints of orthogonality and a determinant of +1, a modified diagonal matrix is constructed:
[0053] Then calculate the rotation matrix:
[0054] Where H is the cross-correlation matrix, U is the left singular matrix, and V is the right singular matrix. As a diagonal matrix constructor, this method can effectively suppress the influence of noise, improve calibration accuracy, and enhance the system's robustness to environmental disturbances.
[0055] Furthermore, if a candidate solution to the attitude matrix obtained through singular value decomposition causes abrupt or discontinuous changes in the direction of gravity in the reference coordinate system, the candidate solution is excluded, and the most stable solution is selected as the final result.
[0056] The gravity direction projection coordinates Calculated using the following formula:
[0057] in, For the i-th observation position, the camera Coordinate system at this location to checkerboard coordinate system The rotation part of the attitude transformation matrix; chessboard coordinate system The unit vector representing the direction of gravity; For the direction of gravity at the camera Projected coordinates in the coordinate system.
[0058] Furthermore, when the candidate solutions to the attitude matrix obtained through singular value decomposition cause abrupt or discontinuous changes in the direction of gravity in the reference coordinate system (e.g., due to image blurring, corner point extraction errors, etc.), the candidate solution is excluded, and the most stable solution is selected as the final result.
[0059] The present invention also provides a multi-camera pose calibration device for non-overlapping regions, comprising: Multiple cameras are used to capture images of a planar checkerboard pattern that is vertically suspended in front of each camera from different positions. Memory, which stores computer programs; The processor is configured as described above to calculate the relative rotation relationship between any two cameras.
[0060] Specifically, the processor runs a preset calibration algorithm program, receives image data acquired by each camera, automatically completes steps such as PnP calculation, gravity projection, cross-correlation matrix construction, and singular value decomposition, and outputs the relative rotation matrix between each pair of cameras. This equipment can be deployed in industrial sites, robotic systems, vehicle surround view platforms, and other scenarios to achieve rapid, low-cost, and high-precision calibration of multi-camera extrinsic parameters.
[0061] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for multi-camera pose calibration in a non-overlapping region, characterized in that, Includes the following steps: S1 suspends a planar checkerboard grid vertically in front of each camera to be calibrated; under static conditions, image data of each camera is collected at at least two different positions. S2 calculates the pose transformation matrix of each camera coordinate system relative to the corresponding checkerboard coordinate system based on the image data; Based on the characteristic that the chessboard is always vertically suspended, S3 determines the unit vector representing the direction of gravity in its coordinate system, and uses the attitude transformation matrix to project the unit vector of gravity direction onto the coordinate system of each camera to obtain the gravity direction projection coordinates of each camera at multiple positions. S4 constructs an error minimization model and a cross-correlation matrix for any two cameras to be calibrated, based on their respective gravity direction projection coordinates. Then, it performs singular value decomposition on the cross-correlation matrix to solve for a rotation matrix that satisfies the orthogonality and determinant constraints of +1, thereby obtaining the relative rotation relationship between the two cameras.
2. The method for multi-camera attitude calibration in non-overlapping regions according to claim 1, characterized in that, The error minimization model: And satisfy the constraints: in, For the i-th observation position, and These are the gravity direction projection coordinates in the coordinate systems of the two cameras at the i-th position, respectively; Indicates that the camera Vector transformation in coordinate system to camera Rotation matrix in coordinate system; N is the number of observation positions, and N≥2; , The coordinate systems representing the individual squares of the chessboard. , The coordinate system for the camera corresponding to the chessboard grid; It is a 3×3 identity matrix. The determinant of a matrix is used to ensure... It is a true rotation matrix.
3. The method for multi-camera attitude calibration in non-overlapping regions according to claim 2, characterized in that, The cross-correlation matrix is constructed as follows: Perform singular value decomposition on the cross-correlation matrix to obtain ,in, and It is an orthogonal matrix. It is a singular value diagonal matrix; Construct the corrected diagonal matrix: Calculate the rotation matrix: Where H is the cross-correlation matrix, U is the left singular matrix, and V is the right singular matrix. This is a constructor for diagonal matrices.
4. The method for multi-camera attitude calibration in non-overlapping regions according to claim 1, characterized in that, The unit vector of the direction of gravity is defined in the chessboard coordinate system as follows: The corresponding chessboard grid's X-axis points downwards along the direction of gravity.
5. The method for multi-camera attitude calibration in non-overlapping regions according to claim 1, characterized in that, The attitude transformation matrix is obtained by solving the correspondence between the image coordinates and three-dimensional coordinates of the corner points of the chessboard using the PnP algorithm.
6. The method for multi-camera attitude calibration in non-overlapping regions according to claim 2 or 3, characterized in that, The gravity direction projection coordinates Calculated using the following formula: in, For the i-th observation position, the camera Coordinate system at this location to checkerboard coordinate system The rotation part of the attitude transformation matrix; chessboard coordinate system The unit vector representing the direction of gravity; For the direction of gravity at the camera Projected coordinates in the coordinate system.
7. The method for multi-camera attitude calibration in non-overlapping regions according to claim 1, characterized in that, Each camera operates independently during image acquisition, without synchronous signal transmission between them. Furthermore, the image acquisition time of each camera is set independently according to its own triggering mechanism, and the sequences of images acquired by each camera are not temporally correlated.
8. The method for multi-camera attitude calibration in non-overlapping regions according to claim 1, characterized in that, If a candidate solution to the attitude matrix obtained through singular value decomposition causes abrupt or discontinuous changes in the direction of gravity in the reference coordinate system, the candidate solution is excluded.
9. A multi-camera attitude calibration device with non-overlapping regions, characterized in that, include: Multiple cameras are used to capture images of a planar checkerboard pattern that is vertically suspended in front of each camera from different positions. Memory, which stores computer programs; A processor configured to perform the method as described in any one of claims 1 to 8 to calculate the relative rotation relationship between any two cameras.