Non-overlapping field of view multi-camera calibration method based on global camera

Abstract

The application relates to the technical field of machine vision, and provides a non-overlapping field of view multi-camera calibration method based on a global camera, which comprises the following steps: arranging a global camera, device cameras and a calibration board; the global camera photographs the calibration board, and initial single-camera internal parameter calibration of the global camera and pose estimation between the global camera and the calibration board are completed; each device camera respectively solves the internal parameter of each device camera based on the photographed calibration board image, and initial single-camera internal parameter calibration of each device camera and pose estimation between each device camera and the calibration board are completed; the initial transformation relationship of each device camera relative to the global camera is calculated, and initial external parameters are obtained; a nonlinear optimization method is used to optimize the process parameters of each initial internal parameter and the derived initial external parameters, and a calibration result is obtained. The non-overlapping field of view calibration problem is converted into a classical overlapping field of view calibration problem by introducing the global camera as a bridge intermediary, and the dependence of a special device in the traditional method is effectively avoided.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method for calibrating a multi-view camera with a non-overlapping field of view based on a global camera. Background Technology

[0002] With the deep integration of industrial technology and artificial intelligence, computer vision <spz str="域内容错误" font="特殊样式" / > The importance of monocular cameras in key areas such as autonomous driving and intelligent transportation is increasing. However, monocular cameras are limited by their limited field of view and difficulty in determining scale, resulting in significant shortcomings in perception capabilities in complex environments. In contrast, multi-camera systems can effectively extend the perception range, building a broader and more comprehensive environmental perception capability. Especially in high-safety applications such as autonomous driving, multi-camera configurations provide crucial redundancy: even if one camera fails due to obstruction or malfunction, the system can still maintain stable operation. Therefore, multi-camera technology lays a solid foundation for building safer and smarter autonomous systems, demonstrating broad application prospects.

[0003] Camera calibration is a fundamental task in computer vision, and its accuracy directly determines the efficiency of subsequent data processing and system applications. Calibration mainly includes intrinsic parameter calibration and extrinsic parameter calibration: intrinsic parameters involve camera-specific properties such as optical center, focal length, and distortion coefficients; extrinsic parameters describe the pose transformations between different cameras, or the positional relationship of the calibration board relative to the cameras. Compared to monocular camera calibration, the intrinsic parameter calibration methods for multi-view cameras are basically the same, but extrinsic parameter calibration is more complex. This is mainly because the fields of view of multiple cameras mounted on unmanned platforms (such as autonomous vehicles and drones) do not overlap, or the overlap area is too small, making it difficult to directly use traditional calibration methods based on common-view relationships. However, accurate extrinsic parameter calibration is a crucial prerequisite for achieving multi-camera information fusion. Only by completing effective calibration can the perceived data from each camera be unified under the same coordinate system, achieving system-level collaboration; otherwise, each camera will still operate independently, failing to achieve the effect of fused perception.

[0004] To address the calibration challenges of multi-camera systems with non-overlapping fields of view, researchers have proposed several innovative methods. Liu Zhen et al. used a one-dimensional target as the calibration object, leveraging its ability to be fabricated to be relatively long, allowing multiple cameras to observe different parts of the target, and then combining this with known point spacing and other information to achieve extrinsic parameter calibration. Furthermore, Liu Zhen et al. proposed a calibration method based on dual-plane targets, calibrating two non-overlapping cameras by repeatedly placing fixed dual-plane targets. Ma Mashuang et al. proposed rigidly connecting multiple small targets and distributing them in the fields of view of each camera, stitching them together to form a large-size calibration plane, and using target stitching and spatial constraint optimization to achieve coordinate unification of the multi-camera system. Wu Ziwei et al. utilized the principle of plane mirror reflection to construct a virtual camera, thus transforming the non-overlapping field of view problem into a virtual overlapping field of view problem. Fu et al. proposed a global calibration method for multi-camera systems with non-overlapping fields of view based on a laser tracker, achieving camera coordinate system unification through high-precision external measurement equipment. Regarding self-calibration and optimization algorithms, Guan Banglei et al. proposed a self-calibration method that utilizes multiple movements of the camera group and combines a hand-eye calibration model to solve for the rotation and translation parameters between cameras. Zhang Tao et al. constructed an ALM adaptive optimization algorithm for low-overlap multi-camera systems, which significantly improved calibration accuracy by iteratively optimizing reprojection errors through adaptive step size.

[0005] Although the above methods can all achieve multi-camera calibration, most of them rely on specialized equipment such as long-sized one-dimensional targets, fixed targets, plane mirrors or laser trackers, which are complex to operate and subject to limited conditions. Summary of the Invention

[0006] The purpose of this invention is to solve at least one technical problem in the background art and to provide a multi-view camera calibration method based on a global camera with non-overlapping field of view.

[0007] To achieve the above objectives, the present invention provides a multi-view camera calibration method based on a global camera with non-overlapping field of view, comprising: At least one global camera, multiple device cameras, and multiple calibration plates are deployed. Each device camera is placed on an unmanned platform, and each calibration plate is placed between the global camera and the device cameras. The global camera and the device cameras share a common field of view, and the global camera and the device cameras can simultaneously observe the calibration plates. The global camera captures images of the calibration board, completing the initial intrinsic parameter calibration of the single camera of the global camera and the pose estimation between the global camera and the calibration board; Each device camera solves its own intrinsic parameters based on the captured images of the calibration board, thus completing the initial intrinsic parameter calibration of each device camera and the pose estimation between the camera and the calibration board. Based on multiple joint observations of the calibration board by the global camera and the cameras of each device, the initial transformation relationship of each device camera relative to the global camera is calculated using the dual-target calibration method. By traversing all device cameras, the initial transformation relationship of each device camera relative to the global camera is obtained, thereby deriving the initial extrinsic parameters between each device camera. Using the sum of reprojection errors of all observation points as the objective function, a nonlinear optimization method is employed to optimize each initial intrinsic parameter and the process parameters for deriving the initial extrinsic parameters, thereby obtaining the calibration results.

[0008] According to one aspect of the present invention, the global camera includes a static global camera and a dynamic global camera, wherein the static global camera remains unchanged relative to the world coordinate system, and the dynamic global camera is supported on a two-dimensional turntable and rotates with the two-dimensional turntable.

[0009] According to one aspect of the invention, the calibration plate is provided with a checkerboard pattern.

[0010] According to one aspect of the present invention, the initial intrinsic parameter calibration of the single camera includes: In the calibration plate coordinate system, all corner points of the chessboard grid lie on the calibration plate plane, therefore each corner point Coordinate 0; any point on the chessboard. In the calibration plate coordinate system, it is represented as After this point is imaged by the global camera or the device camera, the corresponding pixel is The following projection relationship is satisfied: (1) in, A non-zero scale factor. This is the camera intrinsic parameter matrix, including focal length. and the optical center coordinates of the device camera The specific form is as follows: (2) matrix Let r1 be the extrinsic parameter matrix, representing the rotation and translation transformation from the calibration board coordinate system to the global camera coordinate system or the device camera coordinate system, where r1, r2, and r3 are the 1st, 2nd, and 3rd columns of the rotation matrix. It is a translation vector; make , Given a 3×3 homography matrix, establish the direct projection relationship from the calibration plate plane to the camera image plane: (3) in, Represents homogeneous coordinates; Solving using the direct linear transformation algorithm ,make express The Okay, expand equation (3) and eliminate the scale factor. : (4) After simplification, we obtain the system of equations corresponding to each corner point: (5) Represented in homogeneous form: (6) for At each corner point, stack all equations into matrix Solve And satisfy This problem can be solved using singular value decomposition, and the solution is: The eigenvector corresponding to the smallest eigenvalue can be used to obtain the homography matrix. ; Each calibration plate image can be used to calculate a homography matrix. , By utilizing the homography matrices under multiple different poses, the camera intrinsic parameter matrix can be solved. Complete the initial intrinsic parameter calibration of the single camera.

[0011] According to one aspect of the invention, the pose estimation is the extrinsic parameter matrix, which is the transformation matrix of the calibration board relative to the global camera or the device camera, obtained by solving the camera intrinsic parameter matrix. It is recovered through rotation and translation transformations.

[0012] According to one aspect of the present invention, based on multiple joint observations of the calibration board by the global camera and each device camera, the initial transformation relationship of each device camera relative to the global camera is calculated using a dual-target calibration method. This process iterates through all device cameras to obtain the initial transformation relationship of each device camera relative to the global camera, thereby deriving the initial extrinsic parameters between the device cameras, including: When global camera With a certain device camera Multiple calibration plates were observed together. hour, Based on the solved intrinsic parameter matrices of each camera Obtain the values ​​of each calibration board relative to the global camera. and equipment camera Transformation matrix and transformation matrix ; Computing device camera Compared to a global camera Transformation relationship : (7) Total available Group transformation relationship, each group Convert to Li algebra 6-dimensional vector After taking the midpoint, map it back to the Lie group. As a device camera Compared to a global camera Final transformation estimate ; Based on transformation estimation The initial extrinsic parameters between the cameras of each device are obtained.

[0013] According to one aspect of the invention, the transformation-based estimation Obtain the initial extrinsic parameters between the cameras of each device, including: For static global cameras, global cameras Relative to the world coordinate system Remain unchanged; known device camera and equipment camera Compared to a global camera Transformation and They are respectively: , (8) in, and They represent cameras Compared to a global camera The rotation matrix and translation vector, and They represent cameras Compared to a global camera The rotation matrix and translation vector; Then the device camera To the device camera Transformation Calculated using the following formula: (9) For dynamic global cameras, global cameras As the 2D turntable rotates, let the global camera be located at the device camera. and equipment camera The corresponding global camera coordinate systems are as follows: and The turntable rotates around its Z-axis by an angle Then the corresponding transformation matrix for: (10) in, This refers to the translation between the global camera and the rotation center of the turntable; Here is the rotation matrix about the Z-axis: (11) Known device camera and equipment camera Relative to the global camera coordinate system and global camera coordinate system The transformations are as follows: , (12) Unify the two transformations to the global camera coordinate system. Down: (13) Finally, the device camera was obtained. and equipment camera Transformation between: (14).

[0014] According to one aspect of the present invention, the optimization of each initial intrinsic parameter and the process parameters for deriving the initial extrinsic parameters using a nonlinear optimization method with the sum of reprojection errors of all observation points as the objective function includes: The bundled adjustment method is used to perform nonlinear optimization on the initial intrinsic parameters and the process parameters for deriving the initial extrinsic parameters. The process parameters for deriving the initial extrinsic parameters include the transformation matrix of each calibration plate relative to the global camera. Transform estimation of each device's camera relative to the global camera ; The objective of the nonlinear optimization is to minimize the overall reprojection error, encompassing the observation errors of both the global camera and the multi-view camera. Its objective function is defined as follows: (15) in, This represents the device camera index, with a total of N device cameras. This indicates the calibration board index; there are a total of M calibration boards. Represents the corner index on the calibration board, set Includes all valid corner points. The index representing the total number of global camera rotations. Next rotation; For the reprojection error corresponding to the global camera: (16) The first among multi-view cameras Reprojection error corresponding to each camera: (17) In the formula, Indicates the first The first camera observed the The first calibration board The pixel coordinates of the corner points The coordinates of this corner point in the calibration plate coordinate system are: and These are the global camera and the first... The intrinsic parameter matrix of each camera, This is the camera projection function.

[0015] To achieve the above objectives, the present invention also provides a multi-view camera calibration system based on a global camera with a non-overlapping field of view, comprising: The system structure layout module includes at least one global camera, multiple device cameras, and multiple calibration boards. Each device camera is placed on an unmanned platform, and each calibration board is placed between the global camera and the device cameras. The global camera and the device cameras share a common field of view, and the global camera and the device cameras can simultaneously observe the calibration boards. A global camera calibration module, wherein the global camera captures the calibration board, and completes the initial intrinsic parameter calibration of the single camera of the global camera and the pose estimation between the global camera and the calibration board; The device camera intrinsic parameter calibration module solves for the intrinsic parameters of each device camera based on the captured calibration board image, thereby completing the initial intrinsic parameter calibration of each device camera and the pose estimation between the device camera and the calibration board. The device camera initial extrinsic parameter calibration module, based on multiple joint observations of the calibration board by the global camera and each device camera, uses a dual-target calibration method to calculate the initial transformation relationship of each device camera relative to the global camera. By traversing all device cameras, the initial transformation relationship of each device camera relative to the global camera is obtained, thereby deriving the initial extrinsic parameters between each device camera. The calibration result acquisition module uses the sum of reprojection errors of all observation points as the objective function and employs a nonlinear optimization method to optimize each initial intrinsic parameter and the process parameters for deriving the initial extrinsic parameters to obtain the calibration results.

[0016] To achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the non-overlapping field-of-view multi-view camera calibration method based on a global camera as described above.

[0017] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the non-overlapping field-of-view multi-view camera calibration method based on a global camera as described above.

[0018] According to the present invention, considering the limited size of the unmanned platform and the fact that the specifications of different non-overlapping cameras are basically the same, the present invention transforms the original non-overlapping field-of-view calibration problem into an overlapping field-of-view problem. Specifically, a global camera is set up not far outside the platform, so that its field of view overlaps with that of the cameras of each device on the platform. Through this arrangement, the original non-overlapping multi-camera calibration problem is transformed into a classic overlapping field-of-view calibration problem, thereby greatly simplifying the process. In practical applications, depending on whether the depth of field of the global camera meets the coverage requirements, the method can be further divided into two forms: static global camera and dynamic global camera. The static method is suitable for scenarios with sufficient depth of field, where the position of the global camera is fixed; the dynamic method expands the effective observation range by mounting the global camera on a two-dimensional turntable, and is suitable for situations with limited depth of field or large camera layout spans. The proposed method is simple to operate, requires no complex equipment, can meet the needs of rapid calibration of unmanned platforms, and is especially suitable for rapid deployment in field environments, possessing important theoretical significance and broad application prospects.

[0019] This invention transforms the non-overlapping field-of-view calibration problem into the classic overlapping field-of-view calibration problem by introducing a global camera as a bridge intermediary, effectively avoiding the dependence on dedicated equipment in traditional methods. The method of this invention offers advantages such as ease of operation and good stability while ensuring calibration accuracy, and can meet the rapid deployment needs of unmanned platforms in field environments. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a non-overlapping field-of-view multi-camera calibration method based on a global camera according to an embodiment of the present invention; Figure 2 The schematic diagram illustrates the layout of a global camera, device camera, and calibration board based on an unmanned platform according to an embodiment of the present invention. Figure 3 Deployment strategy diagram for dual-device cameras and static global cameras; Figure 4Deployment strategy diagram for dual-device cameras and dynamic global cameras; Figure 5 A diagram illustrating the deployment strategy for calibrating four-device cameras and a static global camera. Figure 6 A diagram illustrating the deployment strategy for calibrating four-device cameras and a dynamic global camera. Figure 7 The setup diagram for the dual-device camera calibration test in Example 1 is shown. Detailed Implementation

[0021] The invention will now be discussed with reference to exemplary embodiments. It should be understood that the described embodiments are merely intended to enable those skilled in the art to better understand and thus implement the invention, and are not intended to imply any limitation on the scope of the invention.

[0022] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment".

[0023] Figure 1 A flowchart illustrating a non-overlapping field-of-view multi-camera calibration method based on a global camera according to an embodiment of the present invention; Figure 2 This schematic diagram illustrates the arrangement of a global camera, device camera, and calibration board based on an unmanned platform according to one embodiment of the present invention. Figure 1 and Figure 2 As shown, in this embodiment, the multi-view camera calibration method based on the non-overlapping field of view of a global camera includes: Deploy at least one global camera G, multiple device cameras C, and multiple calibration boards B, and position each device camera on an unmanned platform (e.g., Figure 2 On the car (in the middle), each calibration plate is placed between the global camera and the equipment camera. The global camera and the equipment camera have a common field of view and can observe the calibration plate at the same time. The global camera captures images of the calibration board, completing the initial intrinsic parameter calibration of the single camera of the global camera and the pose estimation between the global camera and the calibration board; Each device camera solves its own intrinsic parameters based on the captured images of the calibration board, thus completing the initial intrinsic parameter calibration of each device camera and the pose estimation between the camera and the calibration board. Based on multiple joint observations of the calibration board by the global camera and the cameras of each device, the initial transformation relationship of each device camera relative to the global camera is calculated using the dual-target calibration method. By traversing all device cameras, the initial transformation relationship of each device camera relative to the global camera is obtained, thereby deriving the initial extrinsic parameters between each device camera. Using the sum of reprojection errors of all observation points as the objective function, a nonlinear optimization method is employed to optimize each initial intrinsic parameter and the process parameters for deriving the initial extrinsic parameters, thereby obtaining the calibration results.

[0024] In this embodiment, the position of the global camera in space is fixed, and the world coordinate system is set on the global camera, that is, it coincides with the global camera coordinate system. By reasonably arranging the position and orientation of the global camera, it is ensured that its field of view shares a common field of view with every device camera in the multi-view system of the unmanned platform.

[0025] In this embodiment, the calibration board adopts a single-sided or double-sided checkerboard design. With a double-sided checkerboard, the corner points on both sides correspond strictly one-to-one, and the thickness of the calibration board is negligible, thus ensuring that the corner points on both sides represent the same set of points in space. During calibration, the calibration board only needs to be placed in any position multiple times, ensuring that it is clearly observed simultaneously by at least one camera on the global camera and any camera on the platform each time.

[0026] Furthermore, according to one embodiment of the present invention, for common multi-camera layouts on unmanned platforms (such as dual-camera, quad-camera, etc.), different arrangement strategies can be adopted based on static and dynamic global cameras respectively. In a dual-camera system, the static global camera can be arranged along the line connecting the two device cameras, so that its field of view simultaneously covers the two local device cameras; the dynamic global camera is arranged at a suitable position on the center line between the two to ensure clear imaging within the depth of field of each local device camera, as shown in the specific arrangement. Figure 3 and Figure 4 As shown. For a four-camera system, a multi-global camera collaborative calibration scheme can be adopted, that is, using two global cameras to cover two sets of local device cameras respectively. Since there is a common field of view between these global cameras, their extrinsic parameters can be accurately calibrated first using traditional methods, and then used as a unified "bridge" system to calibrate the covered device cameras separately. If a dynamic global camera is used, calibration can be completed simply by placing the turntable at approximately the center of the four device cameras, as shown in the specific arrangement. Figure 5 and Figure 6 As shown.

[0027] In this embodiment, as described above, global cameras are categorized into static and dynamic types based on whether a turntable is configured. Static global cameras employ short-focal-length lenses, possessing a large depth of field and capable of stably extracting checkerboard corner points at varying distances. If the distance between the calibration board and the global camera exceeds the camera's depth of field, a two-dimensional turntable is required to construct a dynamic global camera system. This system adjusts the camera's orientation via the turntable, effectively expanding the global camera's field of view while ensuring its depth of field fully overlaps with that of non-overlapping field-of-view cameras.

[0028] In this embodiment, during the calibration process, the global camera and the multi-view camera need to take chessboard images multiple times from different perspectives. Then, the classic Zhang Zhengyou calibration method is used to calculate the intrinsic parameters of each camera separately.

[0029] Furthermore, according to one embodiment of the present invention, the initial intrinsic parameter calibration of a single camera includes: In the calibration plate coordinate system, all corner points of the chessboard grid lie on the calibration plate plane, therefore each corner point Coordinate 0; any point on the chessboard. In the calibration plate coordinate system, it is represented as After this point is imaged by the global camera or the device camera, the corresponding pixel is The following projection relationship is satisfied: (1) in, A non-zero scale factor. This is the camera intrinsic parameter matrix, including focal length. and the optical center coordinates of the device camera The specific form is as follows: (2) matrix Let r1 be the extrinsic parameter matrix, representing the rotation and translation transformation from the calibration board coordinate system to the global camera coordinate system or the device camera coordinate system, where r1, r2, and r3 are the 1st, 2nd, and 3rd columns of the rotation matrix. It is a translation vector; make , Given a 3×3 homography matrix, establish the direct projection relationship from the calibration plate plane to the camera image plane: (3) in, Represents homogeneous coordinates; Solving using the direct linear transformation algorithm ,make express The Okay, expand equation (3) and eliminate the scale factor. : (4) After simplification, we obtain the system of equations corresponding to each corner point: (5) Represented in homogeneous form: (6) for At each corner point, stack all equations into matrix Solve And satisfy This problem can be solved using singular value decomposition, and the solution is: The eigenvector corresponding to the smallest eigenvalue can be used to obtain the homography matrix. ; Each calibration plate image can be used to calculate a homography matrix. ,according to By utilizing the homography matrices under multiple different poses, the camera intrinsic parameter matrix can be solved. Complete the initial intrinsic parameter calibration of the single camera.

[0030] Furthermore, according to one embodiment of the present invention, the pose estimation is the aforementioned extrinsic parameter matrix, which is the transformation matrix of the calibration board relative to the global camera or the device camera, obtained by solving the camera intrinsic parameter matrix. It is recovered through rotation and translation transformations. For example, the extrinsic parameter matrix T CB , which is the transformation matrix of the calibration board relative to the device camera. Here, T CB This represents the transformation matrix from the calibration board coordinate system to the device camera coordinate system, including the rotation matrix R. CB Translation vector t CB The following text of this invention will use this method to represent the transformation relationships between coordinate systems.

[0031] Furthermore, according to one embodiment of the present invention, based on multiple joint observations of the calibration board by the global camera and each device camera, the initial transformation relationship of each device camera relative to the global camera is calculated using a dual-target calibration method. By traversing all device cameras, the initial transformation relationship of each device camera relative to the global camera is obtained, thereby deriving the initial extrinsic parameters between each device camera, including: When global camera With a certain device camera Multiple calibration plates were observed together. hour, Based on the solved intrinsic parameter matrices of each camera Obtain the values ​​of each calibration board relative to the global camera. and equipment camera Transformation matrix and transformation matrix ; Computing device camera Compared to a global camera Transformation relationship : (7) Total available Group transformation relationship, each group Convert to Li algebra 6-dimensional vector After taking the midpoint, map it back to the Lie group. As a device camera Compared to a global camera Final transformation estimate ; Based on transformation estimation The initial extrinsic parameters between the cameras of each device are obtained.

[0032] In this embodiment, based on transformation estimation Obtain the initial extrinsic parameters between the cameras of each device, including: For static global cameras, global cameras Relative to the world coordinate system Remain unchanged; known device camera and Compared to a global camera Transformation and They are respectively: , (8) in, and They represent cameras Compared to a global camera The rotation matrix and translation vector, and They represent cameras Compared to a global camera The rotation matrix and translation vector; Then the device camera To the device camera Transformation Calculated using the following formula: (9) Similarly, the transformation between any two device cameras can be achieved using a global camera. As an intermediate bridge, the above method was used to derive the result; For dynamic global cameras, global cameras As the 2D turntable rotates, let the global camera be located at the device camera. and equipment camera The corresponding global camera coordinate systems are as follows: and The turntable rotates around its Z-axis by an angle Then the corresponding transformation matrix for: (10) in, This refers to the translation between the global camera and the rotation center of the turntable; Here is the rotation matrix about the Z-axis: (11) Known device camera and equipment camera Relative to the global camera coordinate system and global camera coordinate system The transformations are as follows: , (12) Unify the two transformations to the global camera coordinate system. Down: (13) Finally, the device camera was obtained. and equipment camera Transformation between: (14).

[0033] Furthermore, according to one embodiment of the present invention, using the sum of reprojection errors of all observation points as the objective function, a nonlinear optimization method is employed to optimize each initial intrinsic parameter and the process parameters for deriving the initial extrinsic parameters, including: The bundled adjustment method is used to perform nonlinear optimization on the initial intrinsic parameters and the process parameters for deriving the initial extrinsic parameters. The process parameters for deriving the initial extrinsic parameters include the transformation matrix of each calibration plate relative to the global camera. Transform estimation of each device's camera relative to the global camera ; The objective of the nonlinear optimization is to minimize the overall reprojection error, encompassing the observation errors of both the global camera and the multi-view camera. Its objective function is defined as follows: (15) in, This represents the device camera index, with a total of N device cameras. This indicates the calibration board index; there are a total of M calibration boards. Represents the corner index on the calibration board, set Includes all valid corner points. The index representing the total number of global camera rotations. Next rotation; For the reprojection error corresponding to the global camera: (16) The first among multi-view cameras Reprojection error corresponding to each camera: (17) In the formula, Indicates the first The first camera observed the The first calibration board The pixel coordinates of the corner points The coordinates of this corner point in the calibration plate coordinate system are: and These are the global camera and the first... The intrinsic parameter matrix of each camera, This is the camera projection function, which maps 3D points to image pixel coordinates.

[0034] According to the above-described scheme of this invention, considering the limited size of the unmanned platform and the fact that the specifications of different non-overlapping cameras are basically the same, this invention utilizes this characteristic to transform the original non-overlapping field-of-view calibration problem into an overlapping field-of-view problem. Specifically, a global camera is set up not far outside the platform, so that its field of view overlaps with that of the cameras on the platform. Through this arrangement, the original non-overlapping multi-camera calibration problem is transformed into a classic overlapping field-of-view calibration problem, thereby greatly simplifying the process. In practical applications, depending on whether the depth of field of the global camera meets the coverage requirements, the method can be further divided into two forms: static global camera and dynamic global camera. The static method is suitable for scenarios with sufficient depth of field, where the position of the global camera is fixed; the dynamic method, on the other hand, expands the effective observation range by mounting the global camera on a two-dimensional turntable, and is suitable for situations with limited depth of field or large camera layout spans. The proposed method is simple to operate, requires no complex equipment, and can meet the needs of rapid calibration of unmanned platforms, especially suitable for rapid deployment in field environments, possessing important theoretical significance and broad application prospects.

[0035] Furthermore, to achieve the above objectives, the present invention also provides a multi-view camera calibration system based on a global camera with a non-overlapping field of view, comprising: The system structure layout module includes at least one global camera, multiple device cameras, and multiple calibration plates. The coordinate system of the global camera is set as the world coordinate system. Each device camera is placed on an unmanned platform. Each calibration plate is placed between the global camera and the device cameras. The global camera and the device cameras share a common field of view. The global camera and the device cameras can simultaneously observe the calibration plates. A global camera calibration module, wherein the global camera captures the calibration board, and completes the initial intrinsic parameter calibration of the single camera of the global camera and the pose estimation between the global camera and the calibration board; The device camera intrinsic parameter calibration module solves for the intrinsic parameters of each device camera based on the captured calibration board image, thereby completing the initial intrinsic parameter calibration of each device camera and the pose estimation between the device camera and the calibration board. The device camera initial extrinsic parameter calibration module, based on multiple joint observations of the calibration board by the global camera and each device camera, uses a dual-target calibration method to calculate the initial transformation relationship of each device camera relative to the global camera. By traversing all device cameras, the initial transformation relationship of each device camera relative to the global camera is obtained, thereby deriving the initial extrinsic parameters between each device camera. The calibration result acquisition module uses the sum of reprojection errors of all observation points as the objective function and employs a nonlinear optimization method to optimize each initial intrinsic parameter and the process parameters for deriving the initial extrinsic parameters to obtain the calibration results.

[0036] The non-overlapping field-of-view multi-camera calibration system based on a global camera according to the present invention can realize the above-mentioned non-overlapping field-of-view multi-camera calibration method based on a global camera. The specific process steps are as described above and will not be repeated here.

[0037] Furthermore, to achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the non-overlapping field-of-view multi-view camera calibration method based on a global camera as described above.

[0038] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the non-overlapping field-of-view multi-view camera calibration method based on a global camera as described above.

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described and verified in detail below with reference to the accompanying drawings and verification examples. It should be understood that the specific verification examples described herein are only used to explain and verify this invention and do not limit the scope of protection of this invention.

[0040] Verification Example 1 To verify the effectiveness of the method of the present invention in a real-world scenario, this verification example uses two opposing cameras without a common field of view for the experiment. The scene setup is as follows: Figure 7 As shown. The rotation angle between the two cameras is approximately 180°, and the translation distance is approximately 40cm. In the experiment, the non-overlapping camera was a Daheng Image Mars-2020-38GTM high-definition camera with an image resolution of 4512 × 4512 pixels. This verification example conducts calibration experiments based on both static and dynamic global camera methods, and compares them with the traditional calibration method based on control points.

[0041] The static global camera method uses a short focal length lens of 6mm to increase the depth of field, ensuring that the checkerboard corner points can be stably extracted when photographing the calibration board at different distances. Simultaneously, this method employs a double-sided calibration board to ensure that the same calibration board can be photographed simultaneously by cameras on both sides. The calibration board uses a double-sided checkerboard design, with strict correspondence between the corner points on both sides. The thickness of the calibration board is negligible, ensuring that the corner points on both sides represent the same set of points in space. The dynamic global camera method uses a standard lens, and the camera is mounted on a two-dimensional rotating stage. It does not require a double-sided calibration board; calibration can be completed simply by repeatedly placing a single-sided calibration board. Both methods use the same procedure for camera intrinsic parameter calibration; therefore, the intrinsic parameter results are not differentiated in Table 1, which lists the effective focal lengths. Optical center coordinates of the device camera Radial distortion coefficient and tangential distortion coefficient .

[0042] Table 1 Camera intrinsic parameter calibration results The extrinsic parameters between non-overlapping cameras obtained based on the static and dynamic global camera methods are shown in Table 2, where Euler angles are used for rotation. The translation vector is represented by... express.

[0043] Table 2 Camera extrinsic parameter calibration results Experimental results show that the dynamic global camera method yields results closer to the baseline method, while the static global camera method, while largely consistent with the baseline method in rotation and translation parameters, still exhibits some deviation. Analysis suggests this deviation primarily stems from two aspects: first, the limited depth of field of the static global camera leads to significant pixel errors in corner extraction when photographing distant calibration boards; second, the smaller imaging range of the distant calibration board within the global camera results in a larger physical size corresponding to a single pixel, amplifying even minor corner extraction errors and thus affecting calibration accuracy. Overall, both global calibration methods achieve effective calibration results, with overall errors controlled at the centimeter level, indicating the feasibility of these methods in practical applications. In industrial applications, a more precise global camera can be further selected to improve overall calibration accuracy.

[0044] As can be seen from the above, this invention addresses the challenge of multi-camera calibration in non-overlapping fields of view by proposing two simplified calibration methods based on global camera assistance. By introducing a global camera as a bridge, the non-overlapping field of view calibration problem is transformed into the classic overlapping field of view calibration problem, effectively avoiding the dependence on dedicated equipment in traditional methods. This verification example demonstrates that the method of this invention, while ensuring calibration accuracy, has the advantages of simple operation and good stability, and can meet the needs of rapid calibration for multi-camera systems on unmanned platforms.

[0045] Those skilled in the art will recognize that the modules and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0046] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and equipment can be referred to the corresponding process in the foregoing method implementation, and will not be repeated here.

[0047] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0048] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs.

[0049] In addition, the functional modules in the embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0050] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the sending / receiving methods of various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0051] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

[0052] It should be understood that the sequence number of each step in the invention and its embodiments does not absolutely imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Claims

1. A multi-view camera calibration method based on a global camera with non-overlapping field of view, characterized in that, include: At least one global camera, multiple device cameras, and multiple calibration plates are deployed. Each device camera is placed on an unmanned platform, and each calibration plate is placed between the global camera and the device cameras. The global camera and the device cameras share a common field of view, and the global camera and the device cameras can simultaneously observe the calibration plates. The global camera captures images of the calibration board, completing the initial intrinsic parameter calibration of the single camera of the global camera and the pose estimation between the global camera and the calibration board; Each device camera solves its own intrinsic parameters based on the captured images of the calibration board, thus completing the initial intrinsic parameter calibration of each device camera and the pose estimation between the camera and the calibration board. Based on multiple joint observations of the calibration board by the global camera and the cameras of each device, the initial transformation relationship of each device camera relative to the global camera is calculated using the dual-target calibration method. By traversing all device cameras, the initial transformation relationship of each device camera relative to the global camera is obtained, thereby deriving the initial extrinsic parameters between each device camera. Using the sum of reprojection errors of all observation points as the objective function, a nonlinear optimization method is employed to optimize each initial intrinsic parameter and the process parameters for deriving the initial extrinsic parameters, thereby obtaining the calibration results. The global camera includes a static global camera and a dynamic global camera. The static global camera remains unchanged relative to the set world coordinate system, while the dynamic global camera is supported on a two-dimensional turntable and rotates with the two-dimensional turntable. The calibration board is equipped with a chessboard pattern; The initial intrinsic parameter calibration of the single camera includes: In the calibration plate coordinate system, all corner points of the chessboard grid lie on the calibration plate plane, therefore each corner point Coordinate 0; any point on the chessboard. In the calibration plate coordinate system, it is represented as After this point is imaged by the global camera or the device camera, the corresponding pixel is The following projection relationship is satisfied: （1） in, A non-zero scale factor. This is the camera intrinsic parameter matrix, including focal length. and the optical center coordinates of the device camera The specific form is as follows: （2） matrix Let r1 be the extrinsic parameter matrix, representing the rotation and translation transformation from the calibration board coordinate system to the global camera coordinate system or the device camera coordinate system, where r1, r2, and r3 are the 1st, 2nd, and 3rd columns of the rotation matrix. It is a translation vector; make , Given a 3×3 homography matrix, establish the direct projection relationship from the calibration plate plane to the camera image plane: （3） in, Represents homogeneous coordinates; Solving using the direct linear transformation algorithm ,make express The Okay, expand equation (3) and eliminate the scale factor. : （4） After simplification, we obtain the system of equations corresponding to each corner point: （5） Represented in homogeneous form: （6） for At each corner point, stack all equations into matrix Solve And satisfy This problem can be solved using singular value decomposition, and the solution is: The eigenvector corresponding to the smallest eigenvalue can be used to obtain the homography matrix. ; Each calibration plate image can be used to calculate a homography matrix. ,according to By utilizing the homography matrices under multiple different poses, the camera intrinsic parameter matrix can be solved. Complete the initial intrinsic parameter calibration of the single camera; The pose estimation is the extrinsic parameter matrix, which is the transformation matrix of the calibration board relative to the global camera or the device camera, obtained by solving the camera intrinsic parameter matrix. It is obtained by rotation and translation transformation; Based on multiple joint observations of the calibration board by the global camera and each device camera, the initial transformation relationship of each device camera relative to the global camera is calculated using a dual-target calibration method. By traversing all device cameras, the initial transformation relationship of each device camera relative to the global camera is obtained, thereby deriving the initial extrinsic parameters between each device camera, including: When global camera With a certain device camera Multiple calibration plates were observed together. hour, Based on the solved intrinsic parameter matrices of each camera Obtain the values ​​of each calibration board relative to the global camera. and equipment camera Transformation matrix and transformation matrix ; Computing device camera Compared to a global camera Transformation relationship : （7） Total available Group transformation relationship, each group Convert to Li algebra 6-dimensional vector After taking the midpoint, map it back to the Lie group. As a device camera Compared to a global camera Final transformation estimate ; Based on transformation estimation Obtain the initial extrinsic parameters between the cameras of each device; The transformation-based estimation Obtain the initial extrinsic parameters between the cameras of each device, including: For static global cameras, global cameras Relative to the world coordinate system Remain unchanged; known device camera and equipment camera Compared to a global camera Transformation and They are respectively: ， （8） in, and They represent cameras Compared to a global camera The rotation matrix and translation vector, and They represent cameras Compared to a global camera The rotation matrix and translation vector; Then the device camera To the device camera Transformation Calculated using the following formula: （9） For dynamic global cameras, global cameras As the 2D turntable rotates, let the global camera be located at the device camera. and equipment camera The corresponding global camera coordinate systems are as follows: and The turntable rotates around its Z-axis by an angle Then the corresponding transformation matrix for: （10） in, This refers to the translation between the global camera and the rotation center of the turntable; Here is the rotation matrix about the Z-axis: （11） Known device camera and equipment camera Relative to the global camera coordinate system and global camera coordinate system The transformations are as follows: ， （12） Unify the two transformations to the global camera coordinate system. Down: （13） Finally, the device camera was obtained. and equipment camera Transformation between: （14） The objective function is the sum of reprojection errors at all observation points. A nonlinear optimization method is used to optimize the initial intrinsic parameters and the process parameters for deriving the initial extrinsic parameters, including: The bundled adjustment method is used to perform nonlinear optimization on the initial intrinsic parameters and the process parameters for deriving the initial extrinsic parameters. The process parameters for deriving the initial extrinsic parameters include the transformation matrix of each calibration plate relative to the global camera. Transform estimation of each device's camera relative to the global camera ; The objective of the nonlinear optimization is to minimize the overall reprojection error, encompassing the observation errors of both the global camera and the multi-view camera. Its objective function is defined as follows: （15） in, This represents the device camera index, with a total of N device cameras. This indicates the calibration board index; there are a total of M calibration boards. Represents the corner index on the calibration board, set Includes all valid corner points. The index representing the total number of global camera rotations. Next rotation; For the reprojection error corresponding to the global camera: （16） The first among multi-view cameras Reprojection error corresponding to each device camera: （17） In the formula, Indicates the first The first device camera observed the first The first calibration board The pixel coordinates of the corner points The coordinates of this corner point in the calibration plate coordinate system are: and These are the global camera and the first... The intrinsic parameter matrix of the camera of each device. This is the camera projection function.

2. A multi-camera calibration system based on a global camera and implementing the multi-camera calibration method based on a global camera and non-overlapping field of view as described in claim 1, characterized in that, include: The system structure layout module includes at least one global camera, multiple device cameras, and multiple calibration boards. Each device camera is placed on an unmanned platform, and each calibration board is placed between the global camera and the device cameras. The global camera and the device cameras share a common field of view, and the global camera and the device cameras can simultaneously observe the calibration boards. A global camera calibration module, wherein the global camera captures the calibration board, and completes the initial intrinsic parameter calibration of the single camera of the global camera and the pose estimation between the global camera and the calibration board; The device camera intrinsic parameter calibration module solves for the intrinsic parameters of each device camera based on the captured calibration board image, thereby completing the initial intrinsic parameter calibration of each device camera and the pose estimation between the device camera and the calibration board. The device camera initial extrinsic parameter calibration module, based on multiple joint observations of the calibration board by the global camera and each device camera, uses a dual-target calibration method to calculate the initial transformation relationship of each device camera relative to the global camera. By traversing all device cameras, the initial transformation relationship of each device camera relative to the global camera is obtained, thereby deriving the initial extrinsic parameters between each device camera. The calibration result acquisition module uses the sum of reprojection errors of all observation points as the objective function and employs a nonlinear optimization method to optimize each initial intrinsic parameter and the process parameters for deriving the initial extrinsic parameters to obtain the calibration results.

3. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the non-overlapping field-of-view multi-view camera calibration method based on a global camera as described in claim 1.

4. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements the non-overlapping field-of-view multi-view camera calibration method based on a global camera as described in claim 1.

Images