Non-shared field of view multi-camera relationship external parameter high-precision calibration method

Abstract

The application discloses a kind of non-shared field of view multi-camera relationship external parameter high-precision calibration method, it is related to camera calibration technical field, comprising: S1, the initial calibration position of the calibration plate corresponding to the camera to be calibrated is determined in advance;S2, establish rigid structure and fix each calibration plate;S3, change the pose of rigid structure, and all shoot calibration image by camera;S4, carry out internal parameter calibration, obtain pose information matrix;S5, select reference camera, obtain relationship external parameter matrix;S6, based on pose information matrix and relationship external parameter matrix, obtain transformation matrix, obtain second pose information;S7, based on second pose information, establish optimization problem;S8, solve optimization problem, calculate third pose information, obtain the external parameter information between final camera.This application can be in multi-camera no sharing field of view scene, both utilize the high-precision characteristics of calibration plate, and do not need to obtain the relationship external parameter between calibration plate as prerequisite in advance, obtain the high-precision external parameter information between multi-camera.

Description

Technical Field

[0001] This invention relates to the field of camera calibration technology, and more specifically to a high-precision calibration method for the extrinsic parameters of multi-camera relationships without shared field of view. Background Technology

[0002] In the field of machine vision, multi-camera joint applications are a common technique. For example, in applications such as binocular stereo vision, automotive driver assistance, and laser triangulation of train wheelset dimensions, multi-camera extrinsic parameters are needed to establish the spatial position and pose relationships between the various cameras, thereby combining the image information captured by each camera. The calibration accuracy of the camera extrinsic parameters directly affects the accuracy of object size measurement and target positioning. In scenarios where multiple cameras share a field of view, the calibration method for camera extrinsic parameters involves using multiple cameras to jointly photograph the same high-definition calibration board. By continuously changing the pose of the calibration board, a series of calibration images are captured, and then the camera extrinsic parameters are calculated using these images. This method is quite mature, but it can only be used in scenarios where multiple cameras share a field of view and cannot be applied to scenarios where there is no shared field of view.

[0003] In existing non-shared field-of-view scenarios, when calibration accuracy requirements are not high, simplified calibration objects are generally used for calibration. For example, in the calibration of automotive 360-degree cameras, a calibration cloth or a calibration image is laid on the ground instead of a high-precision calibration board. However, this process has low calibration accuracy, and different sizes of calibration cloth or calibration images are required for vehicles of different sizes. When calibration accuracy requirements are high, the existing method uses multiple high-precision calibration boards to form a calibration system, with different calibration boards corresponding to different cameras. By taking a calibration image, the extrinsic parameters between the calibration boards in the calibration image are used to calculate the extrinsic parameters of the camera relationship. However, the accuracy of this approach depends on the extrinsic parameter data between the calibration boards. The accuracy of the extrinsic parameter data between the calibration boards directly affects the accuracy of the camera extrinsic parameter calibration. In practical applications, the extrinsic parameters between the calibration boards cannot be accurately obtained in some complex scenarios. Even in some simple scenarios, it is difficult to ensure that the calibration boards are completely parallel and perpendicular to each other, thus making it impossible to obtain high-precision extrinsic parameter information between multiple cameras through the calibration boards. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a high-precision calibration method for the extrinsic parameters of multi-camera relationships without shared field of view.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] This application discloses a high-precision calibration method for the extrinsic parameters of multiple cameras with non-shared field of view, including the following steps:

[0007] S1. Select the corresponding calibration plate according to the number of cameras to be calibrated, and predetermine the initial calibration position of each calibration plate according to the spatial position of the camera.

[0008] S2. Based on the initial calibration position of each calibration plate, establish a rigid structural component to fix each calibration plate.

[0009] S3. Change the pose of the rigid structural member, thereby changing the pose of all calibration plates. Each time the pose of the rigid structural member is changed, each camera takes a calibration image.

[0010] S4. Based on all the calibration images, perform intrinsic parameter calibration on each camera to obtain the intrinsic parameter information of each camera and the first pose information of the calibration board corresponding to each camera. Transform the first pose information into a coordinate rotation and translation transformation matrix in space to obtain the pose information matrix.

[0011] S5. Select one of the cameras as the reference camera, take the initial pose of the calibration board corresponding to the reference camera as the reference, establish the relational extrinsic parameters based on the initial pose of the calibration board corresponding to the other cameras, and transform the relational extrinsic parameters into coordinate rotation and translation transformation matrices in space to obtain the relational extrinsic parameter matrix.

[0012] S6. Based on the pose information matrix and the relational extrinsic parameter matrix, obtain the transformation matrix between all cameras, and obtain the second pose information based on the transformation matrix;

[0013] S7. Based on the second pose information, with the initial pose of the calibration board as the optimization variable, and with the relative rotation and minimum translation between the second pose information as the objectives, establish an optimization problem.

[0014] S8. Solve the optimization problem in step S7 to obtain the initial pose between the calibration plates. Then calculate the third pose information and convert it into matrix form to obtain the final extrinsic parameter information between the cameras.

[0015] Preferably, the intrinsic parameter information mentioned in step S4 includes the camera's focal length, pixel size, pixel center, and distortion coefficient.

[0016] Preferably, step S4 includes: obtaining the first pose information of the calibration board corresponding to each camera. , Where k represents the camera number, n represents the calibration map number, rx represents the rotation angle around the x-axis, ry represents the rotation angle around the y-axis, rz represents the rotation angle around the z-axis, lx represents the translation length along the x-axis, ly represents the translation length along the y-axis, and lz represents the translation length along the z-axis; then the first pose information is... Transformed into a pose information matrix , , where R is a 3*3 orthogonal matrix with a determinant of 1, and l is a 1*3 column vector.

[0017] Preferably, step S5 specifically includes: selecting camera number 1 as the reference camera, using the initial pose of the calibration board corresponding to the reference camera as the reference, establishing relational extrinsic parameters based on the initial pose P of the calibration board corresponding to camera number 2, and converting it into matrix form to obtain relational extrinsic parameter matrix T.

[0018] Preferably, step S6 specifically includes: based on the transformation matrix By combining the relational extrinsic parameter matrix T, we obtain the transformation matrix from camera number 2 to camera number 1. Where N represents the number of pose images, and then the rotation matrix is ​​used. T 0n Converted into second pose information P 0n .

[0019] Preferably, the rotation matrix is... T 0n Converted into second pose information P 0n Specifically, this includes: rotation matrix T 0n for , where R 0n It is a 3x3 orthogonal matrix with a determinant of 1, l 0n It is a 1*3 column vector, and the second pose information P is calculated sequentially. 0n Rotation and translation in:

[0020] ,in Indicates the second pose information P 0n The angle of rotation about the y-axis in the middle. Indicates the second pose information P 0n The angle of rotation about the z-axis in the equation. Indicates the second pose information P 0n The angle of rotation about the x-axis in the equation. Indicates the second pose information P 0n The length of the translation along the x-axis in the middle. Indicates the second pose information P 0n The length of the translation along the y-axis in the middle. Indicates the second pose information P 0n The length of the translation along the z-axis.

[0021] Preferably, step S7 specifically includes: transferring N sets of second pose information P 0nSubtract its average, then normalize, then take the norm, multiply all components by their weights, take the norm, and sum them up. The optimization problem is: ;in, P represents the P-norm. This represents the weighting coefficient of rx. This represents the weighting coefficient of ry. This represents the weighting coefficient of rz. This represents the weighting coefficient of lx. This represents the weighting coefficient of ly. This represents the weighting coefficient of lz.

[0022] Preferably, step S7 specifically includes: solving the optimization problem to obtain the initial pose. P This leads to the extrinsic parameter information matrix between the calibration plates. T ,get N Group 2 pose information P 0n Finally, the second pose information was processed separately. P 0n The average of the six components across N sets of information is calculated to obtain the pose information from camera number 2 to camera number 1. P 00 ,Right now Finally, calculate the extrinsic parameters from camera number 2 to camera number 1. T 00 .

[0023] The beneficial effects of this invention are:

[0024] 1) This application uses a rigid structural component to fix multiple calibration plates and takes multiple sets of calibration images by changing the pose of the structural component to calculate the extrinsic parameters of the relationship between cameras. In scenarios where multiple cameras do not share a field of view, it can obtain high-precision extrinsic parameter information between multiple cameras by utilizing the high precision of the calibration plates without prior knowledge of the extrinsic parameters of the relationship between the calibration plates. The application prerequisites are more relaxed, while still maintaining high precision. Attached Figure Description

[0025] Figure 1 This is a schematic diagram illustrating the steps of a high-precision calibration method for the extrinsic parameters of a multi-camera relationship without shared field of view, according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram showing the initial positions of two cameras without a shared field of view and their corresponding calibration plates in an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram illustrating the construction of a rigid structural component according to an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram illustrating how a rigid structural component can be used to change the orientation of a calibration plate according to an embodiment of the present invention. Detailed Implementation

[0029] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] This application discloses a high-precision calibration method for the extrinsic parameters of multiple cameras without shared field of view. In scenarios without shared field of view, multiple calibration plates are used, and the extrinsic parameter data between the calibration plates does not need to be known in advance to obtain the extrinsic parameter calibration of multiple cameras. Multiple calibration plates of the same model are each corresponding to a camera to be calibrated, and a rigid structural member is used to fix each calibration plate within the calibration field of view of each camera. The pose of the rigid structural member is adjusted to cause the pose transformation of each calibration plate within the field of view of each camera. Multiple sets of calibration images are captured by the cameras, and these images are used for calibration to obtain the extrinsic parameters of each camera. A schematic diagram of the steps of the method is shown below. Figure 1 As shown, the specific steps include:

[0031] S1. Select the corresponding calibration board according to the number of cameras to be calibrated, and pre-determine the initial calibration position of each calibration board according to the spatial position of the camera; wherein the number of cameras to be calibrated is greater than or equal to 2; in the process of determining the calibration position, try to make them in the field of view of the two cameras and in the frontal position.

[0032] S2. Based on the initial calibration position of each calibration plate, a rigid structural component is established to fix each calibration plate. In this step, the calibration plates are connected by the rigid structural component. This layout allows the pose of all calibration plates to be changed simultaneously by simply changing the pose of the rigid structural component, while the extrinsic parameters between the calibration plates remain unchanged.

[0033] S3. Change the pose of the rigid structural component, thereby changing the pose of all calibration plates. Each time the pose of the rigid structural component is changed, each camera takes a calibration image. Assume that a total of N pose images are taken, and N≥3.

[0034] S4. Based on all the calibration images, perform intrinsic parameter calibration on each camera to obtain the intrinsic parameter information of each camera and the first pose information of the calibration board corresponding to each camera. Transform the first pose information into a coordinate rotation and translation transformation matrix in space to obtain the pose information matrix.

[0035] S5. Select one of the cameras as the reference camera, and use the initial pose of the calibration board corresponding to the reference camera as the reference. Based on the initial pose of the calibration board corresponding to the other cameras, establish relational extrinsic parameters, and transform the relational extrinsic parameters into coordinate rotation and translation transformation matrices in space to obtain the relational extrinsic parameter matrix. The relational extrinsic parameters established in this step are unknowns, and are automatically obtained with high precision during the calibration process without needing to be known in advance.

[0036] S6. Based on the pose information matrix and the relational extrinsic parameter matrix, obtain the transformation matrix between all cameras, and obtain the second pose information based on the transformation matrix;

[0037] S7. Based on the second pose information, with the initial pose of the calibration board as the optimization variable, and with the relative rotation and minimum translation between the second pose information as the objectives, establish an optimization problem.

[0038] S8. Solve the optimization problem in step S7 to obtain the initial pose between the calibration plates. Then calculate the third pose information and convert it into matrix form to obtain the final extrinsic parameter information between the cameras.

[0039] For example, this embodiment uses two cameras as an example. The situation is similar for multiple cameras, and will not be elaborated on here. Based on the field of view of the two cameras to be calibrated, two suitable high-precision calibration plates are selected, and the initial positions of the two calibration plates are determined, so that they are in a forward-facing position in the field of view of the two cameras as much as possible. A schematic diagram of the initial positions of two cameras without a shared field of view and their corresponding calibration plates is shown below. Figure 2 As shown, camera 1 represents camera number 1, and the calibration board corresponding to camera 1 is calibration board 1. Camera 2 represents camera number 2, and the calibration board corresponding to camera 2 is calibration board 2.

[0040] For example, based on the initial calibration position of each calibration plate, a rigid structure is established to fix the two calibration plates. The structure connects to the calibration plates from the side or rear to avoid obstructing the imaging of the calibration plates in the camera's field of view. A schematic diagram of the rigid structure is shown below. Figure 3 As shown.

[0041] For example, by changing the pose of the rigid structural member, the pose of all calibration plates is altered. A schematic diagram illustrating the use of the rigid structural member to change the pose of the calibration plates is shown below. Figure 4 As shown. Each camera takes a calibration image for each pose of the structural component (assuming a total of N pose images are taken, N≥3).

[0042] For example, the intrinsic parameters of each camera are calibrated using N calibration images of each camera, and the intrinsic parameter information of each camera is obtained, namely the camera's focal length, pixel size, pixel center, and distortion coefficient.

[0043] For example, step S4 includes: obtaining the first pose information of the calibration board corresponding to each camera. , Where k represents the camera number, n represents the calibration map number, rx represents the rotation angle around the x-axis, ry represents the rotation angle around the y-axis, rz represents the rotation angle around the z-axis, lx represents the translation length along the x-axis, ly represents the translation length along the y-axis, and lz represents the translation length along the z-axis; then the first pose information is... Transformed into a pose information matrix , Where R is a 3x3 orthogonal matrix with a determinant of 1 (a rotation matrix in 3D space), in this embodiment The expression is: ; l is a 1*3 column vector (a translation vector in 3D space), in this embodiment The expression is Therefore, the specific characteristics of each camera can be obtained. N The pose information of the calibration plate in the calibration image. P kn The corresponding transformation matrix T kn .

[0044] For example, no matter how the pose of the rigid structural member is changed, the relative pose relationship between the calibration plates remains unchanged. Step S5 specifically includes: selecting camera number 1 (camera 1) as the reference camera, taking the initial pose of the calibration plate (calibration plate 1) corresponding to the reference camera as the reference, establishing the relational extrinsic parameters based on the initial pose P of the calibration plate (calibration plate 2) corresponding to camera number 2 (camera 2), and converting it into matrix form to obtain the relational extrinsic parameter matrix T.

[0045] For example, step S6 specifically includes: based on the transformation matrix Given the extrinsic parameter matrix T, we obtain the transformation matrix (the extrinsic parameters between cameras) from camera number 2 to camera number 1, which is expressed as follows: Where N represents the number of pose images, and then the rotation matrix is ​​used. T 0n Converted into second pose information P 0n .

[0046] For example, the rotation matrix is ​​described T 0n Converted into second pose information P 0n Specifically, this includes: listing the rotation matrix. T 0n expression , where R0n It is a 3x3 orthogonal matrix (a rotation matrix in 3D space) with a determinant of 1. 0n It is a 1*3 column vector (a translation vector in 3D space), from which the second pose information P is calculated sequentially. 0n Rotation and translation in:

[0047] ,in Indicates the second pose information P 0n The angle of rotation about the y-axis in the middle. Indicates the second pose information P 0n The angle of rotation about the z-axis in the equation. Indicates the second pose information P 0n The angle of rotation about the x-axis in the equation. Indicates the second pose information P 0n The length of the translation along the x-axis in the middle. Indicates the second pose information P 0n The length of the translation along the y-axis in the middle. Indicates the second pose information P 0n The length of the translation along the z-axis.

[0048] For example, based on the preceding second pose information P 0n Using the initial pose P between the calibration plates as a variable, and N sets of second pose information P 0n The relative rotation and minimum translation between the two are taken as objectives to establish an optimization problem; step S7 specifically includes: processing N sets of second pose information P 0n Subtract its average, then normalize, then take the norm, multiply all components by their weights, take the norm, and sum them up. The optimization problem is: ;in, P represents the P-norm (which can be a 1-norm, a 2-norm, or an infinite norm). This represents the weighting coefficient of rx. This represents the weighting coefficient of ry. This represents the weighting coefficient of rz. This represents the weighting coefficient of lx. This represents the weighting coefficient of ly. This represents the weighting coefficient of lz.

[0049] For example, step S7 specifically includes: solving the optimization problem to obtain the initial pose. P Then, the extrinsic parameter information matrix between the calibration plates is obtained through the calculation method in step S4. T Then, through the calculation method in step S6, we can further obtain... N Group 2 pose information P0n Finally, the second pose information was processed separately. P 0n The average of the six components across N sets of information is calculated to obtain the pose information from camera number 2 to camera number 1. P 00 ,Right now Finally, calculate the extrinsic parameter information from camera number 2 to camera number 1. T 00 .

[0050] In summary, this application uses rigid structural members to fix multiple calibration plates, changes the pose of the structural members to capture multiple sets of calibration images, and establishes an optimization problem based on the calibration plate pose information obtained during the calibration process and the extrinsic parameters of the relationships between the calibration plates (which are unknowns). Solving this problem yields the extrinsic parameters of the relationships between the calibration plates, and consequently, the extrinsic parameters of the relationships between the cameras. This method can be used in multi-camera scenarios without shared field of view, utilizing the high precision of the calibration plates without requiring prior knowledge of the extrinsic parameters of the relationships between the calibration plates to obtain high-precision extrinsic parameter information between multiple cameras. It offers more relaxed application prerequisites while maintaining high precision.

[0051] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A high-precision calibration method for non-shared field of view multi-camera relationship extrinsic parameters, characterized in that, The method comprises the following steps: S1, according to the number of cameras to be calibrated, selecting the corresponding calibration board, and according to the spatial position of the camera, determining the initial calibration position of each calibration board in advance; S2, according to the initial calibration position of each calibration board, establishing a rigid structure for fixing each calibration board; S3, changing the pose of the rigid structure, thereby changing the pose of all calibration boards, wherein each time the pose of the rigid structure is changed, each camera shoots a calibration image; S4, based on all the calibration pictures, performing intrinsic calibration on each camera respectively, thereby obtaining the intrinsic information of each camera and the first pose information of the calibration board corresponding to each camera, converting the first pose information into a coordinate rotation and translation transformation matrix in space to obtain a pose information matrix; S5, selecting one of the cameras as a reference camera, taking the initial pose of the calibration board corresponding to the reference camera as a reference, and based on the initial pose of the calibration board corresponding to the other cameras, establishing a relationship extrinsic parameter, converting the relationship extrinsic parameter into a coordinate rotation and translation transformation matrix in space to obtain a relationship extrinsic parameter matrix; S6, based on the pose information matrix and the relationship extrinsic parameter matrix, obtaining the transformation matrix between all cameras, and calculating the second pose information according to the transformation matrix; S7, based on the second pose information, taking the initial pose of the calibration board as an optimization variable, and taking the relative rotation amount and the minimum translation amount between the second pose information as a target, an optimization problem is established; S8, solving the optimization problem in step S7 to obtain the initial pose between the calibration boards, then calculating the third pose information and converting it into a matrix form to obtain the final extrinsic parameter information between the cameras.

2. The non-shared field of view multi-camera relationship extrinsic parameter high-precision calibration method according to claim 1, characterized in that: The intrinsic information in step S4 includes the focal length, pixel size, pixel center and distortion coefficient of the camera.

3. The non-shared field of view multi-camera relationship extrinsic parameter high-precision calibration method according to claim 2, characterized in that, Step S4 comprises: acquiring first pose information of the calibration board corresponding to each camera , , wherein k represents the camera number, n represents the calibration chart number, rx represents the rotation angle around the x-axis, ry represents the rotation angle around the y-axis, rz represents the rotation angle around the z-axis, lx represents the length of translation along the x-axis, ly represents the length of translation along the y-axis, and lz represents the length of translation along the z-axis; and then converting the first pose information into a pose information matrix , , wherein R is a 3*3 orthogonal matrix with a determinant of 1, and l is a 1*3 column vector.

4. The non-shared field of view multi-camera relationship extrinsic parameter high-precision calibration method according to claim 3, characterized in that: Step S5 specifically comprises: selecting the camera with camera number 1 as the reference camera, taking the initial pose of the calibration board corresponding to the reference camera as the reference, and based on the initial pose P of the calibration board corresponding to the camera with camera number 2, establishing a relationship extrinsic parameter and converting it into a matrix form to obtain a relationship extrinsic parameter matrix T.

5. The non-shared field of view multi-camera relationship extrinsic parameter high-precision calibration method according to claim 4, characterized in that: Step S6 specifically comprises: obtaining a transformation matrix from a camera numbered 2 to a camera numbered 1 based on the transformation matrix and the relationship extrinsic parameter matrix T, to obtain a transformation matrix from the camera numbered 2 to the camera numbered 1 , wherein N represents the number of pose images, and then the rotation matrix T 0n is converted into the second pose information P 0n .

6. The non-shared field of view multi-camera relationship extrinsic parameter high-precision calibration method according to claim 5, characterized in that: The rotation matrix T 0n is converted into second pose information P 0n The rotation matrix T 0n is where R 0n is a 3*3 orthogonal matrix with a determinant of 1, l 0n is a 1*3 column vector, and the rotation amount and the translation amount in the second pose information P 0n are sequentially calculated. wherein represents the angle of rotation around the y-axis in the second pose information P 0n represents the angle of rotation around the z-axis in the second pose information P 0n represents the angle of rotation around the x-axis in the second pose information P 0n represents the length of translation along the x-axis in the second pose information P 0n represents the length of translation along the y-axis in the second pose information P 0n represents the length of translation along the z-axis in the second pose information P 0n ​​​​​​ 7. The non-shared field of view multi-camera relationship extrinsic parameter high-precision calibration method according to claim 6, characterized in that, Step S7 specifically includes: taking the N groups of second pose information P 0n Subtracting the average amount, then normalizing, taking the norm, multiplying the weight coefficient to all components, taking the norm, and summing, that is, the optimization problem is: ; wherein, P represents the P-norm, represents the weight coefficient of rx, represents the weight coefficient of ry, represents the weight coefficient of rz, represents the weight coefficient of lx, represents the weight coefficient of ly, represents the weight coefficient of lz.

8. The non-shared field of view multi-camera relationship extrinsic parameter high-precision calibration method according to claim 7, characterized in that, The step S7 specifically comprises: solving the optimization problem to obtain the initial pose P , and further obtaining the extrinsic information matrix between the calibration boards T , obtaining N the second pose information P 0n , and finally respectively calculating the average values of the six components of the second pose information P 0n in the N groups of information to obtain the pose information of the camera numbered 2 to the camera numbered 1 P 00 , that is ; and finally calculating the extrinsic information of the camera numbered 2 to the camera numbered 1 T 00 .

Images