A camera focal length and pose calibration method based on two straight lines

Abstract

The application provides a camera focal length and posture calibration method based on two straight lines, and belongs to the technical fields of computer vision, SLAM and photogrammetry. The application only needs to know two straight lines in space, measure the camera position and the two straight line positions, shoot straight line feature images, obtain the imaging relationship of the straight lines in the camera, utilize the important geometric phenomenon that the normal vector angles between two planes are equal, establish a method for quickly solving the camera focal length, and finally convert the corresponding relationship of the straight lines into the corresponding relationship of the space points between two coordinate systems with the same origin to solve the camera posture. Compared with the existing straight line-based calibration method, the application reduces the number of straight lines, shortens the calibration time, has a unique solution, and can simultaneously solve the focal length.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, SLAM and photogrammetry, and particularly relates to a camera focal length and pose calibration method based on two straight lines. BACKGROUND

[0002] In the field of computer vision, SLAM, photogrammetry and the like, in order to measure and estimate, the focal length and pose and other internal and external parameters of a camera need to be calibrated. There are many internal and external parameter calibration methods at present, such as an external parameter calibration method for a camera with known internal parameters or ideal internal parameters [A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation]; an internal and external parameter calibration method for a camera with known partial internal parameters [A general and simple method for camera pose and focal length determination]; and an internal and external parameter calibration method for a camera with known partial pose information [Closed-form solutions to minimal absolute pose problems with known vertical direction], [A Novel Method for Intrinsic and Extrinsic Parameters Estimation by Solving Perspective-Three-Point Problem with Known Camera Position]. With the development of positioning technology, positioning devices are gradually miniaturized, cheapened and meet the engineering requirements in precision, and are beginning to be widely used in camera positioning to obtain the position of a camera. The present application calibrates a camera in the case that the position of the camera is known and the focal length is unknown, including the focal length and pose and other parameters.

[0003] Currently, according to different external features used in calibration, methods for camera calibration are mainly divided into two categories based on feature lines and feature points, and the present application belongs to the former. According to the camera imaging principle, a straight line can bring two constraints, therefore, in the absence of prior knowledge, if the camera external parameters need to be calibrated, at least three straight lines are needed. Literature [A novel algebraic solution to the perspective-three-line pose problem] proposes a camera external parameter calibration method based on three straight lines, which can get up to 8 solutions, and additional constraints are needed to determine a unique solution. If part of the internal parameters is unknown, such as the focal length in many cases where the camera uses a zoom lens, then at least four straight lines are needed for camera calibration in this case. As can be seen, the current camera calibration method based on straight lines needs at least three straight lines, and has the phenomenon of multiple solutions; in order to solve part of the internal parameters, at least four straight lines are needed. SUMMARY

[0004] The purpose of the present application is to provide a camera focal length and pose calibration method based on two straight lines, to overcome the technical problem of multiple solutions in camera parameter calibration based on straight lines, and to minimize the number of straight lines needed for camera parameter calibration, and to overcome the defect of multiple solutions, further expanding the use of camera calibration methods.

[0005] To achieve the above purpose and solve the above technical problems, the technical scheme of the present application is as follows:

[0006] A camera focal length and pose calibration method based on two straight lines, comprising the following steps:

[0007] Two straight lines L1 and L2 in space and camera position O c ,

[0008] Construct a world coordinate system S w (O w _X w Y w Z w ) and a camera coordinate system S c (O c _X c Y c Z c );

[0009] Step 1, calibration of camera focal length

[0010] In the world coordinate system S w (O w _X w Y w Z w ), straight lines L iThe expression for (i = 1, 2) is (V i P i ),V i Let P be the unit direction vector of the line. i Let L be any point on the line. i It can be represented as

[0011] L i =P i +kV i (1)

[0012] Where k is an arbitrary scale factor; the world coordinate system S w (O w _X w Y w Z w Below, through the straight line L i and camera optical center O c plane π i The unit normal vector of (i = 1, 2) is

[0013]

[0014] The angle α between the normal vectors of plane π1 and π2 is

[0015]

[0016] Assume line L i The corresponding imaging line l i The pixel coordinates of the two endpoints are (u 2i-1 v 2i-1 ),(u 2i v 2i ), known; the camera focal length in pixels is f, unknown;

[0017] Then the camera coordinate system S c (O c _X c Y c Z c The normal vectors of π1 and π2 are:

[0018]

[0019] Since the included angles of the normal vectors are equal, then

[0020]

[0021] Let cosα = m5, then

[0022]

[0023] f 2If we consider it as a parameter, then the above equation is about f. 2 The quadratic equation in one variable has two solutions, according to f 2 If all f are greater than zero, then a unique solution for the focal length f is obtained.

[0024] Step 2: Camera attitude calibration

[0025] After obtaining the focal length f, the camera coordinate system S is calculated using formula (7). c (O c _X c Y c Z c The two planes below π i The unit normal vector of (i = 1, 2)

[0026]

[0027] At this time, the unit normal vector n w1 ,n w2 ,n c1 ,n c2 Given that everything is known, based on the camera rotation matrix, we can obtain...

[0028]

[0029] R is the camera coordinate system S c (O c _X c Y c Z c ) and the world coordinate system S w (O w _X w Y w Z w The rotation matrix between points P and P in the world coordinate system represents the camera pose to be determined. w In the camera coordinate system

[0030] P c =R·P w +t (9)

[0031] t is the camera coordinate system S c (O c _X c Y c Z c ) and the world coordinate system S w (O w _X w Y w Z w The translation vector between the world coordinate system and the camera coordinate system, if the world coordinate system is translated to the origin of the camera coordinate system, i.e., t = 0, then

[0032] Pc = R - P w (10)

[0033] Then the relationship between formula (10) and the unit normal vector above is consistent with formula (8);

[0034] Thus a new world coordinate system S w2 (O w2 _X w2 Y w2 Z w2 ) is established, that is, the original world coordinate system S w is translated to the original point of the camera coordinate system, and the relationship between the two coordinate systems is as follows:

[0035] S w2 = S w -O c (11)

[0036] Two space points are established, P c and P c1 in the camera coordinate system S c2 , and P ci = n ci ; P w2 and P w1 in the world coordinate system S w2 , and P wi = n wi ;

[0037] A new camera coordinate system S c2 (O c2 _X c2 Y c2 Z c2 ) is established, and the original point O c2 of the coordinate system is located at the point O c , and the unit direction vectors of each axis are as follows.

[0038]

[0039] The camera coordinate system S c (O c _X c Y c Z c ) is converted to the camera coordinate system S c2 (O c2 _X c2 Y c2 Z c2 ), as follows

[0040]

[0041] A new world coordinate system S w3 (Ow3 _X w3 Y w3 Z w3 ), the coordinate system origin O w3 is located at point O c , the unit direction vectors of each axis are as follows:

[0042]

[0043] Coordinate system S w2 (O w2 _X w2 Y w2 Z w2 ) is converted to S w3 (O w3 _X w3 Y w3 Z w3 ) as follows:

[0044]

[0045] Since the camera coordinate system and the two points P c1 and P c2 are the same as the two points P w1 and P w2 in the world coordinate system, the coordinate system S c2 (O c2 _X c2 Y c2 Z c2 ) and S w3 (O w3 _X w3 Y w3 Z w3 ) are the same coordinate system, the relationship between the original world coordinate system S w (O w _X w Y w Z w ) and the camera coordinate system S c (O c _X c Y c Z c ) is shown in equation (16):

[0046]

[0047] Up to now, the calibration of the camera pose is completed.

[0048] Compared with the prior art, the effective benefits of the present application relative to the prior art are:

[0049] 1. The method solves the problem that the existing linear camera pose calibration algorithm needs at least 3 lines, and has multiple solutions, and needs to calibrate part of the parameters, such as focal length, and needs 4 lines.

[0050] 2. Compared with the camera pose calibration method based on 3 lines, the present application does not involve nonlinear iterative algorithm, so that the accuracy and efficiency are improved.

[0051] 3. The present application is suitable for the application scenarios where the camera position can be obtained in advance, the focal length is unknown, and the number of available line features is small. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 It is an imaging diagram of two space lines in the camera of the present application;

[0053] Figure 2 It is a diagram showing the rotation relationship between the unit normal vectors is transformed into the rotation relationship between the space points;

[0054] Figure 3 It is a diagram showing the establishment of a new camera coordinate system and a world coordinate system;

[0055] Figure 4 It is a diagram showing the conversion relationship between the coordinate systems;

[0056] Figure 5 It is a diagram showing the simulation imaging of two space lines. DETAILED DESCRIPTION

[0057] The present application will be explained and described in detail below in combination with the drawings and examples.

[0058] In order to overcome the multiple solution problem of camera pose calibration, minimize the number of lines, and expand the use scenarios of the calibration method, the present application proposes a camera focal length and pose calibration method based on two lines. The present application only needs to use 2 lines and the known camera position, and can solve the focal length and pose of the camera through geometric constraints and geometric rotation, and has a unique solution without multiple solution problem. The present application is suitable for application scenarios where the camera position can be obtained in advance, and the number of available lines in the camera field of view is small.

[0059] The design idea of the present application is described as follows: the present application calibrates the focal length and attitude of a camera according to two known straight lines and the camera position. The present application solves the technical problem of the traditional algorithm of this type which requires ≥3 straight lines and multiple solutions. The present application decomposes the calibration problem into two sub-problems for solving, which can simplify the technical problem and make the solving more efficient. The first problem is to solve the focal length, and the second problem is to solve the camera attitude. In the first problem, each straight line and the camera position can form a plane, so two identical planes can be established in the camera coordinate system and the world coordinate system respectively. Since the straight line and the camera position are known, the plane is known, and then the normal vector of each plane in the world coordinate system is known; while the normal vector of each plane in the camera coordinate system contains the unknown to be solved, the focal length. By using the geometric relationship that the angle between the normal vectors of the two planes in the two coordinate systems is equal, a polynomial containing only the focal length is established, and the unique solution is obtained by solving the polynomial while using the condition that the focal length is greater than zero. The solving process does not contain nonlinear iterative solving, so the calculation efficiency is high. In the second problem, the focal length obtained from the first problem can obtain the unit normal vectors of the two planes formed by the two straight lines and the camera position in the camera coordinate system, and the unit normal vectors of the two planes in the world coordinate system. The unit normal vectors of each plane in the two coordinate systems can be established through the rotation matrix of the camera model, and then the two equations are regarded as the corresponding equations of two landmarks with the same origin in the camera coordinate system and the world coordinate system. By using this clever conversion, we convert the straight line feature to the point feature when solving the camera attitude, and use the camera attitude solving method of the point feature to quickly solve the camera attitude.

[0060] The present application proposes a camera focal length and attitude calibration method based on camera position and two straight lines, which only needs to know two straight lines in space, measure the camera position and the position of the two straight lines, shoot the straight line feature image, obtain the imaging relationship of the straight line in the camera, and use the important geometric phenomenon that the angle between the normal vectors of the two planes is equal to establish a method for quickly solving the focal length of the camera; finally, the corresponding relationship of the straight line is converted into the corresponding relationship of the space points between the two coordinate systems with the same origin to solve the camera attitude. Compared with the existing straight line-based calibration method, the present application reduces the number of straight lines, shortens the calibration time, has a unique solution, and can simultaneously solve the focal length and other advantages.

[0061] To achieve the above object, the present application adopts the following technical scheme:

[0062] Two straight lines L1, L2 and camera position O in space are known c The planes π1, π2 in the world coordinate system S w can be determined. The imaging straight lines l1, l2 in the image and the camera position O c can also determine the camera coordinate system Sc planes π1, π2 as shown in Figure 1 Using this property, two intermediate coordinate systems are established, and the camera pose and focal length can be solved.

[0063] world coordinate system S w (O w _X w Y w Z w ) known line L i (i = 1, 2) and camera position O c Two planes π1, π2 can be obtained. Since the line and the camera position are known, the two planes are known, and then two normal vectors n w1 , n w2 can be obtained. Similarly, by extracting the corresponding lines l1, l2 in the image plane, each line and the camera position can determine two planes π1, π2, and then we can obtain two normal vectors n c (O c _X c Y c Z c ) in the camera coordinate system S c1 , n c2 , but both contain an unknown focal length variable. The angle between the two normal vectors in the camera coordinate system is equal to the angle between the two normal vectors in the world coordinate system, so an equation can be established according to this relationship to solve the focal length. Then, according to n ci = Rn wi , a plurality of equations are established. According to this rule, it can be considered that the point correspondence n ci and n wi between the camera coordinate system and the world coordinate system established with the camera position as the origin. The specific scheme is as follows.

[0064] Step 1, camera focal length calibration

[0065] In the world coordinate system S w , the expression of the line L i (i = 1, 2) is (V i , P i ), V i is the unit direction vector of the line, P i is any point on the line, and L i can be expressed as

[0066] L i = P i +kV i (1)

[0067] Here, k is an arbitrary scale factor. In the world coordinate system S w , the line Li and the camera optical center O c The plane π i The unit normal vector of the plane π

[0068]

[0069] The angle α between the normal vectors of the planes π

[0070]

[0071] Assume the straight line L i The corresponding imaging straight line l i The pixel coordinates of the two endpoints are (u 2i-1 v 2i-1 ), (u 2i v 2i ), which are known; the camera focal length f in pixels is unknown; then the camera coordinate system S c The normal vectors of the planes π

[0072]

[0073] Since the angles between the normal vectors are equal, we have

[0074]

[0075] Let cos α = m

[0076]

[0077] Take f 2 as a parameter, then the above equation is a quadratic equation in f 2 , which has two solutions. According to the constraint condition that f 2 , f are both greater than zero, we can obtain a unique solution for the focal length f.

[0078] Step 2, pose calibration of the camera

[0079] After solving the focal length, we calculate the unit normal vectors of the two planes in the camera coordinate system using the following formula

[0080]

[0081] At this time, the unit normal vectors n w1 , n w2 , n c1 , n c2 are all known, and according to the camera rotation matrix, we can obtain

[0082]

[0083] R is the camera coordinate system S cWith world coordinate system S w The rotation matrix between points is the pose to be determined in this invention. And a point P in the world coordinate system... w In the camera coordinate system

[0084] P c =R·P w +t (9)

[0085] t is the camera coordinate system S c With world coordinate system S w The translation vector between the world coordinate system and the camera coordinate system. If the world coordinate system is translated to the origin of the camera coordinate system, i.e., t = 0, then...

[0086] P c =R·P w (10)

[0087] Equation (10) is consistent with the relationship between unit normal vectors in equation (8) above. Inspired by this relationship, this invention considers the rotation relationship between unit normal vectors as the transformation relationship of points after the origins of the camera coordinate system and the world coordinate system coincide, and the coordinates corresponding to the points are the unit normal vectors in that coordinate system, such as... Figure 2 As shown.

[0088] Here, we establish a new world coordinate system S. w2 The original world coordinate system S w The origin is translated to the origin of the camera coordinate system, and the relationship between the two coordinate systems is as follows.

[0089] S w2 =S w -O c (11)

[0090] Now, in the camera coordinate system, two points P c1 and P c2 , respectively with two points P in the world coordinate system w1 and P w2 For points that are the same, their coordinates are the unit normal vectors in that coordinate system. Then we establish a new camera coordinate system S. c2 and world coordinate system S w3 ,like Figure 3 As shown.

[0091] Establish a new camera coordinate system S c2 (O c2 _X c2 Y c2 Z c2 O, the origin of the coordinate system c2 Located at point O c The unit direction vectors for each axis are as follows.

[0092]

[0093] Camera coordinate system S c (O c _X c Y c Z c ) is converted to camera coordinate system S c2 (O c2 _X c2 Y c2 Z c2 ), as follows

[0094]

[0095] A new world coordinate system S w3 (O w3 _X w3 Y w3 Z w3 ) is established, with coordinate system origin O w3 located at point O c , and the unit directional vectors of the axes as follows.

[0096]

[0097] Coordinate system S w2 (O w2 _X w2 Y w2 Z w2 ) is converted to S w3 (O w3 _X w3 Y w3 Z w3 ) as follows.

[0098]

[0099] Since the two points P c1 and P c2 in the camera coordinate system are the same as the two points P w1 and P w2 in the world coordinate system, respectively, the coordinate systems S c2 (O c2 _X c2 Y c2 Z c2 ) and S w3 (O w3 _X w3 Y w3 Z w3 ) are the same coordinate system. Now, the conversion relationship between the coordinate systems is as shown in Figure 4 .

[0100] The original world coordinate system S w and the camera coordinate system Sc The relationship is as follows.

[0101]

[0102] Now, the pose calibration of the camera is completed.

[0103] Embodiment 1

[0104] Given two straight lines, the unit direction vectors thereof in the world coordinate system are (-0.4343 -0.2388 -0.8685) and (0.3288 -0.8054 0.4931) respectively, and pass through (10 -1 -30) and (-30 1 10) respectively. The camera position O c (10, -5, 15). The camera focal length is set to 50mm; the remaining camera internal parameters are known, and the camera theoretical extrinsic parameter rotation matrix is set as follows:

[0105]

[0106] The imaging of the two straight lines in the camera is as shown in Figure 5 .

[0107] In the simulation, 0.1 pixel error is added to the image feature extraction. By using the method of the present application, the focal length is calculated to be 50.3mm, and the error is 0.3mm. The measured result of the extrinsic parameter rotation matrix is:

[0108]

[0109] The conversion to the pose angle error is 0.09°. It can be seen that the errors of the focal length and the pose are very small, indicating that the present application has high calibration precision for the focal length and the pose of the camera.

Claims

1. A method for calibrating camera focal length and attitude based on two straight lines, characterized in that, Includes the following steps: Given two straight lines in space L 1, L 2 and camera position O c , Constructing a world coordinate system S w ( O w _X w Y w Z w ) and camera coordinate system S c ( O c _X c Y c Z c ); Step 1: Estimating the camera's focal length World coordinate system S w ( O w _X w Y w Z w (below) straight line L i The expression is ( V i , P i ), V i Let be the unit direction vector of the line, where i =1,2, P i Let be any point on the line. L i Represented as （1） in, k Arbitrary scale factor; world coordinate system S w ( O w _X w Y w Z w Below, through the straight line L i and camera optical core O c plane π i The unit normal vector is （2） in, i =1,2; The angle between the normal vectors of plane π1 and π2 for （3） Assume a straight line L i Corresponding imaging line l i The pixel coordinates of the two endpoints are , Given: The camera focal length in pixels is f ,unknown; Then the camera coordinate system S c ( O c _X c Y c Z c The normal vectors of π1 and π2 are: （4） Since the included angles of the normal vectors are equal, then （5） make ,but （6） Will If we consider it as a parameter, then the above equation is about The quadratic equation in one variable has two solutions, according to If all values ​​are greater than zero, then the focal length is obtained. The only solution; Step 2, Camera pose estimation Obtain focal length Then, the camera coordinate system is calculated using formula (7). S c ( O c _X c Y c Z c The two planes below π i unit normal vector （7） in, i =1,2; At this time, the unit normal vector Given that everything is known, based on the camera rotation matrix, we can obtain... （8） R Camera coordinate system S c ( O c _X c Y c Z c ) and world coordinate system S w ( O w _X w Y w Z w The rotation matrix between the coordinates is the camera pose to be determined; and the point in the world coordinate system is... P w In the camera coordinate system （9） t Camera coordinate system S c ( O c _X c Y c Z c ) and world coordinate system S w ( O w _X w Y w Z w The translation vector between the world coordinate system and the camera coordinate system, if the world coordinate system is translated to the origin of the camera coordinate system, i.e. t =0, then （10） Formula (10) is consistent with the relationship between the unit normal vectors above (8); This established a new world coordinate system. S w2 ( O w2 _ X w2 Y w2 Z w2 ), the original world coordinate system S w The origin is translated to the origin of the camera coordinate system, and the relationship between the two coordinate systems is as follows: （11） Establish two spatial points, camera coordinate system S c ( O c _X c Y c Z c The two spatial points below are P c1 and P c2 ,and World coordinate system S w2 The following two spatial points are P w1 and P w2 ,and ; Establish a new camera coordinate system S c2 ( O c2 _ X c2 Y c2 Z c2 Origin of coordinate system O c2 Located at point O c The unit direction vectors for each axis are as follows: （12） camera coordinate system S c ( O c _X c Y c Z c Transform to camera coordinate system S c2 ( O c2 _ X c2 Y c2 Z c2 ),as follows （13） Establish a new world coordinate system S w3 ( O w3 _ X w3 Y w3 Z w3 Origin of coordinate system O w3 Located at point O c The unit direction vectors for each axis are as follows: （14） coordinate system S w2 ( O w2 _ X w2 Y w2 Z w2 ) Convert to S w3 ( O w3 _ X w3 Y w3 Z w3 As shown below: （15） Due to the camera coordinate system S c The two points below P c1 and P c2 Respectively with the world coordinate system S w2 Next two points P w1 and P w2 Since they are the same points, the coordinate system S c2 ( O c2 _ X c2 Y c2 Z c2 )and S w3 ( O w3 _ X w3 Y w3 Z w3 If they are in the same coordinate system, then the original world coordinate system is... S w ( O w _X w Y w Z w ) and camera coordinate system S c ( O c _X c Y c Z c The relationship between them is shown in formula (16): （16） At this point, the camera pose estimation is complete.

Images