A panoramic camera calibration method

Abstract

The application provides a panoramic camera calibration method, comprising the following steps: placing an identification object on each side of a vehicle, the identification object comprising a black rectangular frame and a blank area in the rectangular frame, four corner points being arranged on the blank area, obtaining the image coordinates of the vertical line and the corner points of the identification object through camera recognition, and then calculating the external parameters of the camera. The panoramic camera calibration method is more convenient and faster than the existing calibration method, and has low requirements for the operation proficiency and accuracy of the implementer, and is not prone to operation errors, thereby improving the splicing accuracy of the panoramic camera.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera calibration, in particular to a panoramic camera calibration method. BACKGROUND

[0002] With the rapid development of computer vision and image technology, more and more technologies are applied to the field of automobile electronics, and the panoramic vision auxiliary system is one of the important applications. The panoramic vision system needs to perceive the 360° environment, so multiple vision sensors cooperate with each other to form a complete set of video images around the vehicle. The panoramic system needs to be calibrated before leaving the factory to ensure the accuracy and precision of the panoramic image display. At present, most of the panoramic calibration patterns use black and white patterns (checkerboard patterns), which need to meet the following conditions:

[0003] 1) Generally four calibration cloths are needed, one in front, one in back, one on the left and one on the right;

[0004] 2) The vehicle placement position and direction need to be relatively accurate;

[0005] 3) The size of each grid of the checkerboard pattern needs to be accurate;

[0006] In view of the above conditions, the operation proficiency and accuracy of the implementer are required to be high, and when the operation error occurs, the splicing accuracy will be affected. SUMMARY

[0007] The technical problem to be solved by the present application is that the calibration method of the vehicle camera in the prior art requires high operation proficiency and accuracy of the implementer, and when the operation error occurs, the splicing accuracy will be affected. The present application provides a panoramic camera calibration method to solve the above problems.

[0008] The technical solution adopted by the present application to solve the technical problem is: a panoramic camera calibration method, comprising the following steps:

[0009] S1, placing one block of marker on each side of the vehicle, the marker comprising a black rectangular frame and a blank area inside the rectangular frame, four corner points are provided on the blank area, the four corner points are distributed in a rectangular shape and located at the four corners of the rectangle, and the long sides of the two markers are parallel and the short sides are collinear;

[0010] S2, obtaining the image coordinates of the vertical lines and the corner points of the marker through the camera recognition, the vertical lines being the straight lines where the long sides of the marker are located;

[0011] S3, obtaining the image coordinates of the vanishing point, the vanishing point being the intersection of the four vertical lines, and calculating the yaw angle and the pitch angle of the front camera and the rear camera through the image coordinates of the vanishing point;

[0012] S4, calculating the roll angle and height of the front camera and the rear camera according to the yaw angle and the pitch angle of the front camera and the rear camera;

[0013] S5, calculating the inside distance of the two markers according to the short side width of the marker, the yaw angle and the pitch angle of the front camera and the rear camera, so as to obtain the world coordinates of the corner point, and obtaining the yaw angle, the pitch angle, the roll angle and the height of the left camera and the right camera according to the corresponding relationship between the world coordinates of the corner point and the image coordinates.

[0014] Specifically, in step S3:

[0015] According to the conversion relationship between the image coordinates P(u, v) and the world coordinates (X w ,Y w ,Z w ) of the vanishing point, the following equation is obtained:

[0016]

[0017] Where λ is a scale factor, M intrinsical is an intrinsic matrix, M extrinsical is an extrinsic matrix, P world is a point in the world coordinate system; H 3*4 =M intrinsical *M extrinsical ;

[0018]

[0019]

[0020] Where d x and d y are pixel sizes, (u0, v0) is the image center coordinate, f x =f / d x , f y =f / d y are the normalized focal lengths of the X axis and the Y axis, R is a rotation matrix, and T is a translation matrix;

[0021]

[0022] Where pitch is the pitch angle, yaw is the yaw angle, and roll is the roll angle; c1 is cos(pitch), c2 is cos(roll), c3 is cos(yaw), s1 is sin(pitch), s2 is sin(roll), and s3 is sin(yaw); for a point on the ground Z w =0, assuming the roll angle roll=0, the following equation is obtained:

[0023]

[0024] In the world coordinate system, according to the characteristics of the vanishing point, X w = infinity, Y w = 0, the following equation is obtained:

[0025] H(3, 1)u-H(1, 1) = 0

[0026] H(3, 1)v-H(2, 1) = 0

[0027] It is calculated that:

[0028] pitch = atan ((v0-v) / f y ), yaw = atan ((u-u0)*atan (pitch) / f x ).

[0029] As preferred, the following steps are further included:

[0030] S6, obtaining the extrinsic parameters of the camera in the vehicle coordinate system according to the extrinsic parameters of the camera in the world coordinate system and the installation information of the camera.

[0031] The present application has the beneficial effect that the panoramic camera calibration method is more convenient and fast than the existing calibration method, and the operation proficiency and precision of the implementer are not high, and operation errors are not easy to occur, thereby improving the splicing precision of the panoramic camera. BRIEF DESCRIPTION OF DRAWINGS

[0032] The present application will be further described below in conjunction with the drawings and examples.

[0033] Figure 1 is a flowchart of an optimal embodiment of a panoramic camera calibration method of the present application.

[0034] Figure 2 is a placement schematic view of a marker and a vehicle according to a panoramic camera calibration method of the present application.

[0035] Figure 3 is a schematic view of a vertical line according to a panoramic camera calibration method of the present application. DETAILED DESCRIPTION

[0036] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0037] Any processes or method descriptions described in flow charts or otherwise herein represent embodiments of processes that can be employed, and the claimed subject matter contemplates other implementations that can be derived from such process descriptions. The claimed subject matter contemplates embodiments in which the order of execution of the steps has been changed relative to the order of execution shown in the flow charts described herein, where the specific order of execution is not critical. Thus, but for specific instances, the order or execution can be varied relative to that shown herein.

[0038] As shown in Figure 1 , the present application provides a panoramic camera calibration method, comprising the following steps:

[0039] S1, as shown in Figure 2 , a piece of identification object is placed on each side of the vehicle, the identification object includes a black rectangular frame and a blank area inside the rectangular frame, four "cross" corner points are provided on the blank area, the four corner points are distributed in a rectangular shape and located at the four corners of the rectangle, the long sides of the two identification objects are parallel and the short sides are collinear; the identification objects are arranged on both sides of the vehicle, in this example, the distance between the two identification objects and the specific parking position of the vehicle are not strictly fixed, as long as the original drawing meets the requirements of camera shooting and subsequent processing. The vehicle is placed as much as possible in the middle position between the two identification objects, which can ensure that there is a complete pattern in the four original drawings, the pattern is clear and complete, and the cumulative error in the calculation process is also reduced.

[0040] S2, the pattern is shot by the camera, then the linear profile of the identification object is recognized by the openCV vision operator, the straight line point set is selected from the linear profile, and the vertical line is obtained by straight line fitting, the vertical line is the straight line where the long side of the identification object is located. The image coordinates of the corner points can also be recognized by the openCV vision operator.

[0041] S3, as shown in Figure 3 , the image coordinates of the vanishing point are obtained according to the four vertical lines, the vanishing point is the intersection of the four vertical lines, the yaw angle and the pitch angle of the front camera and the rear camera are calculated through the image coordinates of the vanishing point:

[0042] According to the conversion relationship between the image coordinates P(u,v) of the vanishing point and the world coordinates (X w ,Y w ,Z w ), the following equation is obtained:

[0043]

[0044] Where λ is the proportionality coefficient, M intrinsical is the intrinsic matrix, M extrinsical is the extrinsic matrix, and P world is the point in the world coordinate system; H 3*4= M intrinsical * M extrinsical ;

[0045]

[0046]

[0047] where d x and d y are the size of the image element, (u0, v0) is the image center coordinate, f x = f / d x , f y = f / d y are the normalized focal length of X axis and Y axis, R is the rotation matrix, T is the translation matrix.

[0048]

[0049]

[0050] where pitch is the pitch angle, yaw is the yaw angle, roll is the roll angle; c1 is cos(pitch), c2 is cos(roll), c3 is cos(yaw), s1 is sin(pitch), s2 is sin(roll), s3 is sin(yaw); the pitch angle, the yaw angle and the roll angle are the extrinsic parameters of the camera, which are also the parameters in the rotation matrix. The parameters in the translation matrix include the horizontal position and the height, the horizontal position is known, so only the height of the camera needs to be obtained to get the translation matrix, which is also the extrinsic parameter of the camera.

[0051] For the point Z w on the ground, Z w = ∞, Z w = 0, assuming the roll angle roll = 0, the following equation is obtained:

[0052]

[0053] In the world coordinate system, according to the characteristics of the vanishing point, X w = ∞, Y w = 0, the following equation is obtained:

[0054] H(3, 1)u - H(1, 1) = 0

[0055] H(3, 1)v - H(2, 1) = 0

[0056] It is calculated that:

[0057] pitch = atan((v0 - v) / f y ), yaw = atan((u - u0) * atan(pitch) / f x ).

[0058] S4, calculating the roll angle and height of the front camera and the rear camera according to the yaw angle and the pitch angle of the front camera and the rear camera;

[0059] S5, calculating the inside distance of the two markers according to the short side width of the marker, the yaw angle and the pitch angle of the front camera and the rear camera, so as to obtain the world coordinates of the corner point, and using openCV, the yaw angle, the pitch angle, the roll angle and the height of the left camera and the right camera can be obtained according to the corresponding relationship between the world coordinates of the corner point and the image coordinates.

[0060] S6, obtaining the extrinsic parameters of the camera in the vehicle coordinate system according to the extrinsic parameters of the camera in the world coordinate system and the installation information of the camera.

[0061] S7, projecting the straight lines in the patterns of the front, rear, left and right four paths into the world coordinate system, and calibrating the extrinsic parameters of the camera in the left and right directions according to the position relationship of the straight lines.

[0062] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0063] Based on the above ideal embodiments according to the present application, through the above description, relevant personnel can make various changes and modifications without deviating from the technical idea of the present application. The technical scope of the present application is not limited to the content of the specification, and must be determined according to the scope of the claims.

Claims

1. A panoramic camera calibration method, characterized in that, The method comprises the following steps: S1, placing one piece of marker on each side of the vehicle, the marker comprising a black rectangular frame and a blank area in the rectangular frame, four corner points being arranged on the blank area in a rectangular distribution and at four corners of the rectangle, long sides of the two markers being parallel and short sides being collinear; S2, obtaining image coordinates of vertical lines and corner points of the marker through camera recognition, the vertical lines being straight lines on which the long sides of the markers are located; S3, obtaining image coordinates of vanishing points, the vanishing points being intersection points of the four vertical lines, and calculating yaw and pitch angles of front and rear cameras through the image coordinates of the vanishing points; S4, calculating roll and height of the front and rear cameras according to the yaw and pitch angles of the front and rear cameras; S5, calculating the inside distance between the two markers according to the short side width of the marker, the yaw and pitch angles of the front and rear cameras, and obtaining world coordinates of the corner points, and obtaining yaw, pitch, roll and height of left and right cameras according to the corresponding relationship between the world coordinates and image coordinates of the corner points.

2. The panoramic camera calibration method according to claim 1, wherein in step S3: wherein pitch is the pitch angle, yaw is the yaw angle, and roll is the roll angle; c1 is cos(pitch), c2 is cos(roll), c3 is cos(yaw), s1 is sin(pitch), s2 is sin(roll), and s3 is sin(yaw); Based on the image coordinates P(u,v) and world coordinates (X) of the vanishing point w ,Y w Z w The transformation relationship yields the following equation: where λ is a scale factor, M intrinsical is an intrinsic matrix, M extrinsical is an extrinsic matrix, P world is a point in the world coordinate system; H 3*4 = M intrinsical * M extrinsical ; where d x and d y are the pixel size, (u0, v0) is the image center coordinate, f x = f / d x , f y = f / d y are the normalized focal lengths of the X and Y axes, R is a rotation matrix, and T is a translation matrix; H(3,1)u-H(1,1)=0 For a point Z on the ground w = 0, assuming a roll angle roll = 0 simplifies to the following equation: In the world coordinate system, according to the characteristics of the vanishing point, X w =∞, Y w =0, the following equation is obtained: H(3,1)v-H(2,1)=0 The calculation result is: The method further comprises the following step: pitch = atan((v0-v) / f y ), yaw = atan((u-u0)*atan(pitch) / f x ).

3. The method of claim 2, wherein, S6, obtaining the extrinsic parameters of the cameras in the vehicle coordinate system according to the extrinsic parameters of the cameras in the world coordinate system and the installation information of the cameras. ​

Images