Calibration method and system for visual camera used in large field of view optical measurement

Abstract

The present invention relates to the field of visual measurement technology, and provides a calibration method for a visual camera applied to large-field optical measurement, comprising the following steps: S1: setting two calibration points within the field of view of the visual camera; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera; S2: leveling the roll direction of the visual camera; S3: the visual camera captures an image, and simultaneously obtains the position information of the visual camera and the two calibration points; S4: obtaining the initial value of the visual camera extrinsic parameter according to the position information of the first calibration point and the position information of the visual camera; S5: obtaining the visual camera extrinsic parameter and the focal length of the camera through a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera extrinsic parameter. This scheme can reduce the number of points, reduce the difficulty of point distribution, and ensure the measurement accuracy within the field of view.

Description

Technical Field

[0001] The present invention relates to the field of visual measurement technology, and in particular to a calibration method and system for a visual camera applied to large-field optical measurement. Background Art

[0002] Camera calibration technology is a technology used for automated measurement and map modeling. It combines the camera image with high-precision ground actual parameters to accurately construct the geographic coordinates within the field of view. In order to calculate the position and direction of the camera, a perspective camera model is required for estimation. Common perspective models include 4-point model, 8-point model, etc.

[0003] The existing camera calibration methods mostly use chessboards, XOR or 4 or 6 or more marker points to calculate the position and attitude angle of the camera, but this method is limited in the case of a large field of view. When the field of view is greater than 5 meters, the chessboard shows its disadvantages. The processing difficulty and processing accuracy of the chessboard that is too large cannot be guaranteed. Similarly, the chessboard that is too large will cause great inconvenience during transportation and actual use. In a field of view greater than 10 meters, the camera calibration is generally achieved by arranging some scattered control points in the air. For general calibration, 4-8 points scattered in the field of view can calibrate the camera more ideally, but as the field of view increases further, if the field of view exceeds 50 meters, the arrangement of control points in the air becomes particularly difficult. At the height level, if you want to arrange points above 5 meters, you need to rely on other surrounding buildings, and the requirements for environmental conditions are more stringent. When the field of view is greater than 1 kilometer, each additional point requires more manpower and material resources, and the difficulty of installation also increases sharply. Therefore, the number of landmarks used for calibration plays a crucial role in the difficulty of calibration for large fields of view.

[0004] With a large field of view at the level of hundreds of meters or even kilometers, how to minimize the number of points, reduce the difficulty of point placement, lower the height of point placement, and ensure the measurement accuracy within the field of view is a technical problem that needs to be solved urgently.

[0005] Therefore, it is necessary to provide a calibration method and system for a visual camera used for large field of view optical measurement, so as to reduce the number of points, reduce the difficulty of point arrangement, and ensure the measurement accuracy within the field of view.

[0006] The above information disclosed in the Background section is only for enhancement of understanding of the background of the present application and therefore it may contain information that does not constitute the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0007] The main purpose of the present invention is to overcome the problems of difficulty in arranging calibration points and inaccurate measurement in large-field-of-view optical measurement, and to provide a calibration method and system for a visual camera applied to large-field-of-view optical measurement, so as to reduce the number of points, reduce the difficulty of point arrangement, and ensure the measurement accuracy within the field of view.

[0008] To achieve the above-mentioned purpose, the first aspect of the present invention provides a calibration method for a visual camera applied to large-field optical measurement, wherein the distance between the visual camera and the target to be measured is greater than 100 meters, comprising the following steps:

[0009] S1: two calibration points are set within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera;

[0010] S2: Level the roll direction of the visual camera;

[0011] S3: The visual camera captures an image and simultaneously obtains the position information of the visual camera and two calibration points;

[0012] S4: Obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera;

[0013] S5: According to the position information of the visual camera and the two calibration points, and the initial value of the visual camera external parameter, the visual camera external parameter and the focal length of the camera are obtained by a two-point calibration method.

[0014] As an exemplary embodiment of the present invention, in step S1, the calibration point is in a cylindrical shape, and two strip-shaped colors are arranged at intervals.

[0015] As an exemplary embodiment of the present invention, in step S2, leveling the visual camera in the rolling direction includes: adjusting the bubble bar until the bubble is in a horizontal position.

[0016] As an exemplary embodiment of the present invention, in step S3, obtaining the position information of the visual camera and the two calibration points includes: recording the position information of the visual camera and the two calibration points by RTK.

[0017] As an exemplary embodiment of the present invention, in step S4, obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera includes:

[0018] Calculate the angle between the line from the optical center to the image point and the optical axis;

[0019] The vertical distance from the optical center to the first calibration point is obtained according to the angle between the optical center and the image point and the optical axis, the position information of the first calibration point and the position information of the visual camera;

[0020] A rotation matrix is ​​obtained according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point;

[0021] Get the translation vector based on the rotation matrix.

[0022] As an example implementation of the present invention, the vertical distance from the first calibration point to the optical center is obtained according to the angle between the optical center and the image point and the optical axis, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point using the following formula:

[0023] ;

[0024] Among them, OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the horizontal angle between the line connecting the optical center to the image point and the optical axis, AngleV represents the vertical angle between the line connecting the optical center to the image point and the optical axis, and L represents the distance from the optical center to the first calibration point, which is obtained according to the position information of the first calibration point and the position information of the visual camera.

[0025] As an exemplary embodiment of the present invention, obtaining the rotation matrix according to the vertical distance from the optical center to the first calibration point, the angle between the line from the optical center to the image point and the optical axis, and the position information of the first calibration point includes:

[0026] The rotation matrix is ​​the rotation matrix from the world coordinate system to the camera coordinate system. The world coordinate system includes the X-axis, Y-axis, and Z-axis. First, rotate around the X-axis to obtain a 90-degree rotation angle around the X-axis, so that the Y-axis points downward; then rotate around the Y-axis so that the Z-axis points to coincide with the camera optical axis in the vertical direction, and finally rotate around the X-axis a second time so that the pitch angle of the visual camera is adjusted to completely coincide with the Z-axis and the camera optical axis;

[0027] The rotation matrix uses the following formula:

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] Where R represents the rotation matrix, AngleOX represents the first rotation angle around the X axis, AngleOY represents the rotation angle around the Y axis, and AngleOZx represents the second rotation angle around the X axis;

[0033] ;

[0034] ;

[0035] OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the horizontal angle between the optical center and the image point and the optical axis, and AngleV represents the vertical angle between the optical center and the image point and the optical axis; the coordinates of the first calibration point are (x0, y0, z0); the coordinates of the optical center of the visual camera are (x oc ,y oc , z oc ).

[0036] As an exemplary embodiment of the present invention, in step S5, the step of obtaining the visual camera extrinsic parameters and the focal length of the camera by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera extrinsic parameters includes:

[0037] The image point coordinates of the two calibration points after projection are obtained by back-projection according to the ideal internal parameters and the initial values ​​of the external parameters of the visual camera;

[0038] Iterate the equivalent focal length and image point coordinate relationship of the visual camera, the angle of rotation around the Z axis and the image point coordinate relationship for multiple times to obtain the equivalent focal length and the angle of rotation around the Z axis of the visual camera;

[0039] The external parameters of the visual camera, that is, the final rotation matrix and the final translation vector, are obtained according to the angle of rotation around the Z axis.

[0040] As an exemplary embodiment of the present invention, the relationship between the equivalent focal length and the image point coordinates of the visual camera is:

[0041] ;

[0042] The relationship between the angle of rotation around the Z axis and the image point coordinates is:

[0043] ;

[0044] Among them, F represents the equivalent focal length of the visual camera, k represents the number of iterations, AngleOZ represents the angle of rotation around the Z axis, the back-projected image point coordinates of the first calibration point are (ru0, rv0), the back-projected image point coordinates of the second calibration point are (ru1, rv1), u represents the horizontal coordinate, v represents the vertical coordinate, the image point coordinates of the first calibration point are (u0, v0), and the image point coordinates of the second calibration point are (u1, v1).

[0045] As a second aspect of the present invention, the present invention provides a calibration system for a visual camera used for large field of view optical measurement, wherein the distance between the visual camera and the target to be measured is greater than 100 meters, comprising:

[0046] Two calibration points are set within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera;

[0047] The visual camera is used to shoot two calibration points. The rolling direction of the visual camera is in a leveling state.

[0048] A position information acquisition module, used to obtain position information of two calibration points of the visual camera; and

[0049] The calibration module is used to obtain the initial value of the visual camera extrinsic parameter according to the position information of the first calibration point and the position information of the visual camera; the visual camera extrinsic parameter and the focal length of the camera are obtained by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera extrinsic parameter.

[0050] The advantages of the present invention are:

[0051] The present invention solves the problem of difficulty in camera calibration in the case of large-field-of-view optical measurement. Under the conditions of a measured distance greater than 100m and a field of view greater than 100m, only two points are used, and there is no requirement for the height of the points. It only needs to ensure that they are visible in the visual camera. The calibration of the camera extrinsic parameters is achieved through two simple calibration points, and the optimization of the equivalent focal length in the camera intrinsic parameters is achieved at the same time, which greatly reduces the difficulties in point arrangement during the calibration process and the difficulty in ensuring the accuracy of a large field of view. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and other objects, features and advantages of the present application will become more apparent by describing in detail the exemplary embodiments thereof with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the present application, and it is clear to a person skilled in the art that other accompanying drawings can be obtained from these accompanying drawings without creative effort.

[0053] Figure 1 The schematic diagram of the visual camera photographing the calibration points is shown schematically.

[0054] Figure 2 The diagram schematically shows the steps of a calibration method for a visual camera used for large field of view optical measurement.

[0055] Figure 3 The definition diagram of the coordinate system is schematically shown.

[0056] Figure 4 A schematic diagram of multiple calibration points is schematically shown. DETAILED DESCRIPTION

[0057] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0058] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0059] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0060] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0061] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the concepts of the present application. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0062] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present application, and therefore cannot be used to limit the scope of protection of the present application.

[0063] According to a first specific embodiment of the present invention, the present invention provides a calibration system for a visual camera used for large field of view optical measurement, wherein the distance between the visual camera and the target to be measured is greater than 100 meters, such as Figure 1As shown, it includes: two calibration points, a visual camera, a position information acquisition module and a calibration module.

[0064] A large field of view means that the distance between the visual camera and the target to be measured is greater than 100 meters. The distance between the target to be measured and the visual camera and the distance between the calibration point and the visual camera may not be equal, and the distance between the target to be measured may be 1 kilometer and the distance between the calibration point may be 100 meters. As a preferred embodiment, the distance between the target to be measured and the visual camera is greater than or equal to the distance between the calibration point and the visual camera.

[0065] Two calibration points are set within the field of view of the visual camera. The distance between the two calibration points and the visual camera is greater than 100 meters. As a preferred embodiment, the vertical distance between the two calibration points and the visual camera is greater than 100 meters. The two calibration points are respectively the first calibration point and the second calibration point. The first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera. The calibration points are cylindrical in shape, and two strips of color are set at intervals. As a preferred embodiment, Figure 1 As shown in the figure, the calibration point is a cylinder made by RTK with red and white stripes, 1 meter high and 0.5 meter in diameter. The red and white stripes can ensure that they are clearly visible even at a long distance. RTK can record the location information of the calibration point.

[0066] The visual camera is used to shoot two calibration points, and the rolling direction of the visual camera is in a leveling state.

[0067] The position information acquisition module (not shown in the figure) is used to obtain the position information of the two calibration points of the visual camera, and the position information of the visual camera and the two calibration points can be obtained from RTK.

[0068] The calibration module (not shown in the figure) is used to obtain the initial value of the visual camera extrinsic parameter based on the position information of the first calibration point and the position information of the visual camera; the visual camera extrinsic parameter and the focal length of the camera are obtained by a two-point calibration method based on the position information of the visual camera and the two calibration points and the initial value of the visual camera extrinsic parameter.

[0069] The system using this solution only needs two calibration points, which simplifies the calibration process.

[0070] As a second specific embodiment of the present invention, the present invention provides a calibration method for a visual camera applied to optical measurement of a large field of view, wherein the distance between the visual camera and the target to be measured is greater than 100 meters, and the calibration system for the visual camera applied to optical measurement of a large field of view of the first specific embodiment is adopted, such as Figure 2 As shown, the following steps are included:

[0071] S1: Two calibration points are set within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively the first calibration point and the second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera.

[0072] The field of view of the visual camera is divided into three equal parts, including the left area, the middle area and the right area from left to right. The height of each area is equal and the width is one third of the field of view width. The first calibration point is located in the middle area, close to the center of the field of view of the visual camera; the second calibration point is located in the left area on the left or the right area on the right, close to the left edge or right edge of the field of view of the visual camera.

[0073] The calibration point is cylindrical in shape, and two strip-shaped colors are arranged at intervals. As a preferred embodiment, Figure 1 As shown in the figure, the calibration point is a cylinder made by RTK with red and white stripes, 1 meter high and 0.5 meter in diameter. The red and white stripes can ensure that they are clearly visible even at a long distance. RTK can record the location information of the calibration point.

[0074] A large field of view means that the distance between the visual camera and the target to be measured is greater than 100 meters. The distance between the target to be measured and the visual camera and the distance between the calibration point and the visual camera may not be equal, and the distance between the target to be measured may be 1 kilometer and the distance between the calibration point may be 100 meters. As a preferred embodiment, the distance between the target to be measured and the visual camera is greater than or equal to the distance between the calibration point and the visual camera.

[0075] S2: Level the roll direction of the visual camera.

[0076] Leveling the visual camera in the roll direction includes: adjusting the bubble bar until the bubble is in a horizontal position.

[0077] S3: The visual camera captures an image and simultaneously obtains the position information of the visual camera and the two calibration points.

[0078] The obtaining of the position information of the visual camera and the two calibration points includes: recording the position information of the visual camera and the two calibration points through RTK.

[0079] S4: Obtain the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera.

[0080] The calibration principle is as follows:

[0081] The conversion formula from object space to image space is:

[0082] (1)

[0083] Among them, Z cis the projection of the distance from the object point to the optical center in the direction of the optical axis, u and v are the horizontal and vertical coordinates of the calibration point in the image (known quantities), respectively, F x 、F y is the equivalent focal length of the visual camera, C x , C y is the principal point parameter of the visual camera, where R represents the 3*3 rotation matrix of the three rotation angles Ax, Ay, and Az around the X, Y, and Z axes of the world coordinate system, T=[T x , T y , T z ] t is the translation vector, where t represents the transpose of the matrix.

[0084] The known conditions of the two-point calibration method are:

[0085] Known F x 、F y The initial value of F = F x =F y , identify C x , C y The ideal value remains unchanged.

[0086] C x = W / 2, C y = H / 2;

[0087] Where W represents the horizontal resolution of the image, and H represents the vertical resolution of the image.

[0088] F x and F y Respectively represent the equivalent focal length of the camera in the horizontal and vertical directions; the initial value is equal to f is the nominal focal length of the lens, in mm, du is the pixel size of the camera, in .

[0089] After the visual camera is leveled, the initial value of Az is considered to be 0 degrees.

[0090] So the known quantity is u n 、v n , C x , C y 、x n ,y n 、z n , n represents the point number, u and v represent the horizontal and vertical coordinates of the image point, x n ,y n 、z n Represents the object point coordinates of point. The quantities to be optimized are F, Ax, Ay, Az, Tx, Ty, Tz, among which the quantities to be optimized F and Az have initial values.

[0091] In addition, the location of the optical center of the visual camera is known, namely:

[0092] ;

[0093] x oc ,y oc , z oc Represents the coordinates of the optical center of the visual camera.

[0094] The method of obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera comprises:

[0095] S41: Calculate the angle between the line from the optical center to the image point and the optical axis.

[0096] ;

[0097] Where AngleU and AngleV represent the angles between the line from the optical center to the image point and the optical axis in the horizontal and vertical directions respectively; u0 represents the horizontal coordinate of the image point, and v0 represents the vertical coordinate of the image point.

[0098] S42: Obtain a vertical distance from the optical center to the first calibration point according to the angle between the optical center and the image point connecting line and the optical axis, the position information of the first calibration point, and the position information of the visual camera.

[0099] The vertical distance from the first calibration point to the optical center is obtained according to the angle between the optical center and the image point and the optical axis, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point using the following formula:

[0100] ;

[0101] Among them, OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the angle between the line connecting the optical center to the image point P and the optical axis Oc-Zc in the X direction, AngleV represents the angle between the line connecting the optical center to the image point and the optical axis in the Y direction, and L represents the distance from the optical center to the first calibration point, which is obtained according to the position information of the first calibration point and the position information of the visual camera.

[0102] like Figure 3 As shown, the coordinate system is defined as follows:

[0103] The world coordinate system is O-XYZ, the Z axis is perpendicular to the plane where the XY plane is located, and the camera coordinate system is Oc-x c y c z c , the pixel coordinates are u, v, where Oc represents the optical center of the camera, z c The axis is the optical axis of the camera, and the photographing direction is taken as the positive direction. x c , yc The axes are usually parallel to the two perpendicular sides of the imaging CCD plane.

[0104] x c The y axis represents the horizontal axis of the camera coordinate system. c The axis represents the vertical axis of the camera coordinate system, z c The axis is perpendicular to the XY plane; the pitch angle of the visual camera is z c The angle between the axis and the horizontal plane.

[0105] like Figure 3 As shown: o-xy represents the camera target surface coordinate system, Oc-x c y c z c represents the camera coordinate system, O-XYZ represents the world coordinate system, u and v represent the pixel horizontal coordinate and pixel vertical coordinate respectively. P represents the object point, and p is the projection of the object point P on the camera target surface, that is, the image point.

[0106] S43: Obtain a rotation matrix according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point.

[0107] The step of obtaining the rotation matrix according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point includes:

[0108] The rotation matrix is ​​the rotation matrix from the world coordinate system to the camera coordinate system. The world coordinate system includes the X-axis, Y-axis, and Z-axis. First, rotate around the X-axis to obtain a 90-degree rotation angle around the X-axis, so that the Y-axis points downward; then rotate around the Y-axis so that the Z-axis points to coincide with the camera optical axis in the vertical direction, and finally rotate around the X-axis a second time so that the pitch angle of the visual camera is adjusted to completely coincide with the Z-axis and the camera optical axis;

[0109] Among them, the X-axis represents the horizontal axis of the camera coordinate system, the Y-axis represents the vertical axis of the camera coordinate system, and the Z-axis is perpendicular to the plane where the XY axis is located; the pitch angle of the visual camera is the angle between the Z-axis and the horizontal plane;

[0110] The rotation matrix uses the following formula:

[0111] ;

[0112] ;

[0113] ;

[0114] ;

[0115] Where R represents the rotation matrix, AngleOX represents the first rotation angle around the X axis, AngleOY represents the rotation angle around the Y axis, and AngleOZx represents the second rotation angle around the X axis;

[0116] ;

[0117] ;

[0118] OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the horizontal angle between the optical center and the image point and the optical axis, and AngleV represents the vertical angle between the optical center and the image point and the optical axis; the coordinates of the first calibration point are (x0, y0, z0); the coordinates of the optical center of the visual camera are (x oc ,y oc , z oc ).

[0119] S44: Obtain a translation vector according to the rotation matrix.

[0120] The translation vector is obtained using the following formula:

[0121] ;

[0122] Among them, T is the translation vector, R is the rotation matrix, and the coordinates of the optical center of the visual camera are (x oc ,y oc , z oc ).

[0123] S5: According to the position information of the visual camera and the two calibration points, and the initial value of the visual camera external parameter, the visual camera external parameter and the focal length of the camera are obtained by a two-point calibration method.

[0124] The method of obtaining the visual camera external parameters and the focal length of the camera by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera external parameters includes:

[0125] S51: The image point coordinates of the two calibration points after projection are obtained through back projection according to the ideal internal parameters and the initial values ​​of the external parameters of the visual camera.

[0126] According to the ideal internal and external parameters, the coordinates of the projected image points are obtained by back projection, ru0, rv0, ru1, and rv1, which represent the back projection coordinates of the two calibration points respectively. ru0 and rv0 represent the back projection pixel coordinates u and v of the first calibration point (u represents the horizontal coordinate and v represents the vertical coordinate). Similarly, ru1 and rv1 represent the back projection coordinates of the second calibration point.

[0127] The internal parameters include Fx, Fy, Cx, and Cy; the external parameters include the rotation matrix R and the translation vector T.

[0128] S52: iterating and looping the equivalent focal length and image point coordinate relationship of the visual camera, and the angle of rotation around the Z axis and the image point coordinate relationship for multiple times, respectively, to obtain the equivalent focal length and the angle of rotation around the Z axis of the visual camera;

[0129] The relationship between the equivalent focal length and image point coordinates of the visual camera is:

[0130] ;

[0131] The relationship between the angle of rotation around the Z axis and the image point coordinates is:

[0132] ;

[0133] Among them, F represents the equivalent focal length of the visual camera, k represents the number of iterations, AngleOZ represents the angle of rotation around the Z axis, the back-projected image point coordinates of the first calibration point are (ru0, rv0), the back-projected image point coordinates of the second calibration point are (ru1, rv1), u represents the horizontal coordinate, v represents the vertical coordinate, the image point coordinates of the first calibration point are (u0, v0), and the image point coordinates of the second calibration point are (u1, v1).

[0134] The optimization iteration termination condition is rms < 0.001 or the number of iterations reaches 30.

[0135]

[0136] Among them, rms represents the reprojection error, (u0, v0) represents the pixel coordinates u and v of the first calibration point (u represents the horizontal coordinate and v represents the vertical coordinate), similarly (u1, v1) represents the pixel coordinates of the second calibration point, (ru0, rv0) represents the back-projection coordinates of the first calibration point, (ru1, rv1) represents the back-projection coordinates of the second calibration point, and k represents the number of iterations. The reprojection error rms can further verify the correctness of the calibration.

[0137] If the result does not converge, the iteration cannot be infinite, so an empirical value of 30 is set as the termination condition.

[0138] Each iteration, all angles (including AngleOZ, AngleOZx, AngleOY, AngleOX) are updated once.

[0139] S53: Obtain the external parameters of the visual camera according to the angle of rotation around the Z axis, that is, the final rotation matrix and the final translation vector.

[0140] After iteration, the external parameters of the visual camera are obtained according to the angle of rotation around the Z axis, and the new rotation matrix and translation vector are obtained:

[0141] ;

[0142] ;

[0143] At the end of the iteration, we get the final rotation matrix R* and translation vector T*. The final rotation matrix R* is R k , Roz k is the kth rotation matrix of the iteration. R* can be decomposed into A x , A y , A z Three Euler angles, T* =[T x , T y , T z ] t The translation vector, where t represents the transpose of the matrix, is obtained by referring to the formula of step S44 through the rotation matrix and the coordinates of the optical center of the visual camera.

[0144] The relationship between the rotation matrix R* and Ax, Ay, Az is:

[0145] .

[0146] like Figure 4 As shown in the figure, the visual camera shoots the calibration points. If the two-point calibration of this solution is adopted, the two points can be arranged close to the ground, such as points P0 and P1. If the four-point or six-point calibration of the existing technology is adopted, it is necessary to ensure that the arrangement height of the control points changes, such as points P2 and P3. If the ground is relatively empty and there are no buildings, it is very difficult to raise the points.

[0147] The measurement accuracy of this scheme is verified by binocular intersection, and the measurement accuracy can reach 3 / 1000, that is, when shooting a field of view of 1km, the measurement accuracy is 3m.

[0148] It can be seen from the above scheme that only two calibration points are needed to calibrate the visual camera. The present invention solves the problem that camera calibration is difficult in the case of large-field-of-view optical measurement. Under the conditions of a measured distance greater than 100m and a field of view greater than 100m, only two points are used. There is no requirement for the height of the points, as long as they are visible in the visual camera. The calibration of the camera's external parameters is achieved through two simple calibration points, and the optimization of the equivalent focal length in the camera's internal parameters is achieved at the same time, which greatly reduces the difficulties in point arrangement during the calibration process and the difficulty in ensuring the accuracy of a large field of view.

[0149] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structure, configuration or implementation method described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A calibration method for a visual camera used for large field of view optical measurement, wherein the distance between the visual camera and the target to be measured is greater than 100 meters, characterized in that: The following steps are involved: S1: two calibration points are set within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera; S2: Level the roll direction of the visual camera; S3: The visual camera captures an image and simultaneously obtains the position information of the visual camera and two calibration points; S4: Obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera; S5: Obtain the visual camera external parameters and the focal length of the camera through a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera external parameters; In step S4, obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera includes: Calculate the angle between the line from the optical center of the visual camera to the image point and the optical axis of the camera; the image point is the projection of the first calibration point on the camera target surface; The vertical distance from the optical center to the first calibration point is obtained according to the angle between the optical center and the image point and the optical axis, the position information of the first calibration point and the position information of the visual camera; A rotation matrix is ​​obtained according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point; Get the translation vector according to the rotation matrix; The vertical distance from the first calibration point to the optical center is obtained according to the angle between the optical center and the image point and the optical axis, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point using the following formula: ； Wherein, OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the horizontal angle between the optical center and the image point and the optical axis, AngleV represents the vertical angle between the optical center and the image point and the optical axis, and L represents the distance from the optical center to the first calibration point, which is obtained according to the position information of the first calibration point and the position information of the visual camera; The step of obtaining the rotation matrix according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point includes: The rotation matrix is ​​the rotation matrix from the world coordinate system to the camera coordinate system. The world coordinate system includes the X-axis, Y-axis, and Z-axis. First, rotate around the X-axis to obtain a 90-degree rotation angle around the X-axis, so that the Y-axis points downward; then rotate around the Y-axis so that the Z-axis points to coincide with the camera optical axis in the vertical direction, and finally rotate around the X-axis a second time so that the pitch angle of the visual camera is adjusted to completely coincide with the Z-axis and the camera optical axis; The rotation matrix uses the following formula: ； ； ； ； Where R represents the rotation matrix, AngleOX represents the first rotation angle around the X axis, AngleOY represents the rotation angle around the Y axis, and AngleOZx represents the second rotation angle around the X axis; ； ； OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the horizontal angle between the optical center and the image point and the optical axis, and AngleV represents the vertical angle between the optical center and the image point and the optical axis; the coordinates of the first calibration point are (x0, y0, z0); the coordinates of the optical center of the visual camera are (x oc ,y oc , z oc ); In step S5, the method of obtaining the visual camera external parameters and the focal length of the camera by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera external parameters includes: The image point coordinates of the two calibration points after projection are obtained by back-projection according to the ideal internal parameters and the initial values ​​of the external parameters of the visual camera; Iterate the equivalent focal length and image point coordinate relationship of the visual camera, the angle of rotation around the Z axis and the image point coordinate relationship for multiple times to obtain the equivalent focal length and the angle of rotation around the Z axis of the visual camera; The external parameters of the visual camera, that is, the final rotation matrix and the final translation vector, are obtained according to the angle of rotation around the Z axis.

2. The calibration method of a visual camera for large field of view optical measurement according to claim 1, characterized in that: In step S1, the calibration point is in the shape of a cylinder, and two strip-shaped colors are arranged at intervals.

3. The calibration method of a visual camera for large field of view optical measurement according to claim 1, characterized in that: In step S2, leveling the rolling direction of the visual camera includes: adjusting the bubble bar until the bubble is in a horizontal position.

4. The calibration method of a visual camera for large field of view optical measurement according to claim 1, characterized in that: In step S3, obtaining the position information of the visual camera and the two calibration points includes: recording the position information of the visual camera and the two calibration points through RTK.

5. The calibration method of a visual camera for large field of view optical measurement according to claim 1, characterized in that: The relationship between the equivalent focal length and image point coordinates of the visual camera is: ； The relationship between the angle of rotation around the Z axis and the image point coordinates is: ； Among them, F represents the equivalent focal length of the visual camera, k represents the number of iterations, AngleOZ represents the angle of rotation around the Z axis, the back-projected image point coordinates of the first calibration point are (ru0, rv0), the back-projected image point coordinates of the second calibration point are (ru1, rv1), the image point coordinates of the first calibration point are (u0, v0), and the image point coordinates of the second calibration point are (u1, v1).

6. A calibration system for a visual camera used for large field of view optical measurement, wherein the distance between the visual camera and the target to be measured is greater than 100 meters, characterized in that: include: Two calibration points are set within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera; The visual camera is used to shoot two calibration points. The rolling direction of the visual camera is in a leveling state. A position information acquisition module is used to obtain the position information of the visual camera and two calibration points; as well as The calibration module is used to obtain the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera; the visual camera external parameter and the focal length of the camera are obtained by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera external parameter; The method of obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera comprises: Calculate the angle between the line from the optical center of the visual camera to the image point and the optical axis of the camera; the image point is the projection of the first calibration point on the camera target surface; The vertical distance from the optical center to the first calibration point is obtained according to the angle between the optical center and the image point and the optical axis, the position information of the first calibration point and the position information of the visual camera; A rotation matrix is ​​obtained according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point; Get the translation vector according to the rotation matrix; The vertical distance from the first calibration point to the optical center is obtained according to the angle between the optical center and the image point and the optical axis, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point using the following formula: ； Wherein, OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the horizontal angle between the optical center and the image point and the optical axis, AngleV represents the vertical angle between the optical center and the image point and the optical axis, and L represents the distance from the optical center to the first calibration point, which is obtained according to the position information of the first calibration point and the position information of the visual camera; The step of obtaining the rotation matrix according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point includes: The rotation matrix is ​​the rotation matrix from the world coordinate system to the camera coordinate system. The world coordinate system includes the X-axis, Y-axis, and Z-axis. First, rotate around the X-axis to obtain a 90-degree rotation angle around the X-axis, so that the Y-axis points downward; then rotate around the Y-axis so that the Z-axis points to coincide with the camera optical axis in the vertical direction, and finally rotate around the X-axis a second time so that the pitch angle of the visual camera is adjusted to completely coincide with the Z-axis and the camera optical axis; The rotation matrix uses the following formula: ； ； ； ； Where R represents the rotation matrix, AngleOX represents the first rotation angle around the X axis, AngleOY represents the rotation angle around the Y axis, and AngleOZx represents the second rotation angle around the X axis; ； ； OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the horizontal angle between the optical center and the image point and the optical axis, and AngleV represents the vertical angle between the optical center and the image point and the optical axis; the coordinates of the first calibration point are (x0, y0, z0); the coordinates of the optical center of the visual camera are (x oc ,y oc , z oc ); The method of obtaining the visual camera external parameters and the focal length of the camera by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera external parameters includes: The image point coordinates of the two calibration points after projection are obtained by back-projection according to the ideal internal parameters and the initial values ​​of the external parameters of the visual camera; Iterate the equivalent focal length and image point coordinate relationship of the visual camera, the angle of rotation around the Z axis and the image point coordinate relationship for multiple times to obtain the equivalent focal length and the angle of rotation around the Z axis of the visual camera; The external parameters of the visual camera, that is, the final rotation matrix and the final translation vector, are obtained according to the angle of rotation around the Z axis.

Images