Large-view-field vision measurement method and large-view-field vision measurement system

Abstract

The invention relates to the technical field of vision measurement, and provides a large-view-field vision measurement method and a large-view-field vision measurement system.The method comprises the following steps that S1, a calibration method of vision cameras applied to large-view-field optical measurement is adopted to calibrate each vision camera, and internal parameters and external parameters of each vision camera are obtained; s2, simultaneously shooting a target to be detected by the visual camera, and obtaining a shot image; s3, obtaining an image point of the to-be-measured target from the shot image; s4, simultaneous collinear equations are established; and S5, solving the collinear equation by adopting a least square method according to the internal and external parameters and the image points of the visual camera to obtain the position information of the to-be-measured target. According to the scheme, simple and high-precision calibration can be carried out before measurement, the point distribution difficulty is reduced, and the measurement precision in a view field is ensured.

Description

Technical Field

[0001] The present invention relates to the field of visual measurement technology, and in particular to a large-field-of-view visual measurement method and a large-field-of-view visual measurement system. Background Art

[0002] Visual inspection is to use machines to replace human eyes to make measurements and judgments. Visual inspection refers to the process of converting the captured target into image signals through machine vision products (i.e. image capture devices, divided into CMOS and CCD), transmitting them to a dedicated image processing system, and converting them into digital signals based on pixel distribution, brightness, color and other information; the image system performs various operations on these signals to extract the characteristics of the target, and then controls the equipment action on site based on the results of the judgment. It is a valuable mechanism for production, assembly or packaging. It has immeasurable value in detecting defects and preventing defective products from being delivered to consumers.

[0003] Visual inspection requires the use of a visual camera to shoot the target to be measured. Before shooting, calibration points need to be set up near the target to be measured. The internal and external parameters of the visual camera need to be calibrated according to the calibration points to obtain accurate measurement data. However, as the field of view increases, the visual camera is far away from the target to be measured, exceeding 100 meters, and the arrangement of calibration points in the air becomes particularly difficult. If you want to arrange points above 5 meters at a height, you need to rely on other surrounding buildings, and the requirements for environmental conditions are more stringent; when the field of view is greater than 1 kilometer, each additional point will require more manpower and material resources, and the difficulty of installation will also increase sharply. Therefore, in a large field of view of hundreds of meters or even kilometers, how to minimize the number of points, reduce the difficulty of point arrangement, reduce the height of point arrangement, and ensure the measurement accuracy within the field of view is a technical problem that needs to be solved urgently.

[0004] Therefore, it is necessary to provide a visual measurement method and a visual measurement system with a large field of view, which can perform simple and high-precision calibration before measurement, reduce the difficulty of point arrangement, and ensure the measurement accuracy within the field of view.

[0005] The above information disclosed in the Background section is only for enhancement of understanding of the background of the present application and therefore it may contain information that does not constitute the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0006] The main purpose of the present invention is to overcome the problems of difficulty in arranging calibration points and inaccurate measurement in large-field-of-view optical measurement, and to provide a large-field-of-view visual measurement method and a large-field-of-view visual measurement system, which can perform simple and high-precision calibration before measurement, reduce the difficulty of arranging points, and ensure the measurement accuracy within the field of view.

[0007] To achieve the above-mentioned purpose, the first aspect of the present invention provides a large-field visual measurement method, using at least two visual cameras, the distance between the visual cameras and the target to be measured is greater than 100 meters, and includes the following steps:

[0008] S1: Calibrate each visual camera using the calibration method for visual cameras used for large field of view optical measurement to obtain the intrinsic and extrinsic parameters of each visual camera;

[0009] S2: The visual camera simultaneously shoots the target to be measured and obtains the captured image;

[0010] S3: Obtaining image points of the target to be measured from the captured image;

[0011] S4: Simultaneous collinear equations;

[0012] S5: According to the internal and external parameters and image points of the visual camera, the least square method is used to solve the collinear equation to obtain the position information of the target to be measured.

[0013] According to an exemplary embodiment of the present invention, in step S1, the calibration method of a visual camera applied to large field of view optical measurement includes:

[0014] S11: setting two calibration points within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera;

[0015] S12: Level the rolling direction of the visual camera;

[0016] S13: The visual camera captures an image and simultaneously obtains position information of two calibration points of the visual camera;

[0017] S14: Obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera;

[0018] S15: According to the position information of the visual camera and the two calibration points, and the initial value of the visual camera external parameter, the visual camera external parameter and the focal length of the camera are obtained by a two-point calibration method.

[0019] According to an exemplary embodiment of the present invention, in step S14, obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera includes:

[0020] Calculate the angle between the line from the optical center to the image point and the optical axis;

[0021] The vertical distance from the optical center to the first calibration point is obtained according to the angle between the optical center and the image point and the optical axis, the position information of the first calibration point and the position information of the visual camera;

[0022] A rotation matrix is ​​obtained according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point;

[0023] Get the translation vector based on the rotation matrix.

[0024] According to an exemplary embodiment of the present invention, in step S15, the step of obtaining the visual camera extrinsic parameters and the focal length of the camera by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera extrinsic parameters includes:

[0025] The image point coordinates of the two calibration points after projection are obtained by back-projection according to the ideal internal parameters and the initial values ​​of the external parameters of the visual camera;

[0026] Iterate the equivalent focal length and image point coordinate relationship of the visual camera, the angle of rotation around the Z axis and the image point coordinate relationship for multiple times to obtain the equivalent focal length and the angle of rotation around the Z axis of the visual camera; the world coordinate system includes the Z axis;

[0027] The external parameters of the visual camera, that is, the final rotation matrix and the final translation vector, are obtained according to the angle of rotation around the Z axis.

[0028] According to an exemplary embodiment of the present invention, in step S4, the simultaneous collinear equations are as follows:

[0029] [(u i -C x )r6-F x r0]X+[(u i -C x )r7-F x r1]Y+[(u i -C x )r8-F x r2]Z

[0030] +[(u i -C x )T z -F x T x ]=0

[0031] [(v i -C y )r6-F y r3]X+[(v i -C y )r7-F y r4]Y+[(v i -C y )r8-F y r5]Z

[0032] +[(v i -C y )T z -F y T y ]=0;

[0033] Among them, (u i , v i ) represents the position of the image point of the i-th visual camera;

[0034] The external parameters of the visual camera include the rotation matrix and translation vector, and the internal parameters include the equivalent focal length and principal point parameters;

[0035] F x 、F y is the equivalent focal length of the visual camera, C x , C y is the principal point parameter of the visual camera, where R is the rotation matrix and T is the translation vector;

[0036]

[0037] T=[T x T y T z ] t .

[0038] According to a second aspect of the present invention, the present invention provides a visual measurement system with a large field of view, comprising:

[0039] At least two visual cameras, used to photograph the target to be measured, with a distance of more than 100 meters from the target to be measured;

[0040] A calibration module, connected to the visual camera, for calibrating each visual camera using a calibration method for visual cameras used for large-field optical measurement, to obtain internal and external parameters of each visual camera; and

[0041] The target position calculation module is connected to the visual camera and the calibration module, and is used to obtain the captured image and the internal and external parameters of each visual camera; obtain the image points of the target to be measured from the captured image; and solve the collinearity equations simultaneously; according to the internal and external parameters of the visual camera and the image points, the least square method is used to solve the collinearity equations to obtain the position information of the target to be measured.

[0042] According to an exemplary embodiment of the present invention, the large field of view visual measurement system further includes two calibration points and a position information acquisition module;

[0043] Two calibration points are set within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera;

[0044] The position information acquisition module is used to obtain the position information of the visual camera and two calibration points;

[0045] The calibration module is used to obtain the initial value of the visual camera extrinsic parameter according to the position information of the first calibration point and the position information of the visual camera; the visual camera extrinsic parameter and the focal length of the camera are obtained by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera extrinsic parameter.

[0046] The advantages of the present invention are:

[0047] The present invention solves the problem of the difficulty of camera calibration in the case of large field of view optical measurement. Under the conditions of a measured distance greater than 100m and a field of view greater than 100m, only two points are used, and there is no requirement for the height of the points. It only needs to ensure that they are visible in the visual camera. The calibration of the camera external parameters is achieved through two simple calibration points, and the optimization of the equivalent focal length in the camera internal parameters is achieved at the same time, which greatly reduces the difficulty of point arrangement during the calibration process and the difficulty in ensuring the accuracy of the large field of view. After completing the accurate calibration, accurate measurement results can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other objects, features and advantages of the present application will become more apparent by describing in detail the exemplary embodiments thereof with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the present application, and it is clear to a person skilled in the art that other accompanying drawings can be obtained from these accompanying drawings without creative effort.

[0049] Figure 1 The schematic diagram of the visual camera photographing the calibration points is shown schematically.

[0050] Figure 2 The diagram schematically shows the steps of a visual measurement method for a large field of view.

[0051] Figure 3 The diagram schematically shows the steps of a calibration method for a visual camera used for large field of view optical measurement.

[0052] Figure 4 The definition diagram of the coordinate system is schematically shown.

[0053] Figure 5 A schematic diagram of multiple calibration points is schematically shown.

[0054] Figure 6The schematic diagram shows two visual cameras photographing a target to be measured. DETAILED DESCRIPTION

[0055] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0056] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0057] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0058] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0059] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the concepts of the present application. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0060] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present application, and therefore cannot be used to limit the scope of protection of the present application.

[0061] According to a first specific embodiment of the present invention, the present invention provides a visual measurement system with a large field of view, comprising:

[0062] At least two visual cameras, used to photograph the target to be measured, with a distance of more than 100 meters from the target to be measured;

[0063] A calibration module is connected to the visual camera and is used to calibrate each visual camera using a calibration method for visual cameras used for large-field optical measurement to obtain internal and external parameters of each visual camera; specifically, the calibration module is used to obtain an initial value of the external parameter of the visual camera according to the position information of the first calibration point and the position information of the visual camera; and obtain the external parameter of the visual camera and the focal length of the camera by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the external parameter of the visual camera;

[0064] The target position calculation module is connected to the visual camera and the calibration module, and is used to obtain the captured image and the internal and external parameters of each visual camera; obtain the image points of the target to be measured from the captured image; and solve the collinearity equations simultaneously; according to the internal and external parameters of the visual camera and the image points, the least square method is used to solve the collinearity equations to obtain the position information of the target to be measured;

[0065] Two calibration points are set within the field of view of the visual camera. The distance between the two calibration points and the visual camera is greater than 100 meters. As a preferred embodiment, the vertical distance between the two calibration points and the visual camera is greater than 100 meters. The two calibration points are respectively the first calibration point and the second calibration point. The first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera. The calibration points are cylindrical in shape, and two strip colors are set at intervals. As a preferred embodiment, Figure 1 As shown in the figure, the calibration point is a cylinder made by RTK with red and white stripes, 1 meter high and 0.5 meter in diameter. The red and white stripes can ensure that they are clearly visible even at a long distance. RTK can record the location information of the calibration point; and

[0066] The position information acquisition module is used to obtain the position information of the visual camera and two calibration points, and can obtain the position information of the visual camera and two calibration points from RTK.

[0067] A large field of view means that the distance between the visual camera and the target to be measured is greater than 100 meters. The distance between the target to be measured and the visual camera and the distance between the calibration point and the visual camera may not be equal, and the distance between the target to be measured may be 1 kilometer and the distance between the calibration point may be 100 meters. As a preferred embodiment, the distance between the target to be measured and the visual camera is greater than or equal to the distance between the calibration point and the visual camera.

[0068] The system using this solution only needs two calibration points, which simplifies the calibration process.

[0069] As a second specific embodiment of the present invention, the present invention provides a large-field visual measurement method, using the large-field visual measurement system of the first specific embodiment, such as Figure 2 As shown, the following steps are included:

[0070] S1: Calibrate each visual camera using the calibration method for visual cameras with large field of view optical measurement to obtain the intrinsic and extrinsic parameters of each visual camera.

[0071] For most camera measurement systems, since the object distance of urban and rural systems is much greater than the focal length, a small error in determining the rays composed of the optical center and the image point will cause a large error in the long-distance spatial intersection result. "A small error can lead to a big mistake" is a vivid description of long-distance and large field of view measurement. Therefore, accurately determining the rays composed of the image point and the optical center is the key to measurement. To achieve this, it is necessary to obtain accurate camera internal and external parameters. Therefore, the core of improving measurement accuracy is to optimize the camera internal and external parameters.

[0072] like Figure 3 As shown, the calibration method of a visual camera applied to large field of view optical measurement includes the following steps:

[0073] S11: two calibration points are set within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera.

[0074] The field of view of the visual camera is divided into three equal parts, including the left area, the middle area and the right area from left to right. The height of each area is equal and the width is one third of the field of view width. The first calibration point is located in the middle area, close to the center of the field of view of the visual camera; the second calibration point is located in the left area on the left or the right area on the right, close to the left edge or right edge of the field of view of the visual camera.

[0075] The calibration point is cylindrical in shape, and two strip-shaped colors are arranged at intervals. As a preferred embodiment, Figure 1 As shown in the figure, the calibration point is a cylinder made by RTK with red and white stripes, 1 meter high and 0.5 meter in diameter. The red and white stripes can ensure that they are clearly visible even at a long distance. RTK can record the location information of the calibration point.

[0076] A large field of view means that the distance between the visual camera and the target to be measured is greater than 100 meters. The distance between the target to be measured and the visual camera and the distance between the calibration point and the visual camera may not be equal, and the distance between the target to be measured may be 1 kilometer and the distance between the calibration point may be 100 meters. As a preferred embodiment, the distance between the target to be measured and the visual camera is greater than or equal to the distance between the calibration point and the visual camera.

[0077] S12: Level the roll direction of the visual camera.

[0078] Leveling the visual camera in the roll direction includes: adjusting the bubble bar until the bubble is in a horizontal position.

[0079] S13: The visual camera captures an image and simultaneously obtains position information of the visual camera and two calibration points.

[0080] The obtaining of the position information of the visual camera and the two calibration points includes: recording the position information of the visual camera and the two calibration points through RTK.

[0081] S14: Obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera.

[0082] The calibration principle is as follows:

[0083] The conversion formula from object space to image space is:

[0084]

[0085] Among them, Z c is the projection of the distance from the object point to the optical center in the direction of the optical axis, u and v are the horizontal and vertical coordinates of the calibration point in the image (known quantities), respectively, F x 、F y is the equivalent focal length of the visual camera, C x , C y is the principal point parameter of the visual camera, where R represents the 3*3 rotation matrix of the three rotation angles Ax, Ay, and Az around the X, Y, and Z axes of the world coordinate system, and T = [T x , T y , T z ] t is the translation vector, where t represents the transpose of the matrix.

[0086] The known conditions of the two-point calibration method are:

[0087] Known F x 、F y The initial value of F = F x =F y , identify C x , C y The ideal value remains unchanged.

[0088] C x =W / 2, C y =H / 2;

[0089] Where W represents the horizontal resolution of the image, and H represents the vertical resolution of the image.

[0090] F x and F y Respectively represent the equivalent focal length of the camera in the horizontal and vertical directions; the initial value is equal to F = 1000·f / du. f is the nominal focal length of the lens, in mm, and du is the pixel size of the camera, in μm.

[0091] After the visual camera is leveled, the initial value of Az is considered to be 0 degrees.

[0092] So the known quantity is u n 、v n , C x , C y 、x n ,y n 、z n , n represents the point number, u and v represent the horizontal and vertical coordinates of the image point, x n ,y n 、z n Represents the object point coordinates of point. The quantities to be optimized are F, Ax, Ay, Az, Tx, Ty, Tz, among which the quantities to be optimized F and Az have initial values.

[0093] In addition, the location of the optical center of the visual camera is known, namely:

[0094]

[0095] x oc ,y oc , z oc Represents the coordinates of the optical center of the visual camera.

[0096] The method of obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera comprises:

[0097] S141: Calculate the angle between the line from the optical center to the image point and the optical axis.

[0098] AngleU=arctan(u0 / F x )

[0099] AngleV=arctan(v0 / F y );

[0100] Where AngleU and AngleV represent the angles between the line from the optical center to the image point and the optical axis in the horizontal and vertical directions respectively; u0 represents the horizontal coordinate of the image point, and v0 represents the vertical coordinate of the image point.

[0101] S142: Obtain a vertical distance from the optical center to the first calibration point according to the angle between the optical center and the image point connecting line and the optical axis, the position information of the first calibration point, and the position information of the visual camera.

[0102] The vertical distance from the first calibration point to the optical center is obtained according to the angle between the optical center and the image point and the optical axis, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point using the following formula:

[0103]

[0104] Among them, OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the angle between the line connecting the optical center to the image point P and the optical axis Oc-Zc in the X direction, AngleV represents the angle between the line connecting the optical center to the image point and the optical axis in the Y direction, and L represents the distance from the optical center to the first calibration point, which is obtained according to the position information of the first calibration point and the position information of the visual camera.

[0105] like Figure 4 As shown, the coordinate system is defined as follows:

[0106] The world coordinate system is O-XYZ, the Z axis is perpendicular to the plane where the XY plane is located, and the camera coordinate system is Oc-x c y c z c , the pixel coordinates are u, v, where Oc represents the optical center of the camera, z c The x axis is the optical axis of the camera, and the shooting direction is taken as the positive direction. c ,y c The axes are usually parallel to the two perpendicular sides of the imaging CCD plane.

[0107] x c The y axis represents the horizontal axis of the camera coordinate system. c The axis represents the vertical axis of the camera coordinate system, z c The axis is perpendicular to the XY plane; the pitch angle of the visual camera is z c The angle between the axis and the horizontal plane.

[0108] like Figure 4 As shown: o-xy represents the camera target surface coordinate system, Oc-x c y c z c represents the camera coordinate system, O-XYZ represents the world coordinate system, u and v represent the pixel horizontal coordinate and pixel vertical coordinate respectively. P represents the object point, and p is the projection of the object point P on the camera target surface, that is, the image point.

[0109] S143: Obtain a rotation matrix according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point.

[0110] The step of obtaining the rotation matrix according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point includes:

[0111] The rotation matrix is ​​the rotation matrix from the world coordinate system to the camera coordinate system. The world coordinate system includes the X-axis, Y-axis, and Z-axis. First, rotate around the X-axis to obtain a 90-degree rotation angle around the X-axis, so that the Y-axis points downward; then rotate around the Y-axis so that the Z-axis points to coincide with the camera optical axis in the vertical direction, and finally rotate around the X-axis a second time so that the pitch angle of the visual camera is adjusted to completely coincide with the Z-axis and the camera optical axis;

[0112] Among them, the X-axis represents the horizontal axis of the camera coordinate system, the Y-axis represents the vertical axis of the camera coordinate system, and the Z-axis is perpendicular to the plane where the XY axis is located; the pitch angle of the visual camera is the angle between the Z-axis and the horizontal plane;

[0113] The rotation matrix uses the following formula:

[0114] R = Rozx·Roy·Rox;

[0115]

[0116] Where R represents the rotation matrix, AngleOX represents the first rotation angle around the X axis, AngleOY represents the rotation angle around the Y axis, and AngleOZx represents the second rotation angle around the X axis;

[0117]

[0118] OO represents the vertical distance from the optical center of the visual camera to the first calibration point, AngleU represents the horizontal angle between the optical center and the image point and the optical axis, and AngleV represents the vertical angle between the optical center and the image point and the optical axis; the coordinates of the first calibration point are (x0, y0, z0); the coordinates of the optical center of the visual camera are (x oc ,y oc , z oc ).

[0119] S144: Obtain a translation vector according to the rotation matrix.

[0120] The translation vector is obtained using the following formula:

[0121]

[0122] Among them, T is the translation vector, R is the rotation matrix, and the coordinates of the optical center of the visual camera are (x oc ,y oc , z oc ).

[0123] S15: According to the position information of the visual camera and the two calibration points, and the initial value of the visual camera external parameter, the visual camera external parameter and the focal length of the camera are obtained by a two-point calibration method.

[0124] The method of obtaining the visual camera external parameters and the focal length of the camera by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera external parameters includes:

[0125] S151: The image point coordinates of the two calibration points after projection are obtained through back projection according to the ideal internal parameters and the initial values ​​of the external parameters of the visual camera.

[0126] According to the ideal internal and external parameter initial values, the coordinates of the projected image point are obtained by back projection, ru0, rv0, ru1, rv1; they represent the back projection coordinates of the two calibration points respectively, ru0, rv0 represent the back projection pixel coordinates u, v values ​​of the first calibration point (u represents the horizontal coordinate, v represents the vertical coordinate), similarly ru1, rv1 represent the back projection coordinates of the second calibration point.

[0127] The internal parameters include Fx, Fy, Cx, and Cy; the external parameters include the rotation matrix R and the translation vector T.

[0128] S152: Iterate the equivalent focal length and image point coordinate relationship of the visual camera, and the angle of rotation around the Z axis and the image point coordinate relationship multiple times to obtain the equivalent focal length and the angle of rotation around the Z axis of the visual camera; the world coordinate system includes the Z axis.

[0129] The relationship between the equivalent focal length and image point coordinates of the visual camera is:

[0130]

[0131] The relationship between the angle of rotation around the Z axis and the image point coordinates is:

[0132]

[0133] Among them, F represents the equivalent focal length of the visual camera, k represents the number of iterations, AngleOZ represents the angle of rotation around the Z axis, the back-projected image point coordinates of the first calibration point are (ru0, rv0), the back-projected image point coordinates of the second calibration point are (ru1, rv1), u represents the horizontal coordinate, v represents the vertical coordinate, the image point coordinates of the first calibration point are (u0, v0), and the image point coordinates of the second calibration point are (u1, v1).

[0134] The optimization iteration termination condition is rms < 0.001 or the number of iterations reaches 30.

[0135]

[0136] Among them, rms represents the reprojection error, (u0, v0) represents the pixel coordinates u and v of the first calibration point (u represents the horizontal coordinate and v represents the vertical coordinate), similarly (u1, v1) represents the pixel coordinates of the second calibration point, (ru0, rv0) represents the back-projected coordinates of the first calibration point, (ru1, rv1) represents the back-projected coordinates of the second calibration point, and k represents the number of iterations. The reprojection error rms can further verify the correctness of the calibration.

[0137] If the result does not converge, the iteration cannot be infinite, so an empirical value of 30 is set as the termination condition.

[0138] Each iteration, all angles (including AngleOZ, AngleOZx, AngleOY, AngleOX) are updated once.

[0139] S153: Obtain the external parameters of the visual camera according to the angle of rotation around the Z axis, that is, the final rotation matrix and the final translation vector.

[0140] After iteration, the external parameters of the visual camera are obtained according to the angle of rotation around the Z axis, and the new rotation matrix and translation vector are obtained:

[0141] R k =Roz k ·Rozx k ·Roy k ·Rox k ;

[0142]

[0143] At the end of the iteration, we get the final rotation matrix R* and translation vector T*. The final rotation matrix R* is R k , Roz k is the kth rotation matrix of the iteration. R* can be decomposed into A x , A y , A z Three Euler angles, T*=[T x , T y , T z ] t The translation vector, where t represents the transpose of the matrix, is obtained by referring to the formula of step S44 through the rotation matrix and the coordinates of the optical center of the visual camera.

[0144] The relationship between the rotation matrix R* and Ax, Ay, Az is:

[0145]

[0146] like Figure 5As shown in the figure, the visual camera shoots the calibration points. If the two-point calibration of this solution is adopted, the two points can be arranged close to the ground, such as points P0 and P1. If the four-point or six-point calibration of the existing technology is adopted, it is necessary to ensure that the arrangement height of the control points changes, such as points P2 and P3. If the ground is relatively empty and there are no buildings, it is very difficult to raise the points.

[0147] The measurement accuracy of this scheme is verified by binocular intersection, and the measurement accuracy can reach 3 / 1000, that is, when shooting a field of view of 1km, the measurement accuracy is 3m.

[0148] It can be seen from the above scheme that only two calibration points are needed to calibrate the visual camera. The present invention solves the problem that camera calibration is difficult in the case of large-field-of-view optical measurement. Under the conditions of a measured distance greater than 100m and a field of view greater than 100m, only two points are used. There is no requirement for the height of the points, as long as they are visible in the visual camera. The calibration of the camera's external parameters is achieved through two simple calibration points, and the optimization of the equivalent focal length in the camera's internal parameters is achieved at the same time, which greatly reduces the difficulties in point arrangement during the calibration process and the difficulty in ensuring the accuracy of a large field of view.

[0149] S2: The visual camera simultaneously photographs the target to be measured and obtains the photographed image.

[0150] S3: Obtain image points of the target to be measured from the captured image.

[0151] S4: Simultaneous collinear equations.

[0152] The simultaneous collinear equations are as follows:

[0153] [(u i -C x )r6-F x r0]X+[(u i -C x )r7-F x r1]Y+[(u i -C x )r8-F x r2]Z+[(u i -C x )T z -F x T x ]=0

[0154] [(v i -C y )r6-F y r3]X+[(v i -C y )r7-F y r4]Y+[(v i -Cy )r8-F y r5]Z+[(v i -C y )T z -F y T y ]=0;

[0155] Among them, (u i , v i ) represents the position of the image point of the i-th visual camera;

[0156] The external parameters of the visual camera include the rotation matrix and translation vector, and the internal parameters include the equivalent focal length and principal point parameters;

[0157] F x 、F y is the equivalent focal length of the visual camera, C x , C y is the principal point parameter of the visual camera, where R is the rotation matrix and T is the translation vector;

[0158]

[0159] T=[T x T y T z ] t .

[0160] For the collinear equation of a single camera, there are only two equations, three unknowns (X, Y, Z), and no unique solution. When the number of cameras n = 2, four equations can be obtained by combining them, and the number of unknowns is still 3. The least squares method can be used to solve the linear equations. The (X, Y, Z) value of the target point P is obtained.

[0161] The (X, Y, Z) value of the target point P is the position where the sum of the squares of the distances of the spatial straight line family determined by the optical center and image point of each camera is the smallest. For the case where two cameras intersect, the target object point represents the midpoint of the common perpendicular line of the intersection of the two spatial straight lines, which has a clear physical meaning.

[0162] S5: According to the internal and external parameters and image points of the visual camera, the least square method is used to solve the collinear equation to obtain the position information of the target to be measured.

[0163] The basic imaging model for visual camera measurement is the central projection model, or pinhole model. Under the condition of satisfying the pinhole model, the image point, optical center, and object point satisfy the collinear equation, that is, the three points are collinear. For a single camera, if the optical center and image point are known, it can be determined that the image point and optical center form a unique ray, and the object point must be on this ray, but the z position of the object point on this ray cannot be determined. Therefore, without other additional information, it is impossible to determine the three-dimensional coordinate position of the target using only a single camera.

[0164] When two or more visual cameras are used to shoot the same target from different positions, the rays composed of the optical center of each visual camera and its corresponding image point of the same name should pass through the same spatial object point, that is, intersect.

[0165] like Figure 6 As shown, OC1 and OC2 represent the optical centers of two visual cameras respectively, the target to be measured is P, P11 and P12 represent the image points with the same name respectively, and the three-dimensional spatial coordinates of the object point (target to be measured) P can be obtained by using the known internal and external parameters of the visual camera and the image point information.

[0166] This solution finds the exact position of the image point based on simple and accurate calibration. According to the camera's internal and external parameters and the image point information, the three-dimensional spatial coordinates of the target to be measured can be obtained quickly, conveniently and accurately. It is suitable for visual measurement with a large field of view.

[0167] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structure, configuration or implementation method described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A visual measurement method with a large field of view, using at least two visual cameras, the distance between the visual cameras and the target to be measured is greater than 100 meters, characterized in that: The following steps are involved: S1: Calibrate each visual camera using the calibration method for visual cameras used for large field of view optical measurement to obtain the intrinsic and extrinsic parameters of each visual camera; S2: The visual camera simultaneously shoots the target to be measured and obtains the captured image; S3: Obtaining image points of the target to be measured from the captured image; S4: Simultaneous collinear equations; S5: According to the internal and external parameters and image points of the visual camera, the least square method is used to solve the collinear equation to obtain the position information of the target to be measured.

2. The large-field visual measurement method according to claim 1, characterized in that: In step S1, the calibration method of the visual camera applied to large field of view optical measurement includes: S11: setting two calibration points within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera; S12: Level the rolling direction of the visual camera; S13: The visual camera captures an image and simultaneously obtains the position information of the visual camera and two calibration points; S14: Obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera; S15: According to the position information of the visual camera and the two calibration points, and the initial value of the visual camera external parameter, the visual camera external parameter and the focal length of the camera are obtained by a two-point calibration method.

3. The large-field visual measurement method according to claim 2, characterized in that: In step S14, obtaining the initial value of the visual camera external parameter according to the position information of the first calibration point and the position information of the visual camera includes: Calculate the angle between the line from the optical center to the image point and the optical axis; The vertical distance from the optical center to the first calibration point is obtained according to the angle between the optical center and the image point and the optical axis, the position information of the first calibration point and the position information of the visual camera; A rotation matrix is ​​obtained according to the vertical distance from the optical center to the first calibration point, the angle between the optical center and the image point and the optical axis, and the position information of the first calibration point; Get the translation vector based on the rotation matrix.

4. The visual measurement method with a large field of view according to claim 2, characterized in that: In step S15, the method of obtaining the visual camera external parameters and the focal length of the camera by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera external parameters includes: The image point coordinates of the two calibration points after projection are obtained by back-projection according to the ideal internal parameters and the initial values ​​of the external parameters of the visual camera; Iterate the equivalent focal length and image point coordinate relationship of the visual camera, the angle of rotation around the Z axis and the image point coordinate relationship for multiple times to obtain the equivalent focal length and the angle of rotation around the Z axis of the visual camera; the world coordinate system includes the Z axis; The external parameters of the visual camera, that is, the final rotation matrix and the final translation vector, are obtained according to the angle of rotation around the Z axis.

5. The visual measurement method with a large field of view according to claim 1, characterized in that: In step S4, the simultaneous collinear equations are as follows: [(u i -C x )r6-F x r0]X+[(u i -C x )r7-F x r1]Y+[(u i -C x )r8-F x r2]Z+[(u i -C x )T z -F x T x ]=0 [(v i -C y )r6-F y r3]X+[(v i -C y )r7-F y r4]Y+[(v i -C y )r8-F y r5]Z+[(v i -C y )T z -F y T y ]=0; Among them, (u i , v i ) represents the position of the image point of the i-th visual camera; The external parameters of the visual camera include the rotation matrix and translation vector, and the internal parameters include the equivalent focal length and principal point parameters; F x 、F y is the equivalent focal length of the visual camera, C x , C y is the principal point parameter of the visual camera, where R is the rotation matrix and T is the translation vector; T=[T x T y T z ] t 。 6. A visual measurement system with a large field of view, characterized in that: include: At least two visual cameras, used to photograph the target to be measured, with a distance of more than 100 meters from the target to be measured; A calibration module, connected to the visual camera, is used to calibrate each visual camera using a calibration method for visual cameras used for large-field optical measurement to obtain internal and external parameters of each visual camera; and The target position calculation module is connected to the visual camera and the calibration module, and is used to obtain the captured image and the internal and external parameters of each visual camera; obtain the image points of the target to be measured from the captured image; and solve the collinearity equations simultaneously; according to the internal and external parameters of the visual camera and the image points, the least square method is used to solve the collinearity equations to obtain the position information of the target to be measured.

7. The large-field-of-view visual measurement system according to claim 6, characterized in that: It also includes two calibration points and position information acquisition modules; Two calibration points are set within the field of view of the visual camera, and the distance between the two calibration points and the visual camera is greater than 100 meters; the two calibration points are respectively a first calibration point and a second calibration point, the first calibration point is close to the center of the field of view of the visual camera, and the second calibration point is close to the edge of the field of view of the visual camera; The position information acquisition module is used to obtain the position information of the visual camera and two calibration points; The calibration module is used to obtain the initial value of the visual camera extrinsic parameter according to the position information of the first calibration point and the position information of the visual camera; the visual camera extrinsic parameter and the focal length of the camera are obtained by a two-point calibration method according to the position information of the visual camera and the two calibration points and the initial value of the visual camera extrinsic parameter.

Images