A Camera Self-Calibration Method and System Based on Distance Measurement

Abstract

The present invention provides a camera self-calibration method and system based on distance measurement, belonging to the technical field of machine vision. The present invention obtains the image information of the calibration object; obtains the position information of the calibration plane through distance measurement; solves the calibration plane equation; selects at least four groups of two-dimensional coordinates corresponding on the calibration plane and the image plane of the calibration object, and solves the homography matrix representing the change relationship between the calibration plane and the image plane of the calibration object; when performing distance measurement, laser is emitted from the distance measurement points on the laser distance measurement device and irradiated on the calibration object, at least 3 calibration points are formed, and all the calibration points are located within the calibration plane and not all on the same straight line; the laser distance measurement device is used to obtain multiple straight-line distances between one or more distance measurement points and the calibration points, and determine the relative position of the straight line and the optical axis of the imaging device. The camera self-calibration method of the present invention does not require the aid of a calibration object, and the camera does not require high-precision special motion, with simple operation and easy solution of calibration parameters.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a camera self-calibration method and system based on distance measurement. Background Art

[0002] With the development of technology, machine vision is widely used in defect detection in various industries. It uses a camera to capture an object to be measured, obtains information from the image through image recognition and processing, and classifies, identifies, tracks, and makes decisions on the target. It has the advantages of no need for the detector to contact, a large detection visual range, stable work, and high efficiency. The acquisition of the image and obtaining image information through the image are key steps in the detection. On the basis of a movable device carrying a camera device to acquire an image, in order to further obtain effective image information, the calibration technology must be concerned.

[0003] Calibration technology, that is, camera calibration, is to solve the geometric model of camera imaging to determine the conversion relationship between the geometric position of a certain point in the actual scene and its corresponding point in the image. The accuracy of its calibration result and the stability of the algorithm will seriously affect the accuracy of the camera working result.

[0004] Existing calibration technologies can be roughly divided into three categories: traditional camera calibration technology, camera calibration technology based on active vision, and self-calibration technology. Traditional camera calibration technology needs to use a calibration object with a determined size, and calculates parameters by establishing the corresponding relationship between known points on the calibration object and their image points. Among them, the Tsai two-step calibration method and the Zhang calibration method are traditional calibration methods based on stereo and planar calibration objects respectively. Although the calibration board in traditional cameras is easier to manufacture than previous calibration objects, and the calibration accuracy of the corresponding methods is also higher, in the defect detection in some fields (such as civil engineering), the danger and limitation of using the calibration object are still relatively large. Camera calibration technology based on active vision needs to control the camera to perform high-precision special movements such as pure rotation or translation, and uses the new constraints provided by the special movement to solve the parameter equation. Although the algorithm of this technology is simple, in the detection of some fields, it is difficult and inflexible to control the camera to perform high-precision special movements. Camera self-calibration technology comes from the concept proposed by Faugeras, Maybank and others in the field of computer vision, which makes it possible to calibrate the camera in any motion form and unknown scene. Among them, Faugeras proposed a method, which realizes calibration by taking multiple shots of the target object or multiple cameras shooting the target simultaneously and using the non-linear constraint relationship between the images. However, the calibration operation is complex and the solution is difficult, which is not suitable for occasions with strong real-time performance, and the calibration accuracy is not high.

[0005] The existing technology has at least the following deficiencies:

[0006] 1. Traditional camera calibration techniques require the use of calibration objects.

[0007] 2. For camera calibration techniques based on active vision, the camera needs to perform high-precision special movements.

[0008] 3. Camera self-calibration techniques are complex to operate and difficult to solve. Summary of the Invention

[0009] To solve the problems existing in the prior art, the present invention provides a range-based camera self-calibration method and system, which acquires image information of a calibration object; acquires position information of a calibration plane through ranging; solves the calibration plane equation; selects at least four groups of two-dimensional coordinates corresponding on the calibration plane and the image plane of the calibration object, and solves the homography matrix representing the change relationship between the calibration plane and the image plane of the calibration object. When ranging, laser is emitted from a ranging point on the laser ranging device and irradiated on the calibration object, at least 3 calibration points are formed, and all the calibration points are located within the calibration plane and not all on the same straight line; the laser ranging device is used to acquire multiple straight-line distances between one or more ranging points and the calibration points, and determine the relative position of the straight line and the optical axis of the imaging device. The camera self-calibration method of the present invention does not require the use of a calibration object, nor does the camera need to perform high-precision special movements. It is simple to operate and easy to solve the calibration parameters.

[0010] The present invention provides a range-based camera self-calibration method, including the following steps:

[0011] Shoot a calibration object to obtain image information of the calibration object on the image plane of the imaging device;

[0012] Synchronously measure the position information of the calibration plane through a ranging device;

[0013] Solve the calibration plane equation;

[0014] Select at least four groups of two-dimensional coordinates corresponding on the calibration plane and the image plane of the calibration object, and solve the homography matrix representing the change relationship between the calibration plane and the image plane of the calibration object.

[0015] Preferably, an industrial camera is used to shoot a calibration object to obtain image information of the calibration object on the image plane of the imaging device, and ranging is performed through a laser ranging device.

[0016] Preferably, the ranging device synchronously measures and acquires measurement information for calibration, including the following steps:

[0017] Emit laser from a ranging point on the laser ranging device and irradiate it on the calibration object, at least 3 calibration points are formed, and all the calibration points are located within the calibration plane and not all on the same straight line;

[0018] Use a laser ranging device to obtain multiple straight-line distances between one or more ranging points and a calibration point, and determine the relative position of the straight line and the optical axis of the imaging device.

[0019] Preferably, the acquisition of the calibration plane position information includes the following steps:

[0020] Determine the position of the calibration point;

[0021] Solve for the three parameters A, B, and C in the equation Ax + By + Cz = 1 of the calibration plane in the camera coordinate system;

[0022] Where,

[0023] x, y, and z are the coordinates of the three coordinate axes of the camera coordinate system;

[0024] A, B, and C are the parameters of each coordinate axis respectively.

[0025] Preferably, determining the position of the calibration point includes: setting the optical center O c of the imaging device as the reference point, then both the reference point and the intersection point O of the above-mentioned optical axis of the imaging device and the plane where the ranging point is located are on the optical axis of the imaging device;

[0026] Taking the optical center O c of the reference point as the origin to establish a three-dimensional coordinate system as the camera coordinate system, then in this coordinate system, the ranging point C i has coordinates (X i , Y i , d), and when and only when the laser is parallel to the optical axis, the calibration point P i has coordinates (X i , Y i , D i + d), i ∈ [1, 2,..., N] and N ≥ 3, otherwise, adjust the coordinates of the ranging point and the calibration point according to the angle between the laser and the optical axis;

[0027] Where,

[0028] (X i , Y i , d) are the coordinates of the ranging point C i in the camera coordinate system;

[0029] (X i , Y i , D i + d) are the coordinates of the calibration point P i in the camera coordinate system;

[0030] d is the distance between the reference point and the plane where the ranging point is located;

[0031] D i is the distance measured by the ranging device for each calibration point Pi to the corresponding ranging point C i the straight-line distance therebetween;

[0032] N is the number of calibration points;

[0033] i is the serial number of the calibration point.

[0034] Preferably, solving the equation of the calibration plane in the camera coordinate system includes the following steps:

[0035] In the camera coordinate system, substitute the coordinates (X i , Y i , D i +d) of the calibration point P i into the calibration plane equation Ax + By + Cz = 1 to obtain the following formula

[0036]

[0037] wherein,

[0038] x, y, and z are the coordinates of the three coordinate axes of the camera coordinate system;

[0039] A, B, and C are the parameters of each coordinate axis in the calibration plane equation respectively;

[0040] i is the serial number of the calibration point, i ∈ [1, 2... N] and N ≥ 3;

[0041] d is the distance between the reference point and the plane where the ranging point is located;

[0042] D i is the straight-line distance measured by the ranging device from each calibration point P i to the corresponding ranging point C i therebetween;

[0043] N is the number of calibration points;

[0044] Solve for the three parameters A, B, and C to determine the plane equation of the calibration plane in the camera coordinate system.

[0045] Preferably, when solving the three parameters A, B, and C of the above calibration plane equation, simplify the linear equation system

[0046]

[0047] and represent it as

[0048] XΨ = E

[0049] to obtain

[0050]

[0051] wherein,

[0052]

[0053]

[0054]

[0055] N is the number of calibration points;

[0056] Solve the coefficient matrix Ψ, that is, determine the plane equation of the calibration plane in the camera coordinate system.

[0057] Preferably, according to the image information of the calibration object and the solved calibration plane equation, obtain at least four sets of two-dimensional coordinates, and solve the homography matrix.

[0058] Preferably, the acquisition of each set of two-dimensional coordinates includes the following steps:

[0059] According to the image information of the calibration object, obtain the calculation point j in the image plane through manual or image recognition technology k The two-dimensional coordinates j in the pixel coordinate system k (x k , y k );

[0060] Convert the two-dimensional coordinates into the three-dimensional coordinates of the calculation point j in the above camera coordinate system according to the camera internal parameters k The three-dimensional coordinates j k (x k , y k , f), k ∈ [1, 2,..., M] and M ≥ 4;

[0061] Obtain the corresponding point J of the calculation point j k in the calibration plane k The three-dimensional coordinates in the camera coordinate system are (Z k / f·x k , Z k / f·y k , Z k ), and M ≥ 4, k ∈ [1, 2,..., M];

[0062] Substitute the coordinates of point J k into the calibration plane equation to get A·Z k / f·x k +B·Z k / f·y k +C·Z k = 1;

[0063] Solve the only variable Z of the equation k , obtain the three-dimensional coordinates of the corresponding point J in the camera coordinate system, and obtain the corresponding point J k in the camera coordinate system, and obtain the corresponding point J kTwo-dimensional coordinates in the calibrated plane coordinate system;

[0064] k is the serial number of the two-dimensional coordinate group;

[0065] M is the number of two-dimensional coordinate groups.

[0066] Preferably, the homography matrix solution includes the following process:

[0067] The conversion formula from the world coordinate system to the pixel coordinate system is known as follows:

[0068]

[0069] Among them, (u, v, 1) T is the homogeneous coordinate of the pixel coordinate system, (x w , y w , z w , 1) T is the homogeneous coordinate of the world coordinate system, K 1 is the camera internal parameter matrix containing the camera internal parameters, K 2 is the camera external parameter matrix containing the camera external parameters, z c is also called the scale factor s; R is the rotation matrix, T is the translation matrix; f is the conversion coefficient from the camera coordinate to the image coordinate; dx is the size of each pixel on the horizontal axis x, and dy is the size of each pixel on the vertical axis y.

[0070] Homogeneous coordinates are a form in which an n-dimensional vector is represented by an n + 1-dimensional vector. Given a point (x, y) on the Euclidean plane, for any non-zero real number w, the triple (xw, yw, w) is called the homogeneous coordinate of the point (x, y).

[0071] Determine an arbitrary position of the calibration object in the world coordinate system, so that the Z-axis coordinate z w = 0, and the scale factor s does not change the coordinate value corresponding to the homogeneous coordinate, then there is

[0072]

[0073] Among them, H is the homography matrix, including 9 parameters. In the homogeneous coordinate system, perform scaling of any scale so that H has only 8 unknowns;

[0074] (x w , y w ) is the world coordinate of any point on the calibration object, and (u, v) is the pixel coordinate corresponding to this point on the calibration object;

[0075] For such a set of corresponding coordinates (x w , y wTwo equations can be obtained from (u, v). Therefore, at least 8 equations are required to solve the homography matrix H, and the two-dimensional coordinates of four sets of corresponding points are needed.

[0076] Obtain at least 4 calculation points j in the image plane from the captured images k The two-dimensional coordinates (u k , v k ) in the pixel coordinate system UOV, and convert them into point j according to the internal parameters of the camera k In the camera coordinate system O c -X c Y c Z c The three-dimensional coordinates (x k , y k , f). From the geometric relationship, it can be obtained that the corresponding point J of point j k on the object plane XOY i in the camera coordinate system O c -X c Y c Z c The coordinates are (Z k / f·x k , Z k / f·y k , Z k ), k ∈ [1, 2,..., N] and N ≥ 4. Substitute the coordinates of point J k into the equation Ax + By + Cz = 1 to get:

[0077] A·Z k / f·x k +B·Z k / f·y k +C·Z k = 1

[0078] Solve for the only variable Z k in the above equation, and the three-dimensional coordinates of at least 4 points J k in the camera coordinate system can be obtained respectively, and then the two-dimensional coordinates of each point on the object plane can be obtained.

[0079] Substitute the two-dimensional coordinates of at least 4 sets of corresponding points on the object plane and the image plane into the following conversion formula between pixel coordinates and world coordinates respectively, and the homography matrix H w2p can be solved.

[0080]

[0081] Based on the homography matrix H w2p , any arrangement of the world coordinate system on the object plane can be realized. Assuming that the world coordinate system is rotated by an angle α and translated by Δx and Δy, we can get:

[0082]

[0083] For H' w2p Take the inverse to obtain H w2p 。H w2p That is the self - calibration result of the final obtained image.

[0084] Referring to the conversion formula between the reference pixel coordinates and the world coordinates, using H w2p , the pixel coordinates of each point in the image can be converted into the world coordinates of the corresponding actual object points of the image points on the object plane.

[0085] The present invention provides a camera self - calibration system, which adopts the above - mentioned camera self - calibration method based on distance measurement, and includes a camera self - calibration device and a calculation module;

[0086] The camera self - calibration device includes a camera module, a distance measurement module, a combination module and a control module;

[0087] The camera module is connected to the combination module and the control module, and is used to photograph the calibration object, obtain the image information of the calibration object, and transmit the image information to the calculation module;

[0088] The distance measurement module is connected to the combination module and the control module, and synchronously measures the calibration object from the distance measurement points on the distance measurement module during shooting, including measuring the distance to the calibration points, and transmits the measurement information to the calculation module;

[0089] The combination module is connected to the camera module, the distance measurement module and the control module, combines the camera module and the distance measurement module into an integral whole, sets the relative position between the camera module and the distance measurement module, and transmits the relative position information to the calculation module;

[0090] The control module controls the camera module and the distance measurement module to rotate pitch - wise and rotate about the vertical axis as an integral whole, measures the direction and angle of rotation, and transmits the rotation information to the calculation module;

[0091] The calculation module obtains the image information and the calibration plane information, and realizes the camera self - calibration based on distance measurement.

[0092] Preferably, the camera module includes an industrial camera, and the distance measurement module includes a laser distance measurement device; by adjusting the relative position between the camera self - calibration device and the calibration object through the control module, using the camera module to photograph the calibration object, obtaining the image information of the calibration object, and transmitting the relative position and the image information to the calculation module;

[0093] During shooting, the distance measurement module is used to synchronously measure the calibration object from the distance measurement points and obtain the measurement information, and at the same time, the combination module is used to set the relative position between the camera module and the distance measurement module and obtain the relative position information;

[0094] The calculation module determines the relative position between the reference point and the calibration point based on the measurement information and the relative position information, and further obtains the position information of the plane where the calibration point is located;

[0095] The calculation module obtains the two-dimensional coordinates according to the acquired image information and position information, and calculates the corresponding homography matrix to realize camera self-calibration based on distance measurement.

[0096] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0097] (1) Based on laser ranging, the present invention obtains the position information of the calibration plane required for the calibration method, solves the plane equation for calibration, without the need for a calibration object to provide known points, and without the need for the camera to perform high-precision special movements to provide new constraints for solving;

[0098] (2) The present invention only needs a single camera to take a picture once while performing laser ranging to obtain the required image information of the measured plane. Based on the image information and the calibration plane equation, at least four groups of coordinates of the calculation points in the image plane pixel coordinate system and the coordinates of the corresponding points in the calibration plane (object plane) world coordinate system (z = 0) can be obtained. Based on the four groups of two-dimensional coordinates, the homography matrix can be solved to complete the calibration, and the calibration process is simple.

[0099] (3) The present invention is easy to operate, applicable to various environments, conducive to improving the degree of automation, reducing the errors caused by manual operations and the time and money spent in the calibration process, and can quickly obtain accurate calibration results. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 It is a schematic diagram for solving the object plane equation of an embodiment of the present invention; in the figure, C i is the ranging point, P i is the calibration point, X, Y are the image plane coordinates; x c , y c , z c are the camera coordinates; x p and y p are the image coordinates; u, v are the pixel coordinates; Oc is the optical center of the imaging device, and O is the intersection point of the plane where the ranging point is located and the optical axis of the imaging device;

[0101] Figure 2 It is a schematic diagram for solving the homography matrix of an embodiment of the present invention; in the figure, C i is the ranging point, P i is the calibration point, X, Y are the image plane coordinates; x c , y c , z c are the camera coordinates; x p and y p are the image coordinates; u, v are the pixel coordinates; Oc is the optical center of the imaging device, ji and J i are the two-dimensional coordinates of a calculation point in a set of image planes and the corresponding point of this calculation point in the calibration plane;

[0102] Figure 3 is a schematic diagram of the laser ranging camera self-calibration device according to an embodiment of the present invention;

[0103] Figure 4 is a test chart for the self-calibration accuracy of a camera based on ranging according to an embodiment of the present invention;

[0104] Figure 5 is a schematic diagram of the positions of laser emission points according to an embodiment of the present invention. In the figure, the laser emission points are L, T, and R;

[0105] Figure 6 is a computer display output diagram after point selection according to an embodiment of the present invention;

[0106] Figure 7 is a computer display output diagram of the iteration result of the distance between the reference point and the plane where the ranging points are located according to an embodiment of the present invention;

[0107] Figure 8 is a visualization schematic diagram of the calibration result displayed on the computer according to an embodiment of the present invention;

[0108] Figure 9 is a flowchart of a camera self-calibration method based on ranging according to an embodiment of the present invention.

[0109] In the figure: 1 - object plane; 2 - image plane; 3 - laser emission plane; 4 - optical center; 5 - laser rangefinder; 6 - camera; 7 - combination module. Specific Embodiments

[0110] The following will describe in detail the specific embodiments of the present invention with reference to the accompanying drawings.

[0111] The present invention provides a camera self-calibration method based on ranging, including the following steps:

[0112] Taking an image of a calibration object to obtain image information of the calibration object on the image plane of the imaging device; also obtaining the camera internal parameters of the imaging device, where the camera internal parameters include pixel size, focal length, and the position information of the origin of the image coordinate system (also known as the principal point) on the pixel coordinate system.

[0113] Synchronously measuring the position information of the calibration plane through a ranging device;

[0114] Solving the calibration plane equation;

[0115] Select at least four groups of two-dimensional coordinates corresponding on the calibration plane and the image plane of the calibration object, and solve the homography matrix representing the transformation relationship between the calibration plane and the image plane of the calibration object.

[0116] According to a specific embodiment of the present invention, an industrial camera is used to photograph the calibration object to obtain image information of the calibration object on the image plane of the photographing device, and distance measurement is performed by a laser ranging device.

[0117] According to a specific embodiment of the present invention, the ranging device synchronously measures and obtains measurement information for calibration, including the following steps:

[0118] Emit laser light from the ranging points on the laser ranging device and irradiate it on the calibration object to form at least 3 calibration points. All the calibration points are located within the calibration plane and not all on the same straight line;

[0119] Use the laser ranging device to obtain multiple straight-line distances between one or more ranging points and the calibration points, and determine the relative position of the straight line and the optical axis of the photographing device.

[0120] When obtaining multiple straight-line distances between one or more ranging points and the calibration points, all the ranging points required for calibration are set in a plane parallel to the image plane. Taking the intersection point O of the optical axis of the photographing device and this plane as the origin, a two-dimensional coordinate system xOy is established in this plane. Then the ranging point C i has coordinates (X i , Y i ), where i ∈ [1, 2,..., N] and N ≥ 1.

[0121] Alternatively, when obtaining multiple straight-line distances between one or more ranging points and the calibration points, the relative position of the optical axis of the photographing device and the straight line where the laser is located is set to be parallel. When the straight-line distance between each calibration point P i and the corresponding ranging point C i measured by the ranging device is D i , taking the intersection point O of the optical axis of the photographing device and this plane as the origin, a three-dimensional coordinate system is established to obtain the three-dimensional coordinates (X i , Y i , D i ) of the calibration point P i in this coordinate system, where i ∈ [1, 2,..., N] and N ≥ 3.

[0122] According to a specific embodiment of the present invention, the acquisition of the calibration plane position information includes the following steps:

[0123] Determine the position of the calibration points;

[0124] Solve the three parameters A, B, and C in the equation Ax + By + Cz = 1 of the calibration plane in the camera coordinate system;

[0125] Among them,

[0126] x, y, and z are the coordinates of the three coordinate axes of the camera coordinate system;

[0127] A, B, and C are the parameters of each coordinate axis respectively.

[0128] According to a specific implementation of the present invention, determining the position of the calibration point includes: setting the optical center O c of the imaging device as the reference point, then both the reference point and the intersection point O of the optical axis of the imaging device and the plane where the ranging point is located are on the optical axis of the imaging device;

[0129] Taking the optical center O c of the reference point as the origin to establish a three-dimensional coordinate system as the camera coordinate system. Then, in this coordinate system, the coordinate of the ranging point C i is (X i , Y i , d). When and only when the laser is parallel to the optical axis, the coordinate of the calibration point P i is (X i , Y i , D i + d), i ∈ [1, 2,..., N] and N ≥ 3. Otherwise, adjust the coordinates of the ranging point and the calibration point according to the angle between the laser and the optical axis;

[0130] Among them,

[0131] (X i , Y i , d) is the coordinate of the ranging point C i in the camera coordinate system;

[0132] (X i , Y i , D i + d) is the coordinate of the calibration point P i in the camera coordinate system;

[0133] d is the distance between the reference point and the plane where the ranging point is located;

[0134] D i is the straight-line distance measured by the ranging device between each calibration point P i and the corresponding ranging point C i ;

[0135] N is the number of calibration points;

[0136] i is the calibration point serial number.

[0137] According to a specific implementation of the present invention, solving the equation of the calibration plane in the camera coordinate system includes the following steps:

[0138] In the camera coordinate system, the coordinates of the calibration point P i (X i , Y i , D i +d) are substituted into the calibration plane equation Ax + By + Cz = 1 to obtain the following formula,

[0139]

[0140] where,

[0141] x, y, and z are the coordinates of the three coordinate axes in the camera coordinate system;

[0142] A, B, and C are the parameters of each coordinate axis in the calibration plane equation, respectively;

[0143] i is the serial number of the calibration point, i ∈ [1, 2... N] and N ≥ 3;

[0144] d is the distance between the reference point and the plane where the ranging point is located;

[0145] D i is the straight-line distance measured by the ranging device from each calibration point P i to the corresponding ranging point C i ;

[0146] N is the number of calibration points;

[0147] Solve for the three parameters A, B, and C to determine the plane equation of the calibration plane in the camera coordinate system;

[0148] According to a specific implementation of the present invention, when solving the three parameters A, B, and C of the above calibration plane equation, the linear equation system

[0149]

[0150] is simplified to

[0151] XΨ = E

[0152] to obtain

[0153]

[0154] where,

[0155]

[0156]

[0157]

[0158] N is the number of calibration points;

[0159] Solve the coefficient matrix Ψ to determine the plane equation of the calibration plane in the camera coordinate system.

[0160] The distance d between the plane where the reference point and the ranging point are located is fixed with the structure of the calibration system. Determining the distance d between the plane where the reference point and the ranging point are located completes the acquisition of the position information of the calibration plane. For the assembled self-calibration system, the actual d value is also fixed. Therefore, an initial calibration can be performed on the instrument system to determine the actual d value, and this value can be substituted into the solved calibration plane equation during subsequent shooting and measurement, thereby completing the image self-calibration. In short, the distance d value between the plane where the reference point and the ranging point are located is only related to the structure of the calibration device, and only one calculation of the d value is required after the device is solidified.

[0161] According to a specific embodiment of the present invention, at least four sets of two-dimensional coordinates are obtained based on the image information of the calibration object and the solved calibration plane equation, and the homography matrix is solved.

[0162] According to a specific embodiment of the present invention, the acquisition of each set of two-dimensional coordinates includes the following steps:

[0163] According to the image information of the calibration object, the calculation point j in the image plane is obtained through manual or image recognition technology k The two-dimensional coordinates j in the pixel coordinate system k (x k , y k );

[0164] Convert the two-dimensional coordinates into the three-dimensional coordinates of the calculation point j k in the above camera coordinate system k (x k , y k , f), k ∈ [1, 2,..., M] and M ≥ 4;

[0165] Obtain the corresponding point J of the calculation point j k on the calibration plane k The three-dimensional coordinates in the camera coordinate system are (Z k / f · x k , Z k / f · y k , Z k ), and M ≥ 4, k ∈ [1, 2,..., M];

[0166] Substitute the coordinates of point J k into the calibration plane equation to get A · Z k / f · x k + B · Z k / f · y k + C · Z k = 1;

[0167] Solve the equation for the only variable Z k , and obtain the corresponding point J k The three-dimensional coordinates in the camera coordinate system, and obtain the corresponding point J k The two-dimensional coordinates in the calibration plane coordinate system;

[0168] k is the serial number of the two-dimensional coordinate group;

[0169] M is the number of the two-dimensional coordinate groups.

[0170] According to a specific embodiment of the present invention, the homography matrix solution includes the following process:

[0171] The known conversion formula from the world coordinate system to the pixel coordinate system is as follows:

[0172]

[0173] Among them, (u, v, 1) T is the homogeneous coordinate of the pixel coordinate system, (x w , y w , z w , 1) T is the homogeneous coordinate of the world coordinate system, K 1 is the camera internal parameter matrix containing the camera internal parameters, K 2 is the camera external parameter matrix containing the camera external parameters, z c is also called the scale factor s; R is the rotation matrix, T is the translation matrix; f is the conversion coefficient from the camera coordinate to the image coordinate; dx is the size of each pixel on the horizontal axis x, and dy is the size of each pixel on the vertical axis y.

[0174] The homogeneous coordinate is a form in which an n-dimensional vector is represented by an n + 1-dimensional vector. Given a point (x, y) on the Euclidean plane, for any non-zero real number w, the triple (xw, yw, w) is called the homogeneous coordinate of the point (x, y).

[0175] Determine an arbitrary world coordinate system position of the calibration object, so that the Z-axis coordinate z w = 0, and the scale factor s does not change the coordinate value corresponding to the homogeneous coordinate, then there is

[0176]

[0177] Among them, H is the homography matrix, including 9 parameters. Perform scaling of any scale in the homogeneous coordinate system so that H has only 8 unknowns;

[0178] (x w , y w ) is the world coordinate of any point on the calibration object, and (u, v) is the pixel coordinate corresponding to this point on the calibration object;

[0179] For such a set of corresponding coordinates (x w ,y w ) and (u,v) can get two equations. Therefore, solving the homography matrix H requires at least 8 equations and four sets of two-dimensional coordinates of corresponding points.

[0180] Obtain at least 4 calculation points j in the image plane from the captured image k The two-dimensional coordinates (u k ,v k ), and transform it into point j according to the camera internal parameters k In the camera coordinate system O c -X c Y c Z c The three-dimensional coordinates (x k ,y k ,f). From the geometric relationship, we can get that point j k Corresponding point J on the object plane XOY i In the camera coordinate system O c -X c Y c Z c The coordinates below are (Z k / f·x k , Z k / f·y k , Z k ), k∈[1,2,…,N] and N≥4. k Substituting the coordinates into the formula Ax+By+Cz=1, we get:

[0181] A.Z k / f·xk+B·Z k / f·y k +C·Z k =1

[0182] The only variable Z in the above equation is solved k , you can get at least 4 points J k The three-dimensional coordinates in the camera coordinate system can then be used to obtain the two-dimensional coordinates of each point on the object plane.

[0183] Substitute the obtained two-dimensional coordinates of at least four groups of corresponding points on the object plane and the image plane into the following conversion formula between pixel coordinates and world coordinates to obtain the homography matrix H: w2p .

[0184]

[0185] Based on the homography matrix H w2p, any arrangement of the world coordinate system on the object surface can be achieved. Assuming that the world coordinate system is rotated by an angle α and translated by Δx and Δy, we can obtain:

[0186]

[0187] For H' w2p Take the inverse to obtain H w2p . H w2p That is the self-calibration result of the final obtained image.

[0188] Referring to the conversion formula between the reference pixel coordinates and the world coordinates, using H w2p , the pixel coordinates of each point in the image can be converted into the world coordinates of the corresponding actual object point of the image point on the object surface.

[0189] The present invention provides a camera self-calibration system, adopting the above-mentioned camera self-calibration method based on distance measurement, including:

[0190] A camera self-calibration device and a calculation module;

[0191] The camera self-calibration device includes a camera module, a distance measurement module, a combination module and a control module;

[0192] The camera module is connected to the combination module and the control module, and is used to photograph the calibration object, obtain the image information of the calibration object, and transmit the image information to the calculation module;

[0193] The distance measurement module is connected to the combination module and the control module, and synchronously measures the calibration object from the distance measurement point on the distance measurement module during shooting, including the distance measurement of the calibration point, and transmits the measurement information to the calculation module;

[0194] The combination module is connected to the camera module, the distance measurement module and the control module, combines the camera module and the distance measurement module into an integrated unit, sets the relative position between the camera module and the distance measurement module, and transmits the relative position information to the calculation module;

[0195] The control module controls the camera module and the distance measurement module to rotate pitch and rotate around the vertical axis as an integrated unit, measures the direction and angle of rotation, and transmits the rotation information to the calculation module;

[0196] The calculation module obtains the image information and the calibration plane information, and realizes the camera self-calibration based on distance measurement.

[0197] According to a specific implementation scheme of the present invention, the camera module includes an industrial camera, and the distance measurement module includes a laser distance measurement device; by adjusting the relative position between the camera self-calibration device and the calibration object through the control module, the camera module is used to photograph the calibration object, obtain the image information of the calibration object, and transmit the relative position and the image information to the calculation module;

[0198] During shooting, a ranging module synchronously measures a calibration object from a ranging point to obtain measurement information, and at the same time, a combination module sets the relative position between the imaging module and the ranging module and obtains relative position information;

[0199] A calculation module determines the relative position between a reference point and a calibration point according to the measurement information and the relative position information, and further obtains the position information of the plane where the calibration point is located;

[0200] The calculation module obtains two-dimensional coordinates according to the acquired image information and position information, and calculates the corresponding homography matrix to realize camera self-calibration based on ranging.

[0201] Embodiment 1

[0202] According to a specific implementation of the present invention, in conjunction with the accompanying drawings, the camera self-calibration method based on ranging of the present invention will be described in detail.

[0203] The present invention provides a camera self-calibration method based on ranging, including the following steps:

[0204] Shoot a calibration object to obtain image information of the calibration object on the image plane of the shooting device; Use an industrial camera to shoot a calibration object to obtain image information of the calibration object on the image plane of the shooting device;

[0205] Synchronously measure the calibration plane position information through a ranging device; Perform ranging through a laser ranging device;

[0206] Solve the calibration plane equation;

[0207] Select at least four groups of two-dimensional coordinates corresponding to each other on the calibration plane and the image plane of the calibration object, and solve the homography matrix representing the change relationship between the calibration plane and the image plane of the calibration object.

[0208] The ranging device synchronously measures to obtain measurement information for calibration, including the following steps:

[0209] Emit laser light from a ranging point on the laser ranging device and irradiate it on the calibration object to form at least 3 calibration points. All the calibration points are located within the calibration plane and not all on the same straight line;

[0210] Use the laser ranging device to obtain multiple straight-line distances between one or more ranging points and the calibration points, and determine the relative position between the straight line and the optical axis of the shooting device.

[0211] When obtaining multiple straight-line distances between one or more ranging points and the calibration points, set the relative position between the optical axis of the shooting device and the straight line where the laser is located to be parallel. When each calibration point P i to the corresponding ranging point C i the straight-line distance between them is Di When taking the intersection point O of the optical axis of the imaging device and this plane as the origin, a three-dimensional coordinate system is established to obtain the calibration point P i In the three-dimensional coordinates (X i , Y i , D i ) in this coordinate system, where i ∈ [1, 2,..., N] and N ≥ 3, and N is the number of calibration points. In the descriptions of other steps, this method is used for all.

[0212] Alternatively, when obtaining the multi-segment straight-line distances from one or more ranging points to the calibration points, all the ranging points required for calibration can be set in a plane parallel to the image plane. Taking the intersection point O of the optical axis of the imaging device and this plane as the origin, a two-dimensional coordinate system xOy is established in this plane. Then, the coordinate of the ranging point C i is (X i , Y i ), where i ∈ [1, 2,..., N] and N ≥ 1, and N is the number of calibration points.

[0213] Obtaining the position information of the calibration plane includes the following steps:

[0214] Determine the position of the calibration point;

[0215] Solve for the three parameters A, B, and C in the equation Ax + By + Cz = 1 of the calibration plane in the camera coordinate system;

[0216] Among them,

[0217] x, y, and z are the coordinates of the three coordinate axes of the camera coordinate system;

[0218] A, B, and C are the parameters of each coordinate axis respectively.

[0219] Determining the position of the calibration point includes: setting the optical center O c of the imaging device as the reference point. Then, both the reference point and the intersection point O of the optical axis of the imaging device and the plane where the ranging points are located are on the optical axis of the imaging device;

[0220] Taking the optical center O c of the reference point as the origin to establish a three-dimensional coordinate system as the camera coordinate system. Then, in this coordinate system, the coordinate of the ranging point C i is (X i , Y i , d). When and only when the lasers are all parallel to the optical axis, the coordinate of the calibration point P i is (X i , Y i , D i +d), where i ∈ [1, 2,..., N] and N ≥ 3. Otherwise, adjust the coordinates of the ranging points and the calibration points according to the angle between the laser and the optical axis;

[0221] Among them,

[0222] (X i , Y i , d) is the coordinate of the ranging point C i in the camera coordinate system;

[0223] (X i , Y i , D i + d) is the coordinate of the calibration point P i in the camera coordinate system;

[0224] d is the distance between the reference point and the plane where the ranging point is located;

[0225] D i is the straight-line distance measured by the ranging device between each calibration point P i and the corresponding ranging point C i ;

[0226] N is the number of calibration points;

[0227] i is the calibration point serial number.

[0228] Solving the equation of the calibration plane in the camera coordinate system includes the following steps:

[0229] In the camera coordinate system, substitute the coordinates (X i , Y i , D i , D i + d) of the calibration point P into the calibration plane equation Ax + By + Cz = 1 to obtain the following formula,

[0230]

[0231] Among them,

[0232] x, y, z are the coordinates of the three coordinate axes of the camera coordinate system;

[0233] A, B, C are the parameters of each coordinate axis in the calibration plane equation respectively;

[0234] i is the calibration point serial number, i ∈ [1, 2…N] and N ≥ 3;

[0235] d is the distance between the reference point and the plane where the ranging point is located;

[0236] D i is the straight-line distance measured by the ranging device between each calibration point P i and the corresponding ranging point C i ;

[0237] N is the number of calibration points;

[0238] Solve for the three parameters A, B, and C to determine the plane equation of the calibration plane in the camera coordinate system;

[0239] When solving for the three parameters A, B, and C in the calibration equation, the linear equations

[0240]

[0241] are simplified and expressed as

[0242] XΨ = E

[0243] to obtain

[0244]

[0245] where

[0246]

[0247]

[0248]

[0249] N is the number of calibration points;

[0250] to obtain

[0251]

[0252] Solve for the coefficient matrix Ψ to determine the plane equation of the calibration plane in the camera coordinate system.

[0253] The distance d between the reference point and the plane where the ranging points are located is fixed with the construction of the calibration system. Determine the distance d between the reference point and the plane where the ranging points are located, and then the acquisition of the position information of the calibration plane is completed. For the assembled self-calibration system, the actual value of d is also fixed. Therefore, the instrument system can be initially calibrated once to determine the actual value of d, and this value can be substituted into the solved calibration plane equation during subsequent shooting and measurement, thereby completing the image self-calibration. In short, the value of the distance d between the reference point and the plane where the ranging points are located is only related to the construction of the calibration device, and only one calculation of the d value is required after the device is solidified.

[0254] According to the image information of the calibration object and the solved calibration plane equation, obtain at least four groups of two-dimensional coordinates, and solve the homography matrix to achieve camera self-calibration based on ranging.

[0255] The process of solving the homography matrix is as follows:

[0256] Obtain at least 4 calculation points j in the image plane from the captured images k The two-dimensional coordinates (u k , v k), and convert it to point j according to the internal parameters of the camera k In the camera coordinate system O c -X c Y c Z c The three-dimensional coordinates (x k , y k , f). From the geometric relationship, it can be obtained that point j k The corresponding point J on the object plane XOY i In the camera coordinate system O c -X c Y c Z c The coordinates below are (Z k / f·x k , Z k / f·y k , Z k ), k ∈ [1, 2,..., N] and N ≥ 4. Substitute the coordinates of point J k into the equation Ax + By + Cz = 1 to get:

[0257] A·Z k / f·x k +B·Z k / f·y k +C·Z k = 1

[0258] Solve for the only variable Z in the above equation k , and then the three-dimensional coordinates of at least 4 points J k in the camera coordinate system can be obtained respectively, and then the two-dimensional coordinates of each point on the object plane can be obtained.

[0259] Substitute the two-dimensional coordinates of at least 4 groups of corresponding points on the object plane and the image plane into the following conversion formula between pixel coordinates and world coordinates respectively, and the homography matrix H w2p can be solved.

[0260]

[0261] Based on the homography matrix H w2p , any arrangement of the world coordinate system on the object plane can be realized. Assume that the world coordinate system is rotated by an angle α and translated by Δx and Δy, and we can get:

[0262]

[0263] Take the inverse of H' w2p to get H w2p . H w2p is the final self-calibration result of the image.

[0264] Refer to the conversion formula between pixel coordinates and world coordinates and apply Hw2p , the pixel coordinates of each point in the image can be converted into the world coordinates of the corresponding actual object point of the image point on the object plane.

[0265] The acquisition of each group of two-dimensional coordinates includes the following steps:

[0266] According to the image information of the calibration object, the calculation point j in the image plane is obtained through manual or image recognition technology k The two-dimensional coordinates j in the pixel coordinate system k (x k , y k );

[0267] According to the camera internal parameters, the two-dimensional coordinates are converted into the three-dimensional coordinates of the calculation point j k in the above camera coordinate system k (x k , y k , f), k ∈ [1, 2,..., M] and M ≥ 4;

[0268] Obtain the corresponding point J of the calculation point j k in the calibration plane k The three-dimensional coordinates in the camera coordinate system are (Z k / f · x k , Z k / f · y k , Z k ), and M ≥ 4, k ∈ [1, 2,..., M];

[0269] Substitute the coordinates of point J k into the calibration plane equation to get A · Z k / f · x k + B · Z k / f · y k + C · Z k = 1;

[0270] Solve the only variable Z of the equation k , obtain the three-dimensional coordinates of the corresponding point J k in the camera coordinate system, and obtain the two-dimensional coordinates of the corresponding point J k in the calibration plane coordinate system;

[0271] Among them,

[0272] k is the serial number of the two-dimensional coordinate group;

[0273] M is the number of two-dimensional coordinate groups.

[0274] Embodiment 2

[0275] According to a specific embodiment of the present invention, in combination with the accompanying drawings, the rangefinder-based camera self-calibration system of the present invention will be described in detail. In this embodiment, the rangefinding device is a laser rangefinder 5. The laser rangefinder camera self-calibration device is as Figure 3 shown.

[0276] The present invention provides a camera self-calibration system, which adopts the above-mentioned rangefinder-based camera self-calibration method, including:

[0277] A camera self-calibration device and a calculation module;

[0278] The camera self-calibration device includes a camera module, a rangefinding module, a combination module and a control module; the camera module includes an industrial camera, and the rangefinding module includes a laser rangefinding device;

[0279] The camera module is connected to the combination module and the control module, and is used to photograph the calibration object, obtain the image information of the calibration object, and transmit the image information to the calculation module;

[0280] The rangefinding module is connected to the combination module and the control module, and synchronously measures the calibration object from the rangefinding point on the rangefinding module during shooting, including measuring the distance to the calibration point, and transmitting the measurement information to the calculation module;

[0281] The combination module is connected to the camera module, the rangefinding module and the control module, combines the camera module and the rangefinding module into a whole, sets the relative position between the camera module and the rangefinding module, and transmits the relative position information to the calculation module;

[0282] The control module controls the camera module and the rangefinding module to rotate pitch and rotate around the vertical axis as a whole, measures the direction and angle of rotation, and transmits the rotation information to the calculation module;

[0283] The calculation module obtains the image information and the calibration plane information, and realizes the rangefinder-based camera self-calibration.

[0284] By adjusting the relative position between the camera self-calibration device and the calibration object through the control module, the camera module is used to photograph the calibration object, obtain the image information of the calibration object, and transmit the relative position and the image information to the calculation module;

[0285] During shooting, the rangefinding module is used to synchronously measure the calibration object from the rangefinding point and obtain the measurement information. At the same time, the combination module sets the relative position between the camera module and the rangefinding module and obtains the relative position information;

[0286] The calculation module determines the relative position between the reference point and the calibration point according to the measurement information and the relative position information, and further obtains the position information of the plane where the calibration point is located;

[0287] The calculation module obtains two-dimensional coordinates based on the acquired image information and position information, and calculates the corresponding homography matrix to achieve camera self-calibration based on ranging.

[0288] Embodiment 3

[0289] To verify the technical solution of the present invention, a camera self-calibration accuracy test based on length measurement was carried out. By measuring a line segment with a known length, the image self-calibration experiment was carried out in the most basic way, and Figure 4 a pattern composed of five line segments AB, CG, DH, EI, and FJ as shown was designed. The actual length of each line segment is 96.0 millimeters. Length measurement is to convert the pixel length of the object in the image into the actual width of the object through the calibration result.

[0290] Using the self-calibration method proposed by the present invention, the measured length of the line segment calculated from the captured image was compared with the true length of the line segment to verify the accuracy of the laser ranging self-calibration method. During the experiment, by adjusting the distance between the object to be measured and the laser rangefinder, shooting and measurement were achieved in five different scenarios (see Table 1 in detail) to comprehensively verify the effectiveness of the method of the present invention.

[0291] The test steps are as follows:

[0292] (1) Build a camera self-calibration system based on laser ranging;

[0293] (2) Through the control module, adjust the relative position of the device and the calibration object, use the imaging module to shoot the calibration object, obtain image information and transmit it to the calculation module;

[0294] (3) The combination module pre-sets the relative position information between the imaging module and the ranging module. During shooting, the ranging module is used for synchronous ranging to obtain measurement information, and the two types of information are transmitted to the calculation module;

[0295] (4) After the calculation module obtains the above measurement information and relative position information, it iteratively solves the distance between the reference point and the plane where the ranging point is located according to the standard line segment, and then solves the plane equation of the calibration plane;

[0296] (5) The calculation module obtains at least four groups of two-dimensional coordinates based on the image information and the calibration plane equation, and calculates the homography matrix from the image plane to the calibration plane through at least four groups of two-dimensional coordinates;

[0297] (6) After sequentially selecting the two endpoints of a line segment to be measured in the image, the calculation module calculates the line segment length between the two endpoints according to the calibration result and visualizes the result;

[0298] (7) Repeat steps (2) to (6) until the test ends.

[0299] For five different test scenarios, the distances between the laser rangefinder and the measured object surface are designed to be 245 mm, 345 mm, 445 mm, 545 mm, and 645 mm respectively. The actually measured distances of each laser rangefinder are summarized in Table 1, and the measurement errors are basically within 1 mm, and the measurement errors at individual points are 2 mm.

[0300] Table 1 Laser ranging values under different test scenarios

[0301]

[0302] Table 2 shows the comparison results of the measured values and the true values of the lengths of each line segment. It can be seen that all the measured values are very close to the true value of 96 mm, and the measurement errors are between -0.49% and 0.15%, and the average error is only -0.14%. It can be seen that the self-calibration method of the present invention has good accuracy and stability.

[0303] Table 2 Measurement results of line segment lengths

[0304]

[0305] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A camera self - calibration method based on distance measurement, characterized in that, it includes the following steps: Shoot a calibration object to obtain the image information of the calibration object on the image plane of the shooting device; Synchronously measure the position information of the calibration plane through a distance - measuring device; Solve the calibration plane equation; Select at least four groups of two - dimensional coordinates corresponding on the calibration plane and the image plane of the calibration object, and solve the homography matrix representing the transformation relationship between the calibration plane and the image plane of the calibration object; Among them, the distance - measuring device synchronously measures and obtains the measurement information for calibration, including the following steps: Emit laser light from the distance - measuring points on the laser distance - measuring device and irradiate it on the calibration object, forming at least 3 calibration points. All the calibration points are located within the calibration plane and not all on the same straight line; Use the laser distance - measuring device to obtain multiple straight - line distances between one or more distance - measuring points and the calibration points, and determine the relative position of the straight line and the optical axis of the shooting device; The acquisition of the calibration plane position information includes the following steps: Determine the position of the calibration points; Solve the three parameters A, B, and C in the equation Ax + By + Cz = 1 of the calibration plane in the camera coordinate system; Among them, x, y, and z are the coordinates of the three coordinate axes of the camera coordinate system; A, B, and C are the parameters of each coordinate axis respectively; Determining the position of the calibration points includes: Set the optical center O of the imaging device c as the reference point, then both the reference point and the intersection point O of the optical axis of the imaging device and the plane where the ranging point is located are on the optical axis of the imaging device; Taking the optical center O of the reference point c as the origin to establish a three-dimensional coordinate system as the camera coordinate system. Then, in this coordinate system, the ranging point C i has coordinates (X i , Y i , d). When and only when the lasers are all parallel to the optical axis, the calibration point P i has coordinates (X i , Y i , D i + d), i ∈ [1, 2, …, N] and N ≥ 3. Otherwise, adjust the coordinates of the ranging point and the calibration point according to the angle between the laser and the optical axis; Among them, (X i , Y i , d) is the coordinate of the ranging point C i in the camera coordinate system; (X i , Y i , D i + d) is the coordinate of the calibration point P i in the camera coordinate system; d is the distance between the reference point and the plane where the distance - measuring points are located; D i The straight-line distance between each calibrated point P i measured by the ranging device and the corresponding ranging point C i ; N is the number of calibration points; i is the serial number of the calibration point.

2. The camera self - calibration method based on distance measurement according to claim 1, characterized in that, Use an industrial camera to shoot the calibration object to obtain the image information of the calibration object on the image plane of the shooting device, and perform distance measurement through a laser distance - measuring device.

3. The camera self - calibration method based on distance measurement according to claim 1, characterized in that, Solving the equation of the calibration plane in the camera coordinate system includes the following steps: In the camera coordinate system, the coordinates of the calibration point P i (X i , Y i , D i + d) are substituted into the calibration plane equation Ax + By + Cz = 1 to obtain the following formula: Among them, x, y, and z are the coordinates of the three coordinate axes of the camera coordinate system; A, B, and C are the parameters of each coordinate axis in the calibration plane equation respectively; i is the serial number of the calibration point, i ∈ [1, 2…N] and N≥3; d is the distance between the reference point and the plane where the distance - measuring points are located; D i The straight-line distance between each calibrated point P i measured by the ranging device and the corresponding ranging point C i ; N is the number of calibration points; Solve the three parameters A, B, and C to determine the plane equation of the calibration plane in the camera coordinate system.

4. The camera self - calibration method based on distance measurement according to claim 3, characterized in that, According to the image information of the calibration object and the solved calibration plane equation, obtain at least four groups of two - dimensional coordinates and solve the homography matrix.

5. The camera self - calibration method based on distance measurement according to claim 4, characterized in that, The acquisition of each group of two - dimensional coordinates includes the following steps: Based on the image information of the calibration object, the calculation point j in the image plane is obtained manually or through image recognition technology k The two-dimensional coordinates j in the pixel coordinate system k (x k , y k ); Convert this two-dimensional coordinate into the calculation point j according to the camera internal parameters k The three-dimensional coordinate j in the above camera coordinate system k (x k , y k , f), k ∈ [1, 2, …, M] and M ≥ 4; Obtain the calculation point j k The corresponding point J on the calibration plane k The three-dimensional coordinates in the camera coordinate system are (Z k / f·x k , Z k / f·y k , Z k ), and M≥4, k∈[1, 2, …, M]; Substitute the coordinates of point J k into the calibration plane equation to obtain A·Z k / f·x k +B·Z k / f·y k +C·Z k = 1; Solve the equation for the only variable Z k , and obtain the corresponding point J k The three-dimensional coordinates in the camera coordinate system, and obtain the corresponding point J k The two-dimensional coordinates in the calibration plane coordinate system; k is the serial number of the two - dimensional coordinate group; M is the number of two - dimensional coordinate groups.

6. A camera self - calibration system, characterized in that, Adopt the camera self - calibration method based on distance measurement according to any one of claims 1 - 5, including: A camera self - calibration device and a calculation module; The camera self - calibration device includes a camera module, a distance - measuring module, a combination module, and a control module; The camera module is connected to the combination module and the control module, and is used to shoot the calibration object, obtain the image information of the calibration object, and transmit the image information to the calculation module; The ranging module is connected to the combination module and the control module. During shooting, the ranging points on the ranging module are used to synchronously measure the calibration object, including measuring the distance to the calibration points, and transmitting the measurement information to the calculation module; The combination module is connected to the imaging module, the ranging module and the control module, combines the imaging module and the ranging module into an integrated unit, sets the relative position between the imaging module and the ranging module, and transmits the relative position information to the calculation module; The control module controls the imaging module and the ranging module as an integrated unit to perform pitching rotation and rotation about the vertical axis, measures the direction and angle of rotation, and transmits the rotation information to the calculation module; The calculation module acquires image information and calibration plane information to achieve camera self-calibration based on ranging.

7. The camera self-calibration system according to claim 6, wherein, the imaging module includes an industrial camera, and the ranging module includes a laser ranging device; By adjusting the relative position between the camera self-calibration device and the calibration object through the control module, the imaging module is used to shoot the calibration object to obtain the image information of the calibration object, and the relative position and image information are transmitted to the calculation module; During shooting, the ranging module is used to synchronously measure the calibration object from the ranging points and obtain the measurement information. At the same time, the combination module is used to set the relative position between the imaging module and the ranging module and obtain the relative position information; The calculation module determines the relative position between the reference point and the calibration point according to the measurement information and the relative position information, and further obtains the position information of the plane where the calibration point is located; The calculation module acquires at least four groups of two-dimensional coordinates according to the acquired image information and position information, and calculates the corresponding homography matrix to achieve camera self-calibration based on ranging.

Images