Camera intrinsic parameter self-calibration method and device based on quadratic cone invariant

Abstract

The present application relates to a method and device for self-calibration of camera intrinsic parameters based on quadratic conical surface invariants. The method comprises: when the radius of a circle is known, obtaining three photos of three circles with the same radius on the same plane at different angles; performing elliptical curve fitting based on the three photos to obtain a corresponding matrix A of more than 9 elliptical equations in a pixel coordinate system; calculating a conical surface invariant consisting of more than 9 groups of ellipses corresponding to the camera pinhole and the pinhole imaging of the circle based on the corresponding matrix A of the more than 9 elliptical equations in the pixel coordinate system and the conical surface equation consisting of the ellipse corresponding to the camera pinhole and the pinhole imaging of the circle; calculating the intrinsic parameters of the camera based on the relationship between the conical surface invariants and the conical surface invariants. The self-calibration method does not require the relative positions of the circles and the corresponding world coordinates, but only requires the radius of the circle and the elliptical equation of each circle in the corresponding pixel coordinates. It does not require precise matching of the circles, making the application scenario more flexible.

Description

Technical Field

[0001] The present application relates to the technical field of camera calibration, and in particular to a method and device for self-calibration of camera intrinsic parameters based on quadratic cone invariants. Background Art

[0002] With the development and rise of technologies such as the Internet of Things, robots with vision, unmanned driving, and drones, more and more video data and image data need to be processed, which makes visual computing more and more important. Camera self-calibration in these scenarios is a key step in extracting real measurement information from two-dimensional images in visual computing. It is widely used in three-dimensional reconstruction, navigation, visual monitoring and other fields, which makes camera self-calibration very important. At present, the representative research teams of self-calibration in the world include Andrew Zisserman from Oxford University in the UK, Luc Van Gool from Belgium, Anders Heyden from Sweden, Marc Pollefeys from Switzerland, Faugeras, Triggs and Sturm from France, Ma Yi from the University of Illinois in the United States, Richard I. Hartley from Australia, Amnon Shashua from Israel, Zhang Zhengyou, director of Tencent AI Laboratory in my country, and Meng Xiaoqiao and Hu Zhanyi's team from the Chinese Academy of Sciences, etc.

[0003] Most of the traditional self-calibration methods involve Kruppa equation, hierarchical step-by-step calibration, vanishing points, vanishing lines, absolute quadratic curves and absolute quadratic surfaces. Due to the discreteness of pixel coordinates, the calculations related to these concepts are prone to large deviations. In 2004, Zhang Zhengyou proposed a camera self-calibration based on a one-dimensional motion calibration object. This method does not involve the related concepts mentioned above. It only needs to consider the imaging of the six states of the line segment with three fixed points rotating around one of the points and the position ratio of the points on the line segment in the world coordinates to calculate the intrinsic parameters. This method is difficult to convert into the calibration of a single image, and the calculation error cannot be ignored in the presence of noise. Summary of the invention

[0004] Based on this, it is necessary to provide a camera intrinsic parameter self-calibration method and device based on quadratic cone invariants to address the above technical problems.

[0005] A camera intrinsic parameter self-calibration method based on quadratic cone invariants, the method comprising:

[0006] When the radius of the circle is known, obtain three photos of three circles with the same radius on the same plane at different angles.

[0007] Elliptic curve fitting is performed based on the three photos to obtain a corresponding matrix A of a predetermined number of elliptic equations in a pixel coordinate system; the predetermined number is three times the number of circles in the same plane.

[0008] According to the matrix A corresponding to a predetermined number of ellipse equations in the pixel coordinate system and the intrinsic parameters, the cone surface equation formed by the ellipse corresponding to the camera pinhole and the pinhole imaging of the circle can be obtained, and the cone surface invariant formed by the ellipse corresponding to the camera pinhole and the pinhole imaging of the circle and a predetermined number of groups or more can be calculated; the cone surface invariant is an expression about the camera intrinsic parameters.

[0009] The intrinsic parameters of the camera are calculated based on the relationship between the cone surface invariants formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging and the cone surface invariants; the relationship between the cone surface invariants is:

[0010]

[0011] Among them, I1, I2, and I3 are three cone invariants; z0 is the distance from the small hole to the plane where the circle is located; R0 is the radius of the circle on the plane.

[0012] In one embodiment,

[0013] According to the matrix A and the internal parameters corresponding to 9 or more ellipse equations in the pixel coordinate system, the cone surface equation formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging can be obtained, so as to calculate the cone surface invariant formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging. The cone surface equation formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging in the step is:

[0014]

[0015] Among them, (xyz) is the coordinate of point P on the cone surface in the world coordinate system, α, β are the scale factors of the image (u, v) coordinate system, (u0, v0) are the coordinates of the main point of the image, and γ is the skew parameter. is the corresponding matrix A of the ellipse equation in the pixel coordinate system.

[0016] The cone invariant formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging is:

[0017] I1=a 11 α 2 +a 33 +u0(a 13 +a 11 u0+a 12 v0)+v0(a 23 +a 12 u0+a 22 v0)+a 13 u0

[0018] +a 23 v0+β(a 22 β+a 12 γ)+γ(a 12 β+a 11 γ)

[0019]

[0020] Among them: a 11 、a 12 、a 13 a 12 、a 22 、a 23 、a 33 is the element in the matrix A corresponding to the elliptic equation.

[0021] In one embodiment, the intrinsic parameters of the camera are calculated based on the relationship between the cone surface invariants formed by the camera pinhole and the ellipse corresponding to the pinhole imaging of the circle and the cone surface invariants, including:

[0022] The ellipses corresponding to the pinhole imaging of three circles on the same plane and the cone surface corresponding to the camera pinhole are selected to form cone surface invariants, and 9 groups of the cone surface invariants in the pixel coordinate system are obtained.

[0023] Substituting the 9 groups of cone surface invariants into the relationship between the cone surface invariants, 9 equations with the camera intrinsic parameters and the distance from the pinhole to the plane where the circle is located as unknowns are obtained.

[0024] The nine equations are solved using a nonlinear least squares solution method to obtain the intrinsic parameters of the camera.

[0025] In one embodiment,

[0026] When the radius of the circle is unknown, the photos obtained are 3 photos of 4 circles with the same radius on the same plane at different angles.

[0027] The intrinsic parameters of the camera are calculated based on the relationship between the cone surface invariants formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging and the cone surface invariants, and the step also includes:

[0028] Four equations are constructed based on the relationship between the cone invariants and the cone invariants corresponding to the cone formed by the camera pinhole and the four ellipses on the same photo; the four equations are:

[0029]

[0030] Eliminate R0 and z0 from the four equations and we get equations that only include cone surface invariants.

[0031]

[0032] Among them, I ij is the i-th invariant corresponding to the j-th circle on the plane, i=1,2,3,4j=1,2,3,4.

[0033] In one embodiment, the intrinsic parameters of the camera are calculated based on the relationship between the cone surface invariants formed by the camera pinhole and the ellipse corresponding to the pinhole imaging of the circle and the cone surface invariants, including:

[0034] Substitute the cone invariants corresponding to the cone formed by the camera aperture and the four ellipses on the same photo into the equation that only includes the cone invariants, and obtain two equations with the camera intrinsic parameters as unknowns, and a total of 6 equations are obtained for the three photos.

[0035] The six equations are solved using a nonlinear least squares solution method to obtain the intrinsic parameters of the camera.

[0036] A camera intrinsic parameter self-calibration device based on quadratic cone invariant, the device comprising:

[0037] The calibration photo acquisition module is used to obtain three photos of three circles with the same radius on the same plane at different angles when the radius of the circle is known.

[0038] The module for determining the corresponding matrix A of the ellipse equation is used to perform ellipse curve fitting based on three photos to obtain the corresponding matrices A of a predetermined number of ellipse equations in the pixel coordinate system; the predetermined number is three times the number of circles in the same plane.

[0039] The cone surface invariant expression determination module is used to calculate the cone surface invariant composed of a predetermined number of groups or more of ellipses corresponding to the camera pinhole and the circular pinhole imaging according to the matrix A corresponding to the predetermined number of ellipse equations in the pixel coordinate system and the cone surface equation composed of the ellipse corresponding to the camera pinhole and the circular pinhole imaging; the cone surface invariant is an expression about the camera intrinsic parameter.

[0040] The camera's intrinsic parameter determination module is used to calculate the camera's intrinsic parameters based on the relationship between the camera's pinhole and the cone invariant formed by the ellipse corresponding to the pinhole imaging of the circle and the cone invariant; the relationship between the cone invariants is:

[0041]

[0042] Among them, I1, I2, and I3 are three cone invariants; z0 is the distance from the small hole to the plane where the circle is located; R0 is the radius of the circle on the plane.

[0043] In one embodiment,

[0044] The cone surface equation formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging in the cone surface invariant expression determination module is:

[0045]

[0046] Among them, (xyz) is the coordinate of point P on the cone surface in the world coordinate system, α and β are the scale factors of the image (u,v) coordinate system, (u0,v0) is the coordinate of the main point of the image, and γ is the skew parameter. is the matrix A corresponding to the ellipse equation in the pixel coordinate system.

[0047] The cone invariant formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging is:

[0048] I1=a 11 α 2 +a 33 +u0(a 13 +a 11 u0+a 12 v0)+v0(a 23 +a 12 u0+a 22 v0)+a 13 u0

[0049] +a 23 v0+β(a 22 β+a 12 γ)+γ(a 12 β+a 11 γ)

[0050]

[0051] Among them: a 11 、a 12 、a 13 a 12 、a 22 、a 23 、a 33 is the A element of the matrix corresponding to the elliptic equation.

[0052] In one of the embodiments, the camera's intrinsic parameter determination module is also used to select the ellipse corresponding to the pinhole imaging of three circles with the same radius on the same plane and the cone invariant corresponding to the cone formed by the camera pinhole, and obtain 9 groups of the cone invariants in the pixel coordinate system; substitute the 9 groups of the cone invariants into the relationship between the cone invariants to obtain 9 equations with the camera intrinsic parameters and the distance from the pinhole to the plane where the circle is located as unknowns; use the nonlinear least squares solution method to solve the 9 equations to obtain the camera's intrinsic parameters.

[0053] In one embodiment, when the radius of the circle is unknown, the acquired photos are three photos of four circles with the same radius on the same plane at different angles; the camera intrinsic parameter determination module also includes: a module for determining the relationship between cone surface invariants in the same photo, which is used to construct four equations according to the relationship between the cone surface invariants corresponding to the cone surface formed by the camera pinhole and the four ellipses on the same photo and the cone surface invariants; the four equations are:

[0054]

[0055] Eliminate R0 and z0 from the four equations and we get equations that only include cone surface invariants.

[0056]

[0057] Among them, I ij is the i-th invariant corresponding to the j-th circle on the plane, i=1,2,3,4j=1,2,3,4.

[0058] In one of the embodiments, the camera's intrinsic parameter determination module is also used to substitute the cone invariants corresponding to the cone formed by the camera aperture and four ellipses on the same photo into the equation that only includes the cone invariants, so as to obtain 6 equations with the camera's intrinsic parameters as unknowns; and use a nonlinear least squares solution method to solve the 6 equations to obtain the camera's intrinsic parameters.

[0059] The above-mentioned camera intrinsic parameter self-calibration method and device based on quadratic conical surface invariants, the method includes: when the radius of a circle is known, obtain three photos of three circles with the same radius on the same plane at different angles; perform elliptical curve fitting based on the three photos to obtain the corresponding matrix A of more than 9 elliptical equations in the pixel coordinate system; calculate the conical surface invariants composed of more than 9 groups of ellipses corresponding to the camera pinhole and the pinhole imaging of the circle based on the corresponding matrix A of the more than 9 elliptical equations in the pixel coordinate system and the conical surface equation composed of the ellipse corresponding to the camera pinhole and the pinhole imaging of the circle; calculate the intrinsic parameters of the camera based on the relationship between the conical surface invariants composed of the ellipse corresponding to the camera pinhole and the pinhole imaging of the circle and the conical surface invariants. This self-calibration method does not require the relative positions of the circles and the corresponding world coordinates, but only requires the radius of the circle and the elliptical equation of each circle in the corresponding pixel coordinates. It does not require precise matching of the circles, making the application scenario more flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 1 is a flow chart of a method for self-calibration of camera intrinsic parameters based on quadratic cone invariants in one embodiment;

[0061] Figure 24 is a structural block diagram of a camera intrinsic parameter self-calibration device based on quadratic cone invariants in one embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] The process of digital camera image capture is actually an optical imaging process. The imaging process of the camera involves four coordinate systems: world coordinate system, camera pinhole coordinate system, imaging plane coordinate system, pixel coordinate system and the conversion of these four coordinate systems.

[0064] The imaging of ordinary cameras is generally regarded as pinhole imaging superimposed with sensor imaging. Before pinhole imaging, there is a coordinate system in the space where the photographed object is located called the world coordinate system. There is a coordinate system at the position of the lens pinhole called the camera pinhole coordinate system. The coordinate system established on the receiving plane of pinhole imaging is the imaging plane coordinate system. After pinhole imaging, the sensor imaging principle is used to stretch, rotate and translate to obtain a pixel image. The coordinate system established according to the row and column positions of the pixel image is called the pixel coordinate system.

[0065] In one embodiment, Figure 1 As shown, a camera intrinsic parameter self-calibration method based on quadratic cone invariant is provided, and the method comprises the following steps:

[0066] Step 100: When the radius of the circle is known, obtain three photos of three circles with the same radius on the same plane at different angles.

[0067] Step 102: Perform elliptic curve fitting based on the three photos to obtain a corresponding matrix A of a predetermined number of ellipse equations in a pixel coordinate system; the predetermined number is three times the number of circles in the same plane.

[0068] Specifically, the elliptic curve fitting equation in the pixel coordinate system includes the following steps:

[0069] 1) Obtain a binary image;

[0070] 2) Extract ellipses from images using connected sets;

[0071] 3) Using the idea of ​​quadratic interpolation to extract the sub-pixel boundary of the image, the ellipse is morphologically expanded, and the sub-pixel boundary points within the ellipse contour polygon are found as the improved contour of the ellipse, and then the flexible boundary of the contour is found to remove the interference points within the ellipse;

[0072] 4) Perform polygonal line segment interpolation on the boundary to make the boundary points sparse and uniform;

[0073] 5) Use the iterative method of nonlinear least squares to solve the general equation of the ellipse with a fixed constant term;

[0074] 6) The corresponding matrix A is obtained from this.

[0075] The replacement method can use the idea of ​​concentric circle sub-pixel contour extraction to more accurately calculate the ellipse corresponding matrix A, refer to the relevant literature on concentric circle sub-pixel contour extraction.

[0076] Step 104: The cone surface equation formed by the ellipses corresponding to the camera pinhole and the pinhole imaging of the circle can be obtained according to the matrix A and the intrinsic parameters corresponding to the predetermined number of ellipse equations in the pixel coordinate system, and the cone surface invariants formed by the ellipses corresponding to the camera pinhole and the pinhole imaging of the circle and the predetermined number of groups or more can be calculated; the cone surface invariants are expressions about the camera intrinsic parameters.

[0077] Step 106: Calculate the intrinsic parameters of the camera based on the relationship between the cone invariants formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging and the cone invariants; the relationship between the cone invariants is:

[0078]

[0079] Among them, I1, I2, and I3 are three cone invariants; z0 is the distance from the small hole to the plane where the circle is located; R0 is the radius of the circle on the plane.

[0080] Specifically, the invariant of the cone formed by the camera pinhole and the circle is given by The derivation principle is:

[0081] Assume that there is a circle with radius R0 on the horizontal plane of the world coordinate system, and the center of the circle is (x c ,y c ), the distance from the small hole to the plane is z0, combined with the world coordinates of the small hole, the origin of the world coordinate system is translated to the small hole, then the equation of the cone formed by the circle and the small hole in this coordinate system is

[0082]

[0083] The quadratic surface has rotation and translation invariants. I1′ is the sum of the diagonal elements of the third-order matrix in the above formula, I2′ is the sum of the second-order cofactors of the diagonal elements of the matrix, and I3′ is the determinant of the matrix, that is,

[0084]

[0085] Since the equations may differ by a constant coefficient k, the invariant will also change accordingly. Suppose the corresponding invariants after the equations are multiplied by k are I1, I2, and I3, then it is not difficult to know that there is the following relationship:

[0086] I1=kI1′,I2=k 2 I2′,I3=k 3 I3′

[0087] Substituting the above formula into formula (3), we can get:

[0088]

[0089] so:

[0090] Eliminate k and simplify to get:

[0091] The cone surface in the world coordinate system and the cone surface on the pinhole imaging side are the same opposite cone surface, so the equations differ by a constant coefficient k. Therefore, the equations of the two differ by a constant coefficient k. Then, the cone surface invariant formed by the ellipse corresponding to the pinhole imaging of the camera and the circle has the formula

[0092] In the above-mentioned camera intrinsic parameter self-calibration method based on quadratic cone surface invariants, the method includes: when the radius of the circle is known, take three photos of three circles with the same radius on the same plane at different angles; perform elliptical curve fitting based on the three photos to obtain the corresponding matrix A of more than 9 ellipse equations in the pixel coordinate system; calculate the cone surface invariants consisting of more than 9 groups of ellipses corresponding to the camera pinhole and the pinhole imaging of the circle based on the corresponding matrix A of the more than 9 ellipse equations in the pixel coordinate system and the cone surface equation consisting of the ellipse corresponding to the camera pinhole and the pinhole imaging of the circle; calculate the intrinsic parameters of the camera based on the relationship between the cone surface invariants consisting of the ellipse corresponding to the camera pinhole and the pinhole imaging of the circle and the cone surface invariants. This self-calibration method does not require the relative positions of the circles and the corresponding world coordinates, but only requires the radius of the circle and the ellipse equation corresponding to the pixel coordinates of each circle. It does not require precise matching of the circles, making the application scenario more flexible.

[0093] In one embodiment, the equation of the cone formed by the ellipse corresponding to the imaging of the camera pinhole and the circular pinhole in step 104 is:

[0094]

[0095] Among them, (xyz) is the coordinate of point P on the cone surface in the world coordinate system, α, β are the scale factors of the image (u, v) coordinate system, (u0, v0) are the coordinates of the main point of the image, and γ is the skew parameter. is the matrix A corresponding to the ellipse equation in the pixel coordinate system;

[0096] The cone invariant formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging is:

[0097]

[0098] Among them: a 11 、a 12 、a 13 a 12 、a 22 、a 23 、a 33 is the element in the matrix A corresponding to the ellipse equation. Specifically, the relationship between the cone invariant and the intrinsic parameter of the ellipse corresponding to the camera pinhole and the circular pinhole imaging is derived as follows:

[0099] Pinhole imaging, stretching and rotation, the imaging of a circle must be an ellipse, and the ellipse equation in the pixel coordinate system is:

[0100] because

[0101] So the ellipse equation of the pinhole imaging is:

[0102]

[0103] In short:

[0104] The point (u1 v1-f) of the pinhole image in the pinhole coordinate system is related to any point (xyz) on the pinhole line: Right now:

[0105] Then, combined with formula (8), the equation of the cone consisting of the pinhole and the ellipse imaged by the pinhole is:

[0106]

[0107] Simplified

[0108]

[0109] Its invariant expressions are shown in equations (5), (6) and (7).

[0110] In one embodiment, step 106 includes: selecting the ellipse corresponding to the pinhole imaging of three circles on the same plane and the cone invariant corresponding to the cone formed by the camera pinhole, and obtaining 9 sets of cone invariants in the pixel coordinate system; substituting the 9 sets of cone invariants into the relationship between the cone invariants, and obtaining 9 equations with the camera intrinsic parameters and the distance from the pinhole to the plane where the circle is located as unknowns; solving the 9 equations using a nonlinear least squares solution method to obtain the camera's intrinsic parameters.

[0111] In one embodiment, when the radius of the circle is unknown, the acquired photos are three photos of four circles with the same radius on the same plane at different angles; before step 106, it also includes: constructing four equations according to the relationship between the cone surface invariants corresponding to the cone surface formed by the camera aperture and the four ellipses on the same photo and the cone surface invariants; the four equations are:

[0112]

[0113] Eliminate R0 and z0 from the four equations and we get equations that only include cone surface invariants.

[0114]

[0115] Among them, I ij is the i-th invariant corresponding to the j-th circle on the plane, i=1,2,3,4j=1,2,3,4.

[0116] In one embodiment, step 106 includes substituting the cone invariants corresponding to the cone formed by the camera aperture and four ellipses on the same photo into an equation that only includes the cone invariants, and obtaining two equations with the camera intrinsic parameters as unknowns, and a total of 6 equations are obtained for three photos; the 6 equations are solved using a nonlinear least squares solution method to obtain the intrinsic parameters of the camera.

[0117] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0118] In one embodiment, Figure 2 As shown, a camera intrinsic parameter self-calibration device based on quadratic cone surface invariants is provided, comprising: a calibration photo acquisition module, a corresponding matrix A determination module of an ellipse equation, a cone surface invariant expression determination module and a camera intrinsic parameter determination module, wherein:

[0119] The calibration photo acquisition module is used to obtain three photos of three circles with the same radius on the same plane at different angles when the radius of the circle is known.

[0120] The module for determining the corresponding matrix A of the ellipse equation is used to perform ellipse curve fitting based on three photos to obtain the corresponding matrices A of a predetermined number of ellipse equations in the pixel coordinate system; the predetermined number is three times the number of circles in the same plane.

[0121] The cone surface invariant expression determination module is used to calculate the cone surface invariant consisting of a predetermined number of groups or more of ellipses corresponding to the camera pinhole and the circular pinhole imaging according to the matrix A corresponding to the predetermined number of ellipse equations in the pixel coordinate system and the cone surface equation consisting of the ellipse corresponding to the camera pinhole and the circular pinhole imaging; the cone surface invariant is an expression about the camera intrinsic parameter.

[0122] The camera's intrinsic parameter determination module is used to calculate the camera's intrinsic parameters based on the relationship between the camera's pinhole and the cone invariant formed by the ellipse corresponding to the pinhole imaging of the circle and the cone invariant; the relationship between the cone invariants is:

[0123]

[0124] Among them, I1, I2, and I3 are three cone invariants; z0 is the distance from the small hole to the plane where the circle is located; R0 is the radius of the circle on the plane.

[0125] In one embodiment, the cone surface equation formed by the ellipse corresponding to the camera pinhole and the pinhole imaging of the circle in the cone surface invariant expression determination module is:

[0126]

[0127] Among them, (xyz) is the coordinate of point P on the cone surface in the world coordinate system, α and β are the scale factors of the image (u,v) coordinate system, (u0,v0) is the coordinate of the main point of the image, and γ is the skew parameter. is the matrix A corresponding to the ellipse equation in the pixel coordinate system.

[0128] The cone invariant formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging is:

[0129] I1=a 11 α 2 +a 33 +u0(a 13 +a 11 u0+a 12 v0)+v0(a 23 +a 12 u0+a 22 v0)+a 13 u0

[0130] +a 23 v0+β(a 22 β+a 12 γ)+γ(a12 β+a 11 γ)

[0131]

[0132] Among them: a 11 、a 12 、a 13 a 12 、a 22 a 23 a 33 is the A element of the matrix corresponding to the elliptic equation.

[0133] In one of the embodiments, the camera's intrinsic parameter determination module is also used to select the ellipse corresponding to the pinhole imaging of three identical circles on the same plane and the cone invariant corresponding to the cone formed by the camera pinhole, and obtain 9 groups of cone invariants in the pixel coordinate system; substitute the 9 groups of cone invariants into the relationship between the cone invariants, and obtain 9 equations with the camera intrinsic parameters and the distance from the pinhole to the plane where the circle is located as unknowns; use the nonlinear least squares solution method to solve the 9 equations to obtain the camera's intrinsic parameters.

[0134] 9. The device according to claim 6, when the radius of the circle is unknown, the acquired photos are 3 photos of 4 circles with the same radius on the same plane at different angles; the camera internal parameter determination module also includes: a module for determining the relationship between cone surface invariants in the same photo, which is used to construct 4 equations based on the relationship between the cone surface invariants corresponding to the cone surface formed by the camera pinhole and the four ellipses on the same photo and the cone surface invariants; the 4 equations are:

[0135]

[0136] Eliminate R0 and z0 from the four equations and we get equations that only include cone surface invariants.

[0137]

[0138] Among them, I ij is the i-th invariant corresponding to the j-th circle on the plane, i=1,2,3,4j=1,2,3,4.

[0139] In one of the embodiments, the camera's intrinsic parameter determination module is also used to substitute the cone invariants corresponding to the cone formed by the camera aperture and four ellipses on the same photo into an equation that only includes the cone invariants, thereby obtaining 6 equations with the camera's intrinsic parameters as unknowns; and the 6 equations are solved using a nonlinear least squares solution method to obtain the camera's intrinsic parameters.

[0140] For the specific definition of the camera intrinsic parameter self-calibration device based on the quadratic cone surface invariant, please refer to the definition of the camera intrinsic parameter self-calibration method based on the quadratic cone surface invariant above, which will not be repeated here. Each module in the above-mentioned camera intrinsic parameter self-calibration device based on the quadratic cone surface invariant can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0141] In a new embodiment of simulation verification, formula (1) can be used to derive that if the distance from the pinhole to a plane is z0, and there is a circle with a known radius on the plane, then a relational expression containing z0 and 5 intrinsic parameters is established. If there are three such non-parallel planes in space, and each plane has at least 2 circles with known radius (because a plane contains one z0, at least 2 circles are required to eliminate z0), a total of 9 circles, then 9 relational expressions can be obtained. Since z0 is the distance from the pinhole to the plane in question, the z0 corresponding to each plane is different, so there are three z0s and 5 intrinsic parameters. The intrinsic parameters of the camera can be obtained from the 9 relational expressions.

[0142] Assume the internal parameter matrix is Assume that the coordinates of the pinhole in the world coordinate system are (0, 0, 200). Initially, the camera is facing downward. The coordinates of the coordinate vector of the initial pinhole coordinate system of the camera in the world coordinate system are column vectors forming the R matrix. Here, we assume Suppose there are nine circles with a radius R0=20 on the horizontal coordinate plane whose distance to the small hole in the world coordinate system is z0=200, and the spatial coordinates of the center of the circle in the initial small hole coordinate system are (xc, yc, 200). As the preferred options, xc=[1,50,100,3,80,200,5,79,168] ​​and yc=[2,100,200,13,130,230,24,140,190].

[0143] In the pinhole coordinate system, with the pinhole as the vertex, the equation of the cone formed by the circles is:

[0144]

[0145] The first three circles are directly imaged to calculate the ellipse equation in the pixel coordinate system

[0146]

[0147] The following six circles are divided into two groups to rotate the horizontal plane of the world coordinate system around the small hole. The corresponding orthogonal matrices are T1 and T2. The equation of the ellipse after rotation and imaging is:

[0148] and

[0149] In this embodiment, The above formulas (12, 13, 14) correspond to nine elliptic equations, so the corresponding matrices in formula (4) are obtained: From equations (5), (6) and (7), we can get the invariants of the corresponding matrices of the ellipse equations. Then, we use equation (1) to establish 9 equations for these nine ellipses. The unknown parameters here are 5 internal parameters and one z0 (because the 9 circles originally come from the same plane, and then rotate around the small hole to separate them, so the distance from the small hole to the plane is the same after rotation). Under normal circumstances, three planes will have three different z0. Since the number of equations is greater than or equal to the number of unknowns, the internal parameters can be solved. The nonlinear least squares solution method is used here to verify the solvability of the equation system and the correctness of the results.

[0150] When R0 is unknown, three more circles need to be added, with four circles as a group, and 12 relationship equations can be obtained. The specific implementation method is consistent with the calculation process of the above three circles.

[0151] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A camera intrinsic parameter self-calibration method based on quadratic cone invariant, characterized in that: The method comprises: When the radius of the circle is known, obtain three photos of three circles with the same radius on the same plane at different angles; Perform elliptic curve fitting based on the three photos to obtain a corresponding matrix A of a predetermined number of ellipse equations in a pixel coordinate system; the predetermined number is three times the number of circles in the same plane; According to the matrix A corresponding to the predetermined number of ellipse equations in the pixel coordinate system and the intrinsic parameters, the cone surface equation formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging can be obtained, and the cone surface invariant formed by the predetermined number of groups of ellipses corresponding to the camera pinhole and the circular pinhole imaging can be calculated; the cone surface invariant is an expression about the camera intrinsic parameters; The intrinsic parameters of the camera are calculated based on the relationship between the cone surface invariants formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging and a predetermined number of groups of the cone surface invariants; the relationship between the cone surface invariants is: Among them, I1, I2, and I3 are three cone invariants; z0 is the distance from the small hole to the plane where the circle is located; R0 is the radius of the circle on the plane.

2. The method according to claim 1, characterized in that: According to the matrix A and the internal parameters corresponding to the 9 ellipse equations in the pixel coordinate system, the cone surface equation formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging can be obtained, so as to calculate the cone surface invariant formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging. The cone surface equation formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging in the step is: Among them, (xyz) is the coordinate of point P on the cone surface in the world coordinate system, α, β are the scale factors of the image (u, v) coordinate system, (u0, v0) are the coordinates of the main point of the image, and γ is the skew parameter. is the corresponding matrix A of the ellipse equation in the pixel coordinate system; The cone invariant formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging is: I1=a 11 α 2 +a 33 +u0(a 13 +a 11 u0+a 12 v0)+v0(a 23 +a 12 u0+a 22 v0)+a 13 u0 +a 23 v0+β(a 22 b+a 12 c)+c(a 12 b+a 11 c) Among them: a 11 、a 12 、a 13 a 12 、a 22 、a 23 、a 33 is the element in the matrix A corresponding to the elliptic equation.

3. The method according to claim 2, characterized in that The intrinsic parameters of the camera are calculated based on the relationship between the cone surface invariants formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging, and the cone surface invariants, including: Select the ellipse corresponding to the pinhole imaging of three circles on the same plane and the cone surface corresponding to the cone surface formed by the camera pinhole to obtain 9 groups of the cone surface invariants in the pixel coordinate system; Substituting the 9 groups of cone surface invariants into the relationship between the cone surface invariants, 9 equations with the camera intrinsic parameters and the distance from the pinhole to the plane where the circle is located as unknowns are obtained; The nine equations are solved using a nonlinear least squares solution method to obtain the intrinsic parameters of the camera.

4. The method according to claim 1, characterized in that: When the radius of the circle is unknown, the photos obtained are 3 photos of 4 circles with the same radius on the same plane at different angles; The intrinsic parameters of the camera are calculated based on the relationship between the camera pinhole and the cone surface invariant formed by the ellipse corresponding to the pinhole imaging of the circle and the cone surface invariant, and the step also includes: Four equations are constructed based on the relationship between the cone invariants and the cone invariants corresponding to the cone formed by the camera pinhole and the four ellipses on the same photo; the four equations are: Eliminate R0 and z0 from the four equations to obtain equations that only include cone invariants; Among them, I ij is the i-th invariant corresponding to the j-th circle on the plane, i=1,2,3,4j=1,2,3,4.

5. The method according to claim 4, characterized in that The intrinsic parameters of the camera are calculated based on the relationship between the cone surface invariants formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging, and the cone surface invariants, including: Substitute the cone invariants corresponding to the cone formed by the camera pinhole and the four ellipses on the same photo into the equation that only includes the cone invariants, and obtain two equations with the camera intrinsic parameters as unknowns, and a total of 6 equations for the three photos; The six equations are solved using a nonlinear least squares solution method to obtain the intrinsic parameters of the camera.

6. A camera intrinsic parameter self-calibration device based on quadratic cone invariant, characterized in that: The device comprises: The calibration photo acquisition module is used to obtain three photos of three circles with the same radius on the same plane at different angles when the radius of the circle is known; A module for determining the corresponding matrix A of the ellipse equation, for performing ellipse curve fitting according to the three photos to obtain the corresponding matrices A of a predetermined number of ellipse equations in the pixel coordinate system; the predetermined number is three times the number of circles in the same plane; A cone surface invariant expression determination module is used to calculate the cone surface invariant composed of a predetermined number of groups of ellipses corresponding to the camera pinhole and the circular pinhole imaging according to the matrix A corresponding to the predetermined number of ellipse equations in the pixel coordinate system and the cone surface equation composed of the ellipse corresponding to the camera pinhole and the circular pinhole imaging; the cone surface invariant is an expression about the camera intrinsic parameter; The camera's intrinsic parameter determination module is used to calculate the camera's intrinsic parameters based on the relationship between the cone surface invariants formed by the ellipse corresponding to the camera's pinhole and the circular pinhole imaging and a predetermined number of groups of the cone surface invariants; the relationship between the cone surface invariants is: Among them, I1, I2, and I3 are three cone invariants; z0 is the distance from the small hole to the plane where the circle is located; R0 is the radius of the circle on the plane.

7. The device according to claim 6, characterized in that The cone surface equation formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging in the cone surface invariant expression determination module is: Among them, (xyz) is the coordinate of point P on the cone surface in the world coordinate system, α and β are the scale factors of the image (u,v) coordinate system, (u0,v0) is the coordinate of the main point of the image, and γ is the skew parameter. is the matrix A corresponding to the ellipse equation in the pixel coordinate system; The cone invariant formed by the ellipse corresponding to the camera pinhole and the circular pinhole imaging is: I1=a 11 α 2 +a 33 +u0(a 13 +a 11 u0+a 12 v0)+v0(a 23 +a 12 u0+a 22 v0)+a 13 u0 +a 23 v0+β(a 22 b+a 12 c)+c(a 12 b+a 11 c) Among them: a 11 、a 12 、a 13 a 12 、a 22 、a 23 、a 33 is the A element of the matrix corresponding to the elliptic equation.

8. The device according to claim 7, characterized in that The camera's intrinsic parameter determination module is also used to select the ellipse corresponding to the pinhole imaging of three circles on the same plane and the cone surface invariant corresponding to the cone surface formed by the camera pinhole, and obtain 9 groups of the cone surface invariants in the pixel coordinate system; substitute the 9 groups of the cone surface invariants into the relationship between the cone surface invariants to obtain 9 equations with the camera intrinsic parameters and the distance from the pinhole to the plane where the circle is located as unknowns; use the nonlinear least square solution method to solve the 9 equations to obtain the camera's intrinsic parameters.

9. The device according to claim 6, characterized in that When the radius of the circle is unknown, the acquired photos are three photos of four circles with the same radius on the same plane at different angles; the camera internal parameter determination module also includes: a module for determining the relationship between cone surface invariants in the same photo, which is used to construct four equations based on the relationship between the cone surface invariants corresponding to the cone surface formed by the camera pinhole and the four ellipses on the same photo and the cone surface invariants; the four equations are: Eliminate R0 and z0 from the four equations to obtain equations that only include cone invariants; Among them, I ij is the i-th invariant corresponding to the j-th circle on the plane, i=1,2,3,4j=1,2,3,4.

10. The device according to claim 9, characterized in that The camera's intrinsic parameter determination module is also used to substitute the cone invariants corresponding to the cone formed by the camera's pinhole and the four ellipses on the same photo into the equation that only includes the cone invariants, and obtain 6 equations with the camera's intrinsic parameters as unknowns; the 6 equations are solved using a nonlinear least squares solution method to obtain the camera's intrinsic parameters.

Images