Camera calibration method and system, device, storage medium, and program product

Abstract

The present application relates to the technical field of camera calibration, and discloses a camera calibration method and system, a device, a storage medium, and a program product. The method comprises: on the basis of correspondences between spatial points and image points of a single image, constructing a linear equation about a rotation matrix, a translation vector, and a focal length element on the basis of a pinhole camera model; converting the linear equation into a homogeneous system of equations about a rotation matrix element and a focal length; using a null-space analysis method and a rotation matrix orthogonality constraint to calculate the homogeneous system of equations so as to obtain the rotation matrix element and the focal length, and selecting initial values for subsequent iterative optimization; and constructing a cost equation on the basis of the rotation matrix orthogonality constraint, and on the basis of the initial values, performing iterative optimization to obtain a global optimal solution for camera calibration. In the present application, a constraint that is not based on intermediate variables is constructed on the basis of the pinhole camera model, and a null-space analysis framework based on a rotation matrix constraint is used to solve for related parameters, thereby shortening operation time and improving the robustness and stability of point distribution.

Description

A camera calibration method, system, device, storage medium, and software product.

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411220353.2, filed on September 2, 2024, entitled "A camera calibration method, system, device, storage medium and program product", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of camera calibration processing technology, specifically to a camera calibration method, system, device, storage medium, and program product. Background Technology

[0004] Due to its advantages such as high precision, non-contact operation, and dynamic measurement, image-based visual measurement technology is widely used in the measurement of deformation of large structures. Measurement system calibration is one of the most crucial steps in measurement system development, as its efficiency and accuracy directly affect the overall measurement efficiency and accuracy of the visual measurement system. Therefore, accurate and rapid calibration of measurement system parameters is extremely important.

[0005] However, in the calibration of large field-of-view cameras in the field, it is often difficult to meet the scene constraints and motion constraints required for self-calibration, the strong stereo conditions required for stereo calibration, or the absolute coplanar conditions required for planar target calibration due to the overly simple scene. Outdoor calibration conditions are often unusual, such as a small number of spatial points, or unusual spatial distribution at long distances or near the plane. Under these conditions, existing methods often fail and have poor robustness.

[0006] Summary of the Invention

[0007] In view of this, this application provides a camera calibration method, system, device, storage medium, and program product to improve the robustness and accuracy of camera calibration.

[0008] In a first aspect, this application provides a camera calibration method, the method comprising:

[0009] Based on the correspondence between spatial points and image points in a single image, linear equations about rotation matrix, translation vector, and focal length elements are constructed according to the pinhole camera model.

[0010] The linear equations are transformed into a system of homogeneous equations concerning the elements of the rotation matrix and the focal length.

[0011] By using null space analysis and combining rotation matrix orthogonality constraints, the homogeneous equations are solved to obtain the rotation matrix elements and focal length.

[0012] The translation vector is calculated based on the elements of the rotation matrix and the focal length, and the initial values ​​for subsequent iterative optimization are selected based on the elements of the rotation matrix, the focal length, and the translation vector.

[0013] A cost equation is constructed based on the orthogonality constraint of the rotation matrix, and the global optimal solution is obtained by iterative optimization based on the selected rotation matrix elements and initial focal length. The final rotation matrix elements, focal length, and translation vector are then used for camera calibration.

[0014] The camera calibration method provided in this embodiment directly constructs constraints without intermediate variables from the pinhole camera model, and uses a null space analysis framework based on rotation matrix constraints to solve the relevant parameters, which greatly reduces the solution time and improves the robustness and stability for point distribution.

[0015] In one alternative implementation, transforming the linear equation into a homogeneous system of equations with respect to the rotation matrix elements and the focal length includes:

[0016] Based on the linear least squares criterion, the translation vector is represented using the elements of the rotation matrix and the focal length.

[0017] Based on whether spatial points are coplanar, construct a homogeneous system of equations about the translation vector, and construct the corresponding rotation matrix constraints.

[0018] In one optional implementation, the step of using null space analysis combined with rotation matrix orthogonality constraints to solve the homogeneous equations to obtain the rotation matrix elements and focal length includes:

[0019] In the non-coplanar configuration, the null space analysis method is used in conjunction with the corresponding rotation matrix orthogonality constraint to solve the null space coefficients of the homogeneous equation system, thereby obtaining the rotation matrix elements and focal length.

[0020] Under coplanar configuration, the null space analysis method is used in conjunction with the corresponding rotation matrix orthogonality constraint to solve the null space coefficients of the homogeneous equation system, thereby obtaining the focal length and the first two columns of the rotation matrix. Then, SVD decomposition is used to obtain the rotation matrix that is closest to the solved rotation matrix and strictly satisfies the rotation matrix constraint.

[0021] Based on whether spatial points are coplanar, this application uses null space analysis combined with rotation matrix orthogonality constraints to obtain the elements of the rotation matrix and the focal length. It adopts a null space analysis framework and designs robust and efficient solutions for multiple polynomial problems in the framework, which has high computational efficiency.

[0022] In one optional implementation, the step of solving the translation vector based on the solved rotation matrix elements and focal length, and then selecting initial values ​​for subsequent iterative optimization based on the solved rotation matrix elements, focal length, and translation vector, includes:

[0023] Based on geometric rationality, the elements of the rotation matrix and focal length are filtered, and the translation vector is obtained based on the filtered rotation matrix elements and focal length.

[0024] Using the rotation matrix elements, focal length, and translation vector obtained through screening, the solution with the smallest reprojection error is selected as the initial value for subsequent iterative optimization.

[0025] In one alternative implementation, the geometrically based rationale-based screening of the solved rotation matrix elements and focal length includes: filtering out values ​​with negative depth factors and values ​​with negative rotation matrix eigenvalues ​​in the pinhole camera model.

[0026] The embodiments of this application select initial values ​​based on geometric rationality to provide a more accurate data foundation for subsequent iterative optimization, which is beneficial to improving computational efficiency.

[0027] In one optional implementation, the step of constructing a cost equation based on the orthogonality constraint of the rotation matrix, and iteratively optimizing it based on the selected rotation matrix elements and initial focal length to obtain a globally optimal solution, and then using the final rotation matrix elements, focal length, and translation vector for camera calibration, includes:

[0028] The rotation matrix is ​​represented as a unit quaternion, and a cost equation is constructed based on the orthogonality constraint of the rotation matrix.

[0029] Based on the selected rotation matrix elements and initial focal length, the damped Newton method is used to iteratively optimize the constructed cost equation to find the global optimal solution, thereby obtaining the rotation matrix elements and focal length.

[0030] The translation vector is obtained based on the elements of the rotation matrix and the focal length;

[0031] Camera calibration is performed using the final rotation matrix elements, focal length, and translation vector obtained through optimization.

[0032] The embodiments of this application represent the rotation matrix as a unit quaternion and combine iterative optimization with the damped Newton method to achieve a globally optimal solution, which has the best accuracy, noise robustness and high computational efficiency.

[0033] Secondly, this application provides a camera calibration system, the system comprising:

[0034] The linear equation construction module is used to construct linear equations about rotation matrix, translation vector and focal length elements based on the correspondence between spatial points and image points in a single image and the pinhole camera model.

[0035] A homogeneous equation system construction module is used to transform the linear equations into a homogeneous equation system with respect to the rotation matrix elements and the focal length.

[0036] The homogeneous equations solving module is used to solve the homogeneous equations using the null space analysis method combined with the orthogonality constraint of the rotation matrix to obtain the rotation matrix elements and focal length;

[0037] The initial value filtering module is used to solve the translation vector based on the elements of the solved rotation matrix and the focal length, and to filter the initial values ​​for subsequent iterative optimization based on the elements of the solved rotation matrix, the focal length and the translation vector.

[0038] The iterative optimization module is used to construct the cost equation based on the orthogonality constraint of the rotation matrix, and to perform iterative optimization based on the selected rotation matrix elements and initial focal length to obtain the global optimal solution. Finally, the rotation matrix elements, focal length, and translation vector are obtained for camera calibration.

[0039] Thirdly, this application provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the camera calibration method of the first aspect or any corresponding embodiment described above.

[0040] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the camera calibration method of the first aspect or any corresponding embodiment described above.

[0041] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the camera calibration method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 is a schematic flowchart of a camera calibration method according to an embodiment of this application;

[0044] Figure 2 is a schematic diagram of the PnPf problem according to an embodiment of this application;

[0045] Figure 3 is a flowchart illustrating the key steps of the camera calibration method according to an embodiment of this application;

[0046] Figure 4 is a structural block diagram of a camera calibration system according to an embodiment of this application;

[0047] Figure 5 is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] According to an embodiment of this application, a camera calibration method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer device such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0050] This embodiment provides a camera calibration method that can be used with the aforementioned computer equipment. The camera parameters that can be calibrated include camera extrinsic parameters (rotation matrix and translation vector from the world coordinate system or reference coordinate system to the camera coordinate system) and camera intrinsic parameters (camera focal length). The camera calibration method is to analytically solve for camera parameters based on the correspondence between three-dimensional spatial points (coordinates can be obtained by marking points with a total station or by other high-precision three-dimensional measuring equipment) and two-dimensional image points, requiring at least one image.

[0051] Figure 1 is a flowchart of a camera calibration method according to an embodiment of this application. As shown in Figure 1, the process includes the following steps:

[0052] Step S101: Based on the correspondence between spatial points and image points in a single image, construct linear equations about rotation matrix, translation vector, and focal length elements according to the pinhole camera model.

[0053] This application embodiment should include at least four control points with known three-dimensional coordinates. The spatial position of the control points is not required, and the coordinates of the control points are obtained through high-precision three-dimensional measurement equipment such as a total station. Starting from the pinhole camera model, constraints are directly constructed without relying on intermediate variables. Constructing constraints from the pinhole camera model helps reduce the sensitivity to the distribution of point features, thereby improving the robustness of the method to point distribution. The pinhole camera model is a simplified optical imaging model. Its basic principle is central perspective projection, which projects an object in three-dimensional space onto a two-dimensional imaging plane through a pinhole. Through this model, the pixel position in the image can be predicted based on known camera parameters and the object's position in the world coordinate system, or conversely, the object's position in the world coordinate system can be estimated based on the pixel position in the image and the camera parameters. Specifically, based on the pinhole camera model, we can obtain:

[0054] Where R and t represent the rotation matrix and translation vector from the world coordinate system to the camera coordinate system, respectively, and λ i Let u be the depth factor, representing the distance u of the i-th point along the camera's optical axis (the z-axis of the camera coordinate system). c and v c The coordinates of the master point are used. The correspondence between spatial points and image points in a single image is a correspondence from the 3D world coordinate system to the 2D camera coordinate system, based on the PnPf problem. Specifically, it depends on the N pairs of 3D-2D point correspondences (P... i ,p i The corresponding estimated rotation matrix R, translation vector t, and focal length f are shown in Figure 2.

[0055] Step S102, transform the linear equations into a homogeneous system of equations concerning the elements of the rotation matrix and the focal length.

[0056] As shown in Figure 3, this embodiment of the application uses the elements of the rotation matrix and the focal length to represent the translation vector based on the linear least squares criterion. A homogeneous system of equations about the translation vector is constructed based on whether the spatial points are coplanar, and corresponding rotation matrix constraints are also constructed. It should be noted that the rotation matrix orthogonality constraint means that any row and any column are unit vectors, and any two rows and any two columns are orthogonal. Based on whether the spatial points are coplanar, the configuration can be divided into non-coplanar and coplanar configurations, specifically:

[0057] In the non-coplanar configuration: λ can be obtained using the third row of equation (1). i Substituting the expression into the first two lines of the equation eliminates λ. i get:

[0058] Dividing both sides of equation (2) by f simultaneously yields:

[0059] Then, two intermediate vectors are defined:

[0060] in Based on this equation (3), it can be rewritten as:

[0061] Equation (5) is the constraint corresponding to the i-th pair of 3D-2D points. Since there are N pairs of point correspondences, according to the linear least squares criterion, the constraints corresponding to all points (N points) can be combined to obtain the following system of equations:

[0062] in

[0063] Using a false rebellion Represented as:

[0064] Substituting equation (7) back into equation (6), we obtain a system of homogeneous equations that depend only on r:

[0065] Under non-coplanar configurations, the sublinear equations described by the constraint equation (8) of the constructed rotation matrix are considered for four cases: m = 1, 2, 3, 4 in the null space, so that the method covers the case N >= 4 (theoretically, when there are 4 pairs of points corresponding). The larger the number of points, the smaller m becomes (m is at least 1), thus covering the minimum number of points configuration and the over-determined configuration.

[0066] In a coplanar configuration, since the constructed rotation matrix constraint (8) is for a non-coplanar configuration, it degenerates in the case of 3D points being coplanar (planar point configuration). A common preprocessing method in a planar point configuration is to perform a pre-(rigid) transformation on the world coordinate system so that the Z-axis of the new world coordinate system is perpendicular to the plane where the 3D point is located. Then, the solution is obtained based on the new world coordinate system, and finally the rigid transformation is compensated back into the solution result to obtain the true rotation matrix and translation vector. In the new world coordinate system, the coordinates of the 3D point are P. i =[x i ,y i ,0] T Substituting this into equation (1), the third column element of the rotation matrix R is eliminated. Therefore, under the planar point configuration, the vector The vector remains unchanged. Transform into a 6-dimensional vector:

[0067] Similarly, the plane point configuration is as follows: W in equation (5) i The dimension remains unchanged, while V i Transform into a 2×6 matrix:

[0068] In equation (6), the dimension of W remains unchanged, while V becomes a matrix. The 2N×6 planar point configuration can also be obtained using the pseudo-inverse, which is only related to... The relevant homogeneous system of equations has the same form as equation (8), but the following equations are different: V must be in the form provided in equations (9) and (10).

[0069] As shown in equation (9), the plane point configuration is as follows: With 6 variables, we consider the cases where m = 1 and 2 respectively. This allows the planar point placement method to cover the case where the number of points N >= 4 (theoretically, when 4 points correspond). (Consider that m=2 can improve the method's noise robustness). The above-described planar point placement method is both an overdetermined placement method and can solve for the minimum number of points.

[0070] Step S103: Using the null space analysis method and combined with the orthogonality constraint of the rotation matrix, solve the homogeneous equation system to obtain the rotation matrix elements and focal length.

[0071] This application utilizes null space analysis combined with rotation matrix orthogonality constraints to solve the null space coefficients of a homogeneous system of equations, obtaining initial values ​​for the rotation matrix elements and focal length. Null space analysis is a method for solving homogeneous linear equations, representing the solution vectors as a linear combination of null space vectors. This requires singular value decomposition of the coefficient matrix, selecting the right singular vectors corresponding to the zero singular values ​​as null space vectors. In real-world data, noise-induced singular value perturbations may cause the singular values ​​of correct right singular vectors to be greater than those of incorrect right singular vectors, leading to the selection of incorrect right singular vectors as null space vectors.

[0072] This application removes the restriction on the null space dimension and classifies and discusses its various values ​​(e.g., considering four cases), thereby covering more null space cases, reducing the possibility of selecting an incorrect null space vector, and improving the method's noise robustness. Unlike existing methods that solve based on intermediate variables (control point coordinates) and partial constraints (distance between control points), this application directly uses the elements of the rotation matrix and fully utilizes the constraints of the rotation matrix itself for solving, resulting in higher computational efficiency. Furthermore, it proposes several efficient and robust polynomial solutions to address the polynomial problems that arise, ensuring both accuracy and robustness. Specifically:

[0073] In a non-coplanar configuration, null space analysis can directly yield the focal length and rotation matrix; however, in a coplanar configuration, null space analysis only outputs the solutions for the first two columns c1 and c2 of the focal length and rotation matrix, requiring cross product to recover the third column c3 = c1 × c2. Due to noise, the output rotation matrix does not strictly satisfy the rotation matrix constraint. According to Frobenius theory, SVD decomposition can be used to obtain the closest strictly satisfied rotation matrix. Let the output value be... Perform SVD decomposition on it Then the closest Furthermore, the rotation matrix that strictly satisfies the rotation matrix constraint is R = UV. T .

[0074] Step S104: Solve the translation vector based on the solved rotation matrix elements and focal length, and select initial values ​​for subsequent iterative optimization based on the solved rotation matrix elements, focal length, and translation vector.

[0075] In this embodiment, the elements of the rotation matrix and the focal length are filtered based on geometric rationality, and a translation vector is obtained based on the filtered rotation matrix elements and focal length. Using the filtered rotation matrix elements, focal length and translation vector, the solution with the smallest reprojection error is selected as the initial value for subsequent iterative optimization.

[0076] Specifically, invalid solutions with det(R) = -1 (corresponding to left-handed systems) in R obtained in the previous step need to be removed, as well as solutions where the 3D point is behind the camera (with a negative depth factor) need to be removed. After removing geometrically unreasonable solutions, the rotation matrix elements and focal length are substituted into equation (7) to obtain the translation vector.

[0077] In camera calibration, reprojection error is typically used to evaluate the accuracy of the calibration results. The smaller the reprojection error, the more accurate the camera calibration results and the more precise the camera model's description of the actual imaging process. Therefore, this embodiment utilizes the rotation matrix, focal length, and translation vector after removing geometric inconsistencies, and selects the solution with the smallest reprojection error as the initial value for subsequent iterative optimization.

[0078] Step S105: Construct a cost equation based on the orthogonality constraint of the rotation matrix, and perform iterative optimization based on the selected rotation matrix elements and initial focal length to obtain the global optimal solution. Finally, use the rotation matrix elements, focal length, and translation vector to perform camera calibration.

[0079] To reduce computational complexity and improve computational efficiency, this application adopts the unit quaternion form of the rotation matrix:

[0080] Based on equation (8), the cost equation is constructed. In this embodiment, the least squares error function with respect to the parameters to be solved is constructed as follows:

[0081] This application uses the damped Newton method for iterative optimization. Compared to other first-order methods, it converges to the optimal solution faster, saving computation time and resources, and exhibits good convergence performance in local regions close to the optimal solution. It can efficiently converge to the optimal solution when the initial value is close to it. Let the value of the k-th iteration be m. k =[a k ,b k ,c k ,d k ,f k If ], then in the (k+1)th iteration, it is updated to: m k+1 =m k -[H(m k )+λI5] -1 g(m k ),(13)

[0082] Where λ is the step size, I5 is the 5-dimensional identity matrix, and H(m) k ) and g(m k ) are the cost equations with respect to m k The Hessian matrix and gradient vector.

[0083] Because the initial values ​​provided in this application embodiment have high accuracy, optimal accuracy can be achieved in just a few iterations, which is one of the keys to its high computational efficiency. The method provided in this application embodiment has optimal accuracy, noise robustness, and computational efficiency, and is robust to three singular point feature distribution scenarios: planar point configuration, near-planar configuration, and far-distance configuration. It can greatly reduce camera calibration time and has high accuracy.

[0084] This embodiment also provides a camera calibration system for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0085] This embodiment provides a camera calibration system, as shown in Figure 4, including:

[0086] The linear equation construction module 401 is used to construct linear equations about rotation matrix, translation vector and focal length elements based on the correspondence between spatial points and image points in a single image and the pinhole camera model.

[0087] Homogeneous equation system construction module 402 is used to convert the linear equations into a homogeneous equation system with respect to the rotation matrix elements and focal length;

[0088] The homogeneous equations solving module 403 is used to solve the homogeneous equations using the null space analysis method and combined with the orthogonality constraint of the rotation matrix to obtain the rotation matrix elements and focal length.

[0089] The initial value filtering module 404 is used to solve the translation vector based on the solved rotation matrix elements and focal length, and to filter the initial values ​​for subsequent iterative optimization based on the solved rotation matrix elements, focal length and translation vector.

[0090] The iterative optimization module 405 is used to construct the cost equation based on the orthogonality constraint of the rotation matrix, and to perform iterative optimization based on the selected rotation matrix elements and initial focal length to obtain the global optimal solution, and to obtain the final rotation matrix elements, focal length and translation vector for camera calibration.

[0091] In some alternative implementations, the linear equation construction module 401 includes:

[0092] The parameter transformation unit is used to represent the translation vector using the elements of the rotation matrix and the focal length according to the linear least squares criterion.

[0093] The homogeneous equation system and rotation matrix constraint construction unit is used to construct a homogeneous equation system about the translation vector based on whether spatial points are coplanar and corresponding, and to construct the corresponding rotation matrix constraints.

[0094] In some alternative implementations, the homogeneous equations solving module 403 includes:

[0095] The first solving unit is used for non-coplanar configuration. It uses null space analysis combined with the corresponding rotation matrix orthogonality constraint to solve the null space coefficients of the homogeneous equation system and obtain the rotation matrix elements and focal length.

[0096] In the second solution unit, under coplanar configuration, the null space analysis method is used in conjunction with the corresponding rotation matrix orthogonality constraint to solve the null space coefficients of the homogeneous equation system, obtain the focal length and the first two columns of the rotation matrix, and use SVD decomposition to obtain the rotation matrix that is closest to the solved rotation matrix and strictly satisfies the rotation matrix constraint.

[0097] In some optional implementations, the iterative optimization module 405 includes:

[0098] The cost equation construction unit is used to represent the rotation matrix as a unit quaternion and construct the cost equation based on the orthogonality constraint of the rotation matrix.

[0099] The iterative unit is used to iteratively optimize the constructed cost equation based on the selected rotation matrix elements and initial focal length using the damped Newton method to find the global optimal solution, thereby obtaining the rotation matrix elements and focal length.

[0100] The translation vector calculation unit is used to obtain the translation vector based on the elements of the rotation matrix and the focal length.

[0101] The calibration unit is used to perform camera calibration using the final rotation matrix elements, focal length, and translation vector obtained through optimization.

[0102] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0103] In this embodiment, the camera calibration system is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0104] This application also provides a computer device having a camera calibration system as shown in FIG4 above.

[0105] Please refer to Figure 5, which is a schematic diagram of the structure of a computer device provided in an optional embodiment of this application. As shown in Figure 5, the computer device includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other using different buses and can be installed on a common motherboard or otherwise as needed. The processor can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output system (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 uses one processor 10 as an example.

[0106] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0107] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0108] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0109] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0110] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0111] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; optionally, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0112] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0113] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A camera calibration method, characterized in that, The method includes: Based on the correspondence between spatial points and image points in a single image, linear equations about rotation matrix, translation vector, and focal length elements are constructed according to the pinhole camera model. The linear equations are transformed into a system of homogeneous equations concerning the elements of the rotation matrix and the focal length. By using null space analysis and combining rotation matrix orthogonality constraints, the homogeneous equations are solved to obtain the rotation matrix elements and focal length. The translation vector is calculated based on the elements of the rotation matrix and the focal length, and the initial values ​​for subsequent iterative optimization are selected based on the elements of the rotation matrix, the focal length, and the translation vector. A cost equation is constructed based on the orthogonality constraint of the rotation matrix, and the global optimal solution is obtained by iterative optimization based on the selected rotation matrix elements and initial focal length. The final rotation matrix elements, focal length, and translation vector are then used for camera calibration.

2. The method according to claim 1, characterized in that, The process of transforming the linear equation into a homogeneous system of equations concerning the elements of the rotation matrix and the focal length includes: Based on the linear least squares criterion, the translation vector is represented using the elements of the rotation matrix and the focal length. Based on whether spatial points are coplanar, construct a homogeneous system of equations about the translation vector, and construct the corresponding rotation matrix constraints.

3. The method according to claim 2, characterized in that, The method of using null space analysis combined with rotation matrix orthogonality constraints to solve the homogeneous equations to obtain the rotation matrix elements and focal length includes: In the non-coplanar configuration, the null space analysis method is used in conjunction with the corresponding rotation matrix orthogonality constraint to solve the null space coefficients of the homogeneous equation system, thereby obtaining the rotation matrix elements and focal length. Under coplanar configuration, the null space analysis method is used in conjunction with the corresponding rotation matrix orthogonality constraint to solve the null space coefficients of the homogeneous equation system, thereby obtaining the focal length and the first two columns of the rotation matrix. Then, SVD decomposition is used to obtain the rotation matrix that is closest to the solved rotation matrix and strictly satisfies the rotation matrix constraint.

4. The method according to claim 3, characterized in that, The process of solving the translation vector based on the elements of the solved rotation matrix and the focal length, and then selecting initial values ​​for subsequent iterative optimization based on the solved rotation matrix elements, focal length, and translation vector, includes: Based on geometric rationality, the elements of the rotation matrix and focal length are filtered, and the translation vector is obtained based on the filtered rotation matrix elements and focal length. Using the rotation matrix elements, focal length, and translation vector obtained through screening, the solution with the smallest reprojection error is selected as the initial value for subsequent iterative optimization.

5. The method according to claim 4, characterized in that, The geometrically based rationale for filtering the elements of the solved rotation matrix and focal length includes: Filter out negative values ​​for the depth factor and negative values ​​for the rotation matrix eigenvalues ​​in the pinhole camera model.

6. The method according to claim 4, characterized in that, The cost equation is constructed based on the orthogonality constraint of the rotation matrix, and the global optimal solution is obtained through iterative optimization based on the selected rotation matrix elements and initial focal length. The final rotation matrix elements, focal length, and translation vector are then used for camera calibration. include: The rotation matrix is ​​represented as a unit quaternion, and a cost equation is constructed based on the orthogonality constraint of the rotation matrix. Based on the selected rotation matrix elements and initial focal length, the damped Newton method is used to iteratively optimize the constructed cost equation to find the global optimal solution, thereby obtaining the rotation matrix elements and focal length. The translation vector is obtained based on the elements of the rotation matrix and the focal length; Camera calibration is performed using the final rotation matrix elements, focal length, and translation vector obtained through optimization.

7. A camera calibration system, characterized in that, The system includes: The linear equation construction module is used to construct linear equations about rotation matrix, translation vector and focal length elements based on the correspondence between spatial points and image points in a single image and the pinhole camera model. A homogeneous equation system construction module is used to transform the linear equations into a homogeneous equation system with respect to the rotation matrix elements and the focal length. The homogeneous equations solving module is used to solve the homogeneous equations using the null space analysis method combined with the orthogonality constraint of the rotation matrix to obtain the rotation matrix elements and focal length; The initial value filtering module is used to solve the translation vector based on the elements of the solved rotation matrix and the focal length, and to filter the initial values ​​for subsequent iterative optimization based on the elements of the solved rotation matrix, the focal length and the translation vector. The iterative optimization module is used to construct the cost equation based on the orthogonality constraint of the rotation matrix, and to perform iterative optimization based on the selected rotation matrix elements and initial focal length to obtain the global optimal solution. Finally, the rotation matrix elements, focal length, and translation vector are obtained for camera calibration.

8. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the camera calibration method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the camera calibration method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the camera calibration method according to any one of claims 1 to 6.

Images