Method and system for estimating camera relative pose and camera intrinsic parameters

Abstract

This invention discloses a method and system for estimating camera relative pose and camera intrinsic parameters, belonging to the field of computer vision technology. The method includes: acquiring an external prior direction and using this prior to transform two views acquired by the camera, constraining the relative rotation between the two views to a one-dimensional rotation about a single axis pointing to the prior direction; constructing a semi-calibrated focal length model of the camera and establishing a nonlinear least squares optimization model based on epipolar geometry constraints, with rotation angle, inverse focal length, and translation vector as optimization variables; using the structural characteristics of the objective function to determine the target association between the translation vector and other variables, thereby reducing the dimensionality of the optimization model; finally, using a multi-initial-value strategy and an iterative optimization algorithm to solve the dimensionality-reduced model to obtain the globally optimal estimate. This invention can jointly estimate the camera relative pose and unknown camera focal length between two views with high accuracy and robustness, and is applicable to monocular vision systems.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a method and system for estimating camera relative pose and camera intrinsic parameters. Background Technology

[0002] In the field of computer vision, estimating the relative pose of the camera between two or more images (i.e., relative rotation and relative translation) is a fundamental and core problem, and a key link in higher-level applications such as Structure-from-Motion (SfM) and Simultaneous Localization and Mapping (SLAM).

[0003] When dealing with unknown intrinsic parameters, a common simplification is the semi-calibration case, which assumes that parameters such as the camera's principal point and pixel distortion are known or can be reasonably ignored, with only the focal length being the main unknown. To address this problem, existing techniques primarily rely on minimal solvers. These methods recover motion and focal length from a minimum number of corresponding feature points by solving a system of polynomial equations.

[0004] However, methods based on minimum solvers have inherent limitations. Because they use only a minimal number of data points, their solution process is highly sensitive to image noise and feature matching errors, resulting in poor numerical stability and accuracy of the estimation results. Furthermore, with the widespread use of inertial measurement units (IMUs) in various devices, the external orientation information they provide, such as the direction of gravity, has become a valuable geometric prior. While existing technologies have attempted to utilize external prior orientations to simplify the problem (e.g., reducing 3D rotation estimation to one dimension), they typically remain focused on designing new minimum solvers, failing to fundamentally address their sensitivity to noise and insufficient accuracy. Therefore, a new technical solution is urgently needed that can effectively utilize external prior orientations under semi-calibrated conditions to achieve high-precision and robust joint estimation of camera relative pose and focal length. Summary of the Invention

[0005] To address the technical problems of low accuracy and sensitivity to noise in the joint estimation of camera relative pose and focal length under semi-calibration conditions in existing technologies, this invention provides a method and system for estimating camera relative pose and camera intrinsic parameters.

[0006] This invention provides a method for estimating camera relative pose and camera intrinsic parameters, comprising: Obtain an external prior direction and use the external prior direction to transform the two views acquired by the camera, so as to constrain the relative rotation between the two views to a one-dimensional rotation about a single axis. A semi-calibrated focal length model of the camera is constructed. Based on the semi-calibrated focal length model and epipolar geometry constraints, a nonlinear least squares optimization model is constructed with the sum of squared algebraic residuals of all feature corresponding points in the transformed two views as the objective function and the rotation angle, inverse focal length and translation vector of the one-dimensional rotation as optimization variables. The structural features of the objective function are used to determine the target association between the translation vector, the rotation angle, and the inverse focal length, so as to remove the translation vector from the optimization variables and achieve dimensionality reduction of the nonlinear least squares optimization model; The nonlinear least squares optimization model after dimensionality reduction is iteratively optimized to obtain the global optimal estimate of the rotation angle and the inverse focal length, and the global optimal estimate of the translation vector is determined by combining the target association.

[0007] The present invention also provides a system for estimating camera relative pose and camera intrinsic parameters, comprising: The image transformation module is used to obtain the external prior direction and use the external prior direction to transform the two views acquired by the camera, so as to constrain the relative rotation between the two views to a one-dimensional rotation about a single axis. The model building module is used to build a semi-calibrated focal length model of the camera. Based on the semi-calibrated focal length model and epipolar geometric constraints, a nonlinear least squares optimization model is built with the sum of squared algebraic residuals of all feature corresponding points in the transformed two views as the objective function and the rotation angle, inverse focal length and translation vector of the one-dimensional rotation as optimization variables. The model dimensionality reduction module is used to determine the target association between the translation vector, the rotation angle, and the inverse focal length by utilizing the structural features of the objective function, so as to remove the translation vector from the optimization variables and achieve dimensionality reduction of the nonlinear least squares optimization model; The model solving module is used to iteratively optimize and solve the dimensionality-reduced nonlinear least squares optimization model to obtain the global optimal estimate of the rotation angle and the inverse focal length, and to determine the global optimal estimate of the translation vector in combination with the target association.

[0008] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the camera relative pose and camera intrinsic parameter estimation method as described above.

[0009] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for estimating camera relative pose and camera intrinsic parameters as described above.

[0010] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the camera relative pose and camera intrinsic parameter estimation method as described above.

[0011] Compared with the prior art, the method and system for estimating camera relative pose and camera intrinsic parameters provided by the present invention have the following beneficial effects: First, the 3D rotation estimation is reduced to one dimension using external prior directions, simplifying the subsequent optimization problem. Second, a nonlinear least squares model is constructed to jointly optimize the relative pose (including translation and rotation) and focal length. This nonlinear least squares model uses the sum of squared algebraic residuals of all feature-corresponding points in the transformed two views as the objective function for optimization. Compared to a minimum solver that only uses the fewest corresponding points, this method can significantly improve the estimation accuracy of relative pose and focal length. To further improve the solution efficiency, the structural characteristics of the objective function are used to determine the target association between the translation vector, the rotation angle, and the inverse focal length. This allows the optimal estimate of the translation vector to be determined based on the optimal estimates of the rotation angle and the inverse focal length, thereby removing the translation vector from the optimization variables, reducing the computational complexity of the nonlinear least squares model, and improving optimization efficiency. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating the method for estimating the relative pose and intrinsic parameters of a camera provided by the present invention.

[0014] Figure 2 A comparison chart showing how focal length error varies with image noise on synthetic data using different estimation methods.

[0015] Figure 3 A comparison chart showing how rotation error varies with image noise on synthetic data using different estimation methods.

[0016] Figure 4 A comparison chart showing how translation error varies with image noise on synthetic data for different estimation methods.

[0017] Figure 5 A comparison chart showing how focal length error varies with pitch angle noise on synthetic data using different estimation methods.

[0018] Figure 6A comparison chart showing how rotation error varies with pitch noise on synthetic data using different estimation methods.

[0019] Figure 7 A comparison chart showing how translation error varies with pitch noise on synthetic data using different estimation methods.

[0020] Figure 8 A comparison chart showing how focal length error varies with roll angle noise on synthetic data using different estimation methods.

[0021] Figure 9 A comparison chart showing how rotational error varies with roll noise on synthetic data using different estimation methods.

[0022] Figure 10 A comparison chart showing the variation of translation error with roll noise on synthetic data for different estimation methods. Detailed Implementation

[0023] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0024] In the description of this invention, it should be noted that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0025] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0026] In one embodiment of this application, the method for estimating the camera's relative pose and camera intrinsic parameters specifically includes: Step S110: Obtain the external prior direction and use the external prior direction to transform the two views acquired by the camera, so as to constrain the relative rotation between the two views to a one-dimensional rotation about a single axis.

[0027] Step S120: Construct a semi-calibrated focal length model of the camera. Based on the semi-calibrated focal length model and epipolar geometric constraints, construct a nonlinear least squares optimization model with the sum of squared algebraic residuals of corresponding feature points in the transformed two views as the objective function and the rotation angle, inverse focal length, and translation vector of one-dimensional rotation as optimization variables.

[0028] Step S130: Utilize the structural features of the objective function to determine the objective relationship between the translation vector, rotation angle, and inverse focal length, so as to remove the translation vector from the optimization variables and achieve dimensionality reduction of the nonlinear least squares optimization model.

[0029] Step S140: Iteratively optimize and solve the dimension-reduced nonlinear least squares optimization model to obtain the global optimal estimates of the rotation angle and inverse focal length, and combine the target association to determine the global optimal estimate of the translation vector.

[0030] This method is applicable to the alignment and stitching of two images captured sequentially by a monocular camera. Therefore, the camera in this method is a monocular camera, and the two views are two images obtained by the monocular camera capturing the same scene sequentially. Generally, the parameters required for aligning and stitching these two images include camera intrinsic parameters and the relative camera pose between the two images. The relative camera pose refers to the change in camera pose when capturing the two images, including attitude changes (relative rotation) and position changes (relative displacement).

[0031] Generally, relative rotation is a three-dimensional variable, encompassing rotation amounts in all three dimensions. Directly estimating this three-dimensional rotation results in low overall computational efficiency. Therefore, this method first utilizes external prior directions to reduce the dimensionality of the three-dimensional rotation estimation to one dimension, thereby simplifying the subsequent optimization problem. Specifically, transforming the two views acquired by the camera using external prior directions means aligning the camera coordinate systems of the two views to the prior directions, ensuring that the directions of the same coordinate axis in both camera coordinate systems are the prior directions. For example, if the Y-axis directions of both camera coordinate systems are the prior directions, then one camera coordinate system only needs to rotate around the Y-axis to maintain orientation alignment with the other camera coordinate system.

[0032] Secondly, a nonlinear least squares model is constructed to jointly optimize relative pose (including translation and rotation) and focal length. This nonlinear least squares model uses the sum of squared algebraic residuals of all feature-corresponding points in the transformed two views as the objective function for optimization. Compared with the minimum solver that only uses the fewest corresponding points, this method can significantly improve the estimation accuracy of relative pose and focal length. To further improve the solution efficiency, the structural characteristics of the objective function are used to determine the target correlation between the translation vector, rotation angle, and inverse focal length. This allows the optimal estimate of the translation vector to be determined based on the optimal estimates of the rotation angle and inverse focal length, thereby removing the translation vector from the optimization variables, reducing the computational complexity of the nonlinear least squares model, and improving optimization efficiency.

[0033] In summary, the camera relative pose and camera intrinsic parameter estimation method provided in this application can achieve high-precision, high-robustness and high-efficiency joint estimation of camera relative pose and focal length.

[0034] In one specific implementation, the external prior direction is the gravity direction provided by the inertial measurement unit (IMU), and the rotation angle of the one-dimensional rotation is the yaw angle. For example, the Y-axis direction in the camera coordinate system can be made to be the gravity direction, in which case one camera coordinate system only needs to rotate around the Y-axis to maintain attitude alignment with the other camera coordinate system.

[0035] The transformation of the two views acquired by the camera is performed using the external prior direction, specifically including: calculating the rotation matrix that aligns the camera coordinate system of the two views to the direction of gravity; and performing gravity alignment transformation on the image feature points of the two views using the two rotation matrices respectively.

[0036] The specific transformation process includes: given two images (two views) observing the same scene, obtaining the image through feature extraction and matching algorithms (such as SIFT, ORB, etc.). For feature correspondence points (first, feature points are extracted from both images using a feature extraction algorithm, and then feature correspondence points between the two images are determined using an image matching algorithm, such as first calculating the feature descriptors of each feature point, and then using feature points with the same feature descriptors between the two images as feature correspondence points). Simultaneously, gravity direction information at corresponding moments in the two views is obtained from the inertial measurement unit (IMU). In the vision-inertial system, the camera and IMU are rigidly coupled, and the gravity direction provided by the IMU can be transformed to the camera coordinate system using a calibrated camera-IMU extrinsic parameter rotation matrix.

[0037] Then calculate the two rotation matrices that align the camera coordinate system to the direction of gravity. and For the first For the corresponding feature points, their homogeneous image coordinates are and , No. The coordinates of the corresponding feature points after gravity alignment are:

[0038] and These are the coordinates of image points in the two transformed views. After gravity alignment, the relative rotation is constrained to rotation only around the gravity axis (Y-axis), i.e. ,in Yaw angle Let be the rotation matrix about the Y-axis. In this way, the original three-dimensional rotation estimation problem is reduced to a one-dimensional rotation estimation problem involving only one yaw angle, which significantly reduces the computational complexity and sensitivity to initial values.

[0039] In one specific implementation, a semi-calibrated focal length model of the camera is established, which specifically includes: under semi-calibration conditions, modeling the camera intrinsic parameter matrix as a diagonal matrix containing only focal length and generating its inverse matrix; establishing a mapping relationship between the inverse focal length and feature points in the two views based on the inverse matrix and the transformed two views, so as to introduce the inverse focal length into the optimization variables of the optimization model.

[0040] Specifically, under semi-calibration conditions, the camera intrinsic parameter matrix Only focal length variable : ; Its inverse matrix for: ; in, It is the inverse focal length.

[0041] Camera intrinsic parameter matrix and its inverse matrix The semi-calibrated focal length model that constitutes the camera.

[0042] Then, by normalizing the corresponding feature points after gravity alignment using the inverse matrix, we can obtain the focal length-related normalized ray vector: ; ; in, and These are used to represent the mapping relationship between the inverse focal length and feature points in the two views, respectively.

[0043] In one specific implementation, the translation vector is a normalized translation vector that satisfies the constraint that the magnitude is 1.

[0044] Furthermore, the structural characteristics of the objective function are used to determine the target association between the translation vector, the rotation angle, and the inverse focal length. Specifically, this includes: expressing the objective function as a quadratic form with respect to the translation vector, and solving the minimum eigenvalue problem of an objective matrix, and using the eigenvector corresponding to the minimum eigenvalue as the translation vector.

[0045] By following the steps above, the translation vector can be removed from the optimization variables, thereby reducing the dimensionality of the nonlinear least squares optimization model.

[0046] Specifically, the dimensionality reduction process of the nonlinear least squares optimization model is explained as follows: The optimization objective of a nonlinear least squares optimization model is to find a set of optimization variables (a set of optimization variables constitutes an optimization variable vector). Such that the sum of squared algebraic residuals of all corresponding feature points Minimum.

[0047] ; Among them, the optimization variable vector It includes yaw angle Reverse focal length and translation vector The three components , and The algebraic residual of the i-th pair of feature points Defined based on the constraint that the triplet product is zero in epipolar geometry, specifically: .

[0048] Furthermore, due to the translation vector It can only recover to the scale uncertainty, therefore in the optimization model, the translation vector For a normalized translation vector, apply a constraint with a magnitude of 1, i.e., an identity norm constraint: ; Meanwhile, to ensure the physical meaning and numerical stability of the solution, boundary constraints can be set for the optimization variables: .

[0049] in, and These are the lower and upper boundaries of the optimization variable vector, respectively.

[0050] The establishment of a nonlinear least squares optimization model provides a unified mathematical framework for the joint estimation of pose and focal length.

[0051] The process of determining the target association using the structural features of the objective function is as follows: First, the objective function Rewritten as about translation vectors The quadratic form. Therefore, an auxiliary vector is defined. Then, all of them N The auxiliary vectors corresponding to the feature points are stacked to form a matrix. At this point, the objective function can be expressed as: = In order to meet the unit norm constraint Minimizing this quadratic form requires solving a Rayleigh quotient minimization problem. This is achieved by introducing an objective matrix. The problem is transformed into finding... smallest unit vector According to linear algebra theory, the solution to this problem is a matrix. The smallest eigenvalue The corresponding eigenvectors. Therefore, by utilizing the target correlation, i.e., solving the problem of finding the minimum eigenvalue of the target matrix, the eigenvector corresponding to the minimum eigenvalue can be used as the optimal translation vector. This method will translate the vector. An analytical removal of the optimization variables reduces the dimensionality of the original optimization problem to only concerning... and Two-parameter optimization problem This greatly improves the solution efficiency.

[0052] After reducing the dimensionality of the optimization problem, the dimensionality-reduced nonlinear least squares optimization model is solved iteratively using an iterative optimization algorithm.

[0053] In one specific implementation, the iterative optimization solution employs the Sequential Quadratic Programming (SQP) algorithm. The SQP algorithm is an efficient nonlinear constrained optimization method that generates a search direction in each iteration by solving a quadratic programming (QP) subproblem. In each iteration of the Sequential Quadratic Programming algorithm, the Hessian matrix is ​​approximated using the Gauss-Newton method, and the search direction is obtained by solving the quadratic programming subproblem. The optimization variables are then updated using a line search strategy.

[0054] Specifically, the optimization process of each iteration of the sequential quadratic programming algorithm is as follows: At the current iteration point To approximate the original problem, construct a quadratic programming subproblem: ; ; ; in, This is the search direction to be determined. This is the identity norm constraint function. To improve computational efficiency, the Gauss-Newton method is used to approximate the Hessian matrix of the objective function, i.e., ,in It is the residual vector Regarding optimization variables The Jacobian matrix. By solving this QP subproblem, the search direction can be obtained. Subsequently, a suitable linear search step size is determined by combining a linear search strategy (such as the Armijo criterion). And according to the updated formula This is used to update and optimize the variable vector. This iterative process continues until a convergence condition is met (such as the gradient norm being less than a threshold or the iteration step size being too small).

[0055] Since the focal length-pose joint optimization problem is typically non-convex, conventional gradient descent methods are prone to getting trapped in local optima. To obtain a global optimum estimate, this method preferably employs a multi-initial-value strategy to iteratively optimize the dimensionality-reduced nonlinear least-squares optimization model. This strategy significantly increases the probability of finding the global optimum by initiating the iterative optimization process with multiple sets of different initial values ​​and selecting the optimal result from all obtained local optima.

[0056] In one specific implementation, the multiple initialization strategy involves systematically sampling in the parameter space to generate initial values. Specifically, at the rotation angle (i.e., yaw angle)... Within the preset range, for example Perform uniform sampling to generate Initial values ​​of rotation angles At the same time, at focal length Within the preset range, for example Perform uniform sampling to generate Initial value of each focal length And convert these initial focal length values ​​into corresponding inverse focal length initial values. For each set of initial values ​​( , The initial value of the corresponding translation vector can be calculated using the aforementioned target association method. This generates complete initial values. Subsequently, the aforementioned SQP optimization algorithm is executed independently for each set of initial values, yielding a series of local optima. Finally, the objective function values ​​corresponding to all these solutions are compared. The solution that minimizes the objective function value is selected as the final global optimal estimate. ; This strategy effectively overcomes the challenges posed by nonconvexity by extensively exploring the solution space, significantly improving the reliability and accuracy of the estimation results.

[0057] To verify the effectiveness of the method of this invention, experimental evaluation was conducted on synthetic data. This invention addresses the conditions of monocular semi-calibration, which differs from multi-view full calibration schemes (which utilize fixed baseline geometric constraints between multiple lenses to perform pose estimation under conditions where intrinsic parameters are fully known). Therefore, this experiment focuses on verifying the joint estimation performance of this invention under conditions of unknown focal length. The specific experimental setup is as follows: 1. Data Generation: Randomly sample 100 3D points from the cube, with x and y coordinates ranging from... meters, z-coordinate range is Meters. The principal point is located at Focal length from interval Randomly selected from the data. The range of rotation angles between two consecutive views is... .

[0058] 2. Noise Settings: Image noise, roll noise, and pitch noise are added to the composite data. When evaluating the effects of pitch and roll noise, pixel noise is fixed at 1 pixel.

[0059] 3. Comparison method: –Ding-4DOF: A 4-point minimum solver for joint estimation of relative pose and focal length given the direction of gravity.

[0060] –Kuke-6DOF: A 6-point solver for joint estimation of relative pose and focal length under semi-calibrated conditions.

[0061] 4. Evaluation indicators: –Relative error of focal length: .

[0062] – Rotational error: .

[0063] – Translation error: .

[0064] 5. Experimental Results: (1) Experimental results of image noise: such as Figures 2 to 4 As shown ( Figure 2 The four sub-figures from top to bottom show the changes in focal length error with image noise on synthetic data, respectively, under random motion, planar motion, lateral motion, and forward motion. Figure 3 The four sub-images from top to bottom show the comparison of rotation error with image noise on synthetic data, respectively, under random motion, planar motion, lateral motion, and forward motion. Figure 4The charts show a comparison of translation error versus image noise on synthetic data. The four sub-plots (from top to bottom) sequentially illustrate the differences under random motion, planar motion, lateral motion, and forward motion. As the image noise level increases, the errors of all methods tend to rise. However, the method of this invention achieves the lowest error in almost all sub-plots. Its advantage is particularly evident in rotation and translation estimation; the errors of Ding-4DOF and Kuke-6DOF increase much faster with increasing noise. For focal length estimation, the method of this invention also maintains significantly higher accuracy across all motion modes.

[0065] (2) Pitch angle noise test results: such as Figures 5 to 7 As shown ( Figure 5 The four sub-plots, from top to bottom, show the comparison of focal length error with pitch angle noise on synthetic data, under random motion, planar motion, lateral motion, and forward motion. Figure 6 The four sub-plots, from top to bottom, show the comparison of rotation error with pitch noise on synthetic data, illustrating the differences under random motion, planar motion, lateral motion, and forward motion. Figure 4 The four sub-plots (from top to bottom) show the variation of translation error with pitch noise on synthetic data, illustrating the differences under random motion, planar motion, lateral movement, and forward motion. As pitch noise increases, the errors of Ding-4DOF and Kuke-6DOF increase more significantly, especially in translation estimation. In contrast, the method of this invention remains quite stable, continuing to achieve the lowest error in most cases. Its focal length and rotation estimates show only minor changes across the entire noise range, indicating a strong ability to recover from pitch disturbances.

[0066] (3) Roll angle noise test results: such as Figures 8 to 10 As shown ( Figure 8 The four sub-plots, from top to bottom, show the comparison of focal length error with roll angle noise in synthetic data, under random motion, planar motion, lateral motion, and forward motion. Figure 9 The four sub-plots, from top to bottom, show the comparison of rotational error with roll angle noise on the synthetic data, illustrating the differences under random motion, planar motion, lateral motion, and forward motion. Figure 10 The four subplots (from top to bottom) show the translation error versus roll noise on the synthetic data, comparing the results under random motion, planar motion, lateral motion, and forward motion. Similar to pitch noise, roll noise reduces the quality of the gravity prior. Nevertheless, the method of this invention is far less affected by roll noise than competing methods, and still produces the lowest error in most subplots.

[0067] To further verify the performance of the method of this invention in real-world scenarios, experiments were conducted on the KITTI dataset. KITTI is a widely used benchmark dataset in the fields of autonomous driving and geometric vision, providing monocular image sequences acquired in diverse outdoor environments such as city streets, residential areas, and highways, as well as accurate ground-truth poses obtained by GPS / IMU-based positioning systems. Unlike multi-view full calibration schemes that require three or more synchronized cameras, this experiment only uses a single camera image from the KITTI dataset to verify the performance of the method of this invention.

[0068] 1. Experimental setup: SIFT was used for feature extraction and matching. Sequences from 00 to 10 in the KITTI dataset were selected for testing.

[0069] 2. Experimental Results: Table 1. Quantitative comparison results of the method of the present invention with Ding-4DOF and Kuke-6DOF on the KITTI dataset.

[0070] As can be seen from Table 1, the method of this invention outperforms the two comparative methods in all three metrics across all sequences. Specifically: (1) Focal length estimation: The average focal length error of the method of the present invention is about 0.022%, while Ding-4DOF is about 0.086% and Kuke-6DOF is about 0.213%. The method of the present invention improves the focal length estimation accuracy by about 4-10 times.

[0071] (2) Rotation estimation: The average rotation error of the method of the present invention is about 0.052°, while Ding-4DOF is about 0.129° and Kuke-6DOF is about 0.865°. The method of the present invention improves the rotation estimation accuracy by about 2.5-16 times.

[0072] (3) Translation estimation: The average translation error of the method of the present invention is about 1.286°, while Ding-4DOF is about 1.751° and Kuke-6DOF is about 3.514°. The method of the present invention improves the translation estimation accuracy by about 1.4-2.7 times.

[0073] This application also provides a system for estimating the relative pose and focal length of a camera, comprising: The image transformation module is used to obtain the external prior direction and use the external prior direction to transform the two views acquired by the camera, so as to constrain the relative rotation between the two views to a one-dimensional rotation about a single axis. The model building module is used to build a semi-calibrated focal length model of the camera. Based on the semi-calibrated focal length model and epipolar geometric constraints, a nonlinear least squares optimization model is built with the sum of squared algebraic residuals of all feature corresponding points in the transformed two views as the objective function and the rotation angle, inverse focal length and translation vector of the one-dimensional rotation as optimization variables. The model dimensionality reduction module is used to determine the target association between the translation vector, the rotation angle, and the inverse focal length by utilizing the structural features of the objective function, so as to remove the translation vector from the optimization variables and achieve dimensionality reduction of the nonlinear least squares optimization model; The model solving module is used to iteratively optimize and solve the dimensionality-reduced nonlinear least squares optimization model to obtain the global optimal estimate of the rotation angle and the inverse focal length, and to determine the global optimal estimate of the translation vector in combination with the target association.

[0074] The aforementioned system is used to implement the camera relative pose and camera intrinsic parameter estimation method provided in this application. Each module is used to implement each step of the method. Therefore, a detailed description of this system can be found in the detailed description of the camera relative pose and camera intrinsic parameter estimation method described above. This system has the same technical effects and solves the same technical problems as the aforementioned camera relative pose and camera intrinsic parameter estimation method.

[0075] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to perform the estimation methods for camera relative pose and camera intrinsic parameters provided by the above methods.

[0076] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods for estimating camera relative pose and camera intrinsic parameters provided by the methods described above.

[0077] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0078] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for estimating camera relative pose and camera intrinsic parameters, characterized in that, include: Obtain an external prior direction and use the prior direction to transform the two views acquired by the camera, so as to constrain the relative rotation between the two views to a one-dimensional rotation about a single axis pointing to the prior direction; A semi-calibrated focal length model of the camera is constructed. Based on the semi-calibrated focal length model and epipolar geometry constraints, a nonlinear least squares optimization model is constructed with the sum of squared algebraic residuals of all feature corresponding points in the transformed two views as the objective function and the rotation angle, inverse focal length and translation vector of the one-dimensional rotation as optimization variables. The structural features of the objective function are used to determine the target association between the translation vector, the rotation angle, and the inverse focal length, so as to remove the translation vector from the optimization variables and achieve dimensionality reduction of the nonlinear least squares optimization model; The nonlinear least squares optimization model after dimensionality reduction is iteratively optimized to obtain the global optimal estimate of the rotation angle and the inverse focal length, and the global optimal estimate of the translation vector is determined by combining the target association.

2. The method for estimating camera relative pose and camera intrinsic parameters according to claim 1, characterized in that, The external prior direction is the gravity direction provided by the inertial measurement unit, and the rotation angle of the one-dimensional rotation is the yaw angle; Transforming the two views acquired by the camera using the aforementioned external prior direction specifically includes: Calculate the rotation matrix for aligning the camera coordinate system to the direction of gravity in both views; The two rotation matrices are used to perform gravity alignment transformation on the image feature points of the two views respectively.

3. The method for estimating camera relative pose and camera intrinsic parameters according to claim 1, characterized in that, Establishing the semi-calibrated focal length model of the camera specifically includes: Under semi-calibration conditions, the camera intrinsic parameter matrix is ​​modeled as a diagonal matrix containing only the focal length and its inverse matrix is ​​generated; Based on the inverse matrix and the transformed two views, a mapping relationship is established between the inverse focal length and the feature points in the two views, so as to introduce the inverse focal length into the optimization variables of the optimization model.

4. The method for estimating camera relative pose and camera intrinsic parameters according to claim 1, characterized in that, The translation vector is a normalized translation vector and satisfies the constraint that the magnitude is 1.

5. The method for estimating camera relative pose and camera intrinsic parameters according to claim 1, characterized in that, Determining the target association between the translation vector, the rotation angle, and the inverse focal length using the structural features of the objective function specifically includes: The objective function is expressed as a quadratic form with respect to the translation vector, and the eigenvector corresponding to the minimum eigenvalue of the objective matrix is ​​taken as the translation vector by solving the problem of finding the minimum eigenvalue of the objective matrix.

6. The method for estimating camera relative pose and camera intrinsic parameters according to claim 1, characterized in that, The iterative optimization solution employs a sequential quadratic programming algorithm.

7. The method for estimating camera relative pose and camera intrinsic parameters according to claim 6, characterized in that, In each iteration of the sequential quadratic programming algorithm, the Hessian matrix is ​​approximated using the Gauss-Newton method, and the search direction is obtained by solving the quadratic programming subproblem. The optimization variables are then updated using a line search strategy.

8. The method for estimating camera relative pose and camera intrinsic parameters according to claim 1, characterized in that, The dimensionality-reduced nonlinear least squares optimization model is iteratively optimized and solved, specifically including: A multi-initial-value strategy is adopted, which initiates iterative optimization of the dimension-reduced nonlinear least squares optimization model by using multiple sets of initial values ​​for the rotation angle and the inverse focal length, and selects the optimal result from the multiple local optimal solutions as the global optimal estimate of the rotation angle and the inverse focal length.

9. The method for estimating camera relative pose and camera intrinsic parameters according to claim 8, characterized in that, The multiple initial value strategy includes: Sampling is performed within a preset range of the rotation angle and the focal length to generate multiple sets of initial values ​​for the rotation angle and the focal length, and the initial value of the focal length is converted into the initial value of the inverse focal length.

10. A system for estimating camera relative pose and camera intrinsic parameters, characterized in that, include: The image transformation module is used to obtain the external prior direction and use the external prior direction to transform the two views acquired by the camera, so as to constrain the relative rotation between the two views to a one-dimensional rotation about a single axis. The model building module is used to build a semi-calibrated focal length model of the camera. Based on the semi-calibrated focal length model and epipolar geometric constraints, a nonlinear least squares optimization model is built with the sum of squared algebraic residuals of all feature corresponding points in the transformed two views as the objective function and the rotation angle, inverse focal length and translation vector of the one-dimensional rotation as optimization variables. The model dimensionality reduction module is used to determine the target association between the translation vector, the rotation angle, and the inverse focal length by utilizing the structural features of the objective function, so as to remove the translation vector from the optimization variables and achieve dimensionality reduction of the nonlinear least squares optimization model; The model solving module is used to iteratively optimize and solve the dimensionality-reduced nonlinear least squares optimization model to obtain the global optimal estimate of the rotation angle and the inverse focal length, and to determine the global optimal estimate of the translation vector in combination with the target association.

Images