Camera pose measurement method and device based on coplanar feature points
By obtaining the normalized coordinates of coplanar feature points and the first coordinates of the world coordinate system, calculating the homography matrix and iteratively generating local pose data, the ambiguity problem of camera pose measurement under coplanar feature points is solved, and more stable and accurate pose measurement results are achieved.
Patent Information
- Application Number
- CN202110223982.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-01
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-03-01
AI Technical Summary
Existing camera pose measurement methods suffer from ambiguity when feature points are coplanar, leading to unstable calculation results and low accuracy.
By obtaining the normalized coordinates of coplanar feature points and the first coordinates of the world coordinate system, the homography matrix is calculated. The initial pose data is calculated using the homography matrix, and the first and second local pose data are generated through an iterative algorithm. Finally, the local pose data with smaller error is selected as the camera pose measurement result.
It improves the stability and accuracy of camera pose measurement, solves the ambiguity problem of pose measurement under coplanar feature points, and ensures the accuracy of calculation results.
Smart Images

Figure CN112907669B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a camera posture measurement method and device based on coplanar feature points. Background Art
[0002] Camera pose measurement technology has been widely used in augmented reality and robot autonomous positioning and navigation. This technology calculates the position and pose of the camera in three-dimensional space under the condition that the distribution of several feature points in three-dimensional space and their projection positions on the camera image are known. In particular, when all feature points are located on the same plane, the problem is transformed into a pose measurement problem based on coplanar feature points. Existing measurement methods can generally be divided into two types: analytical methods and iterative methods. The analytical method is very fast but not very accurate. The iterative method generally has higher accuracy, but it faces the problem of ambiguity in the pose solution when the feature points are coplanar, resulting in unstable calculation results. Summary of the invention
[0003] The present invention provides a camera pose measurement method and device based on coplanar feature points, which can alleviate the ambiguity problem of pose measurement when the feature points are coplanar, and the final camera pose measurement result is more stable and accurate.
[0004] In the first aspect, an embodiment of the present invention provides a camera pose measurement method based on coplanar feature points, the method comprising: obtaining normalized coordinates of the coplanar feature points and first coordinates of the coplanar feature points on a target plane in a world coordinate system; the plane where the coplanar feature points are located coincides with the target plane; calculating a homography matrix based on the normalized coordinates and the first coordinates; using the homography matrix to calculate first initial pose data of the camera; determining first local pose data and second local pose data based on the first initial pose data; and determining a camera pose measurement result based on the first local pose data and the second local pose data.
[0005] In the second aspect, an embodiment of the present invention also provides a camera pose measurement device based on coplanar feature points, the device comprising: an acquisition module, used to acquire the normalized coordinates of the coplanar feature points and the first coordinates of the coplanar feature points on the target plane of the world coordinate system; the plane where the coplanar feature points are located coincides with the target plane; a matrix module, used to calculate the homography matrix according to the normalized coordinates and the first coordinates; a pose module, used to calculate the first initial pose data of the camera using the homography matrix; an iteration module, used to determine the first local pose data and the second local pose data according to the first initial pose data; a measurement module, used to determine the camera pose measurement result according to the first local pose data and the second local pose data.
[0006] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned camera pose measurement method based on coplanar feature points when executing the computer program.
[0007] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing the above-mentioned camera pose measurement method based on coplanar feature points.
[0008] The embodiments of the present invention bring the following beneficial effects: The embodiments of the present invention provide a camera pose measurement scheme based on coplanar feature points, the scheme comprising: obtaining the normalized coordinates of the coplanar feature points and the first coordinates of the coplanar feature points on the target plane of the world coordinate system; the plane where the coplanar feature points are located coincides with the target plane; calculating the homography matrix according to the normalized coordinates and the first coordinates; calculating the first initial pose data of the camera using the homography matrix; determining the first local pose data and the second local pose data according to the first initial pose data; determining the camera pose measurement result according to the first local pose data and the second local pose data. The embodiments of the present invention take into account the ambiguity problem of pose measurement when the feature points are coplanar, and through the process of generating the first local pose data and the second local pose data, and then optimizing the camera pose measurement result based on the first local pose data and the second local pose data, the final camera pose measurement result is more stable and accurate.
[0009] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
[0010] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0012] Figure 1 A flow chart of a camera pose measurement method based on coplanar feature points provided by an embodiment of the present invention;
[0013] Figure 2A schematic diagram of the relationship between the world coordinate system and the plane where the feature points are located provided by an embodiment of the present invention;
[0014] Figure 3 A schematic diagram of mirror flipping provided by an embodiment of the present invention;
[0015] Figure 4 A structural block diagram of a camera pose measurement device based on coplanar feature points provided by an embodiment of the present invention;
[0016] Figure 5 A structural block diagram of another camera pose measurement device based on coplanar feature points provided by an embodiment of the present invention;
[0017] Figure 6 A structural block diagram of a posture module provided in an embodiment of the present invention;
[0018] Figure 7 A structural block diagram of an iteration module provided in an embodiment of the present invention;
[0019] Figure 8 A structural block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0021] An embodiment of the present invention provides a camera posture measurement method and device based on coplanar feature points. The method is applied to the field of image processing and computer vision technology, and improves the stability of calculation results while ensuring high accuracy.
[0022] To facilitate understanding of this embodiment, a camera pose measurement method based on coplanar feature points disclosed in an embodiment of the present invention is first introduced in detail.
[0023] The embodiment of the present invention provides a camera pose measurement method based on coplanar feature points, see Figure 1 A flow chart of a camera pose measurement method based on coplanar feature points is shown, the method comprising the following steps:
[0024] Step S102, obtaining the normalized coordinates of the coplanar feature points and the first coordinates of the coplanar feature points on the target plane of the world coordinate system.
[0025] In the embodiment of the present invention, the normalized coordinates are the coordinates of the coplanar feature points in the camera coordinate system. The plane where the coplanar feature points are located in the physical space coincides with a plane in the world coordinate system, and the plane is used as the target plane. The first coordinates are the coordinates of the coplanar feature points in the world coordinate system.
[0026] Step S104, calculating a homography matrix according to the normalized coordinates and the first coordinates.
[0027] In the embodiments of the present invention, see Figure 2 The schematic diagram of the relationship between the world coordinate system and the plane where the feature point is located is shown, and the normalized coordinates and the first coordinates are used to calculate the homography matrix from the plane where the feature point is located in the physical space to the image plane of the camera.
[0028] Step S106, calculating the first initial pose data of the camera using the homography matrix.
[0029] In the embodiment of the present invention, after the homography matrix is obtained, the homography matrix is decomposed to obtain the first initial pose data.
[0030] Step S108, determining first local pose data and second local pose data according to the first initial pose data.
[0031] In an embodiment of the present invention, iterative processing is performed based on the first initial pose data to obtain first local pose data, and the first local pose data is used to obtain second local pose data through calculation and iteration. The embodiment of the present invention uses an iterative algorithm to obtain the first local pose data and the second local pose data, which can improve the accuracy of the final measurement result.
[0032] Step S110, determining a camera pose measurement result according to the first local pose data and the second local pose data.
[0033] In an embodiment of the present invention, the first local pose data and the second local pose data are screened to obtain a more accurate global optimal pose as a camera pose measurement result.
[0034] The embodiment of the present invention provides a camera pose measurement scheme based on coplanar feature points, the scheme comprising: obtaining the normalized coordinates of the coplanar feature points and the first coordinates of the coplanar feature points on the target plane of the world coordinate system; the plane where the coplanar feature points are located coincides with the target plane; calculating the homography matrix according to the normalized coordinates and the first coordinates; calculating the first initial pose data of the camera using the homography matrix; determining the first local pose data and the second local pose data according to the first initial pose data; determining the camera pose measurement result according to the first local pose data and the second local pose data. The embodiment of the present invention takes into account the ambiguity problem of pose measurement when the feature points are coplanar, and makes the final camera pose measurement result more stable and accurate by generating the first local pose data and the second local pose data, and then optimizing the camera pose measurement result based on the first local pose data and the second local pose data.
[0035] In one embodiment, before obtaining the normalized coordinates of the coplanar feature points and the first coordinates of the coplanar feature points on the target plane of the world coordinate system, the following steps may be performed:
[0036] The pixel coordinates of the coplanar feature points on the image are normalized to obtain normalized coordinates; the image is captured by a camera; a world coordinate system is established; and the plane where the coplanar feature points are located coincides with a plane of the world coordinate system.
[0037] In an embodiment of the present invention, an image including a plurality of coplanar feature points is captured by a camera, and the pixel coordinates of the coplanar feature points on the image are normalized, thereby converting the pixel coordinates in the image coordinate system into normalized coordinates in the camera coordinate system. In order to facilitate data processing, a world coordinate system needs to be established so that the plane where the feature points are located coincides with a certain plane of the world coordinate system.
[0038] In one embodiment, the pixel coordinates of the coplanar feature points on the image are normalized according to the following formula to obtain normalized coordinates:
[0039]
[0040] Among them, (x i y i ) T is the pixel coordinate of the feature point, (u i v i ) T is the normalized coordinate of the feature point, f x and f y is the focal length of the camera, c x and c y is the principal point coordinate of the camera, and N is the number of coplanar feature points.
[0041] In one embodiment, calculating the homography matrix according to the normalized coordinates and the first coordinates may be performed according to the following steps.
[0042] The direct linear transformation method is used to calculate the homography matrix from the feature point plane to the image plane.
[0043] In the embodiment of the present invention, a direct linear transformation method is used for calculation, and a formula of the form
[0044] The homography matrix of .
[0045] In one embodiment, the calculation of the first initial pose data of the camera using the homography matrix may be performed according to the following steps:
[0046] Determine the first matrix and the second matrix according to the homography matrix; perform singular value decomposition on the first matrix to obtain the singular value decomposition result; determine the rotation matrix of the camera using the singular value decomposition result; determine the translation vector of the camera using the singular value decomposition result and the second matrix; use the rotation matrix and the translation vector as the first initial pose data of the camera.
[0047] In the embodiment of the present invention, the first matrix includes some elements in the homography matrix. The second matrix includes elements in the homography matrix other than the elements included in the first matrix.
[0048] In one embodiment, the first matrix and the second matrix are determined according to the homography matrix according to the following formula: H 1:2 =[h1 h2], where H is the homography matrix, H 1:2 is the first matrix and h3 is the second matrix.
[0049] In one embodiment, singular value decomposition is performed on the first matrix according to the following formula to obtain a singular value decomposition result:
[0050]
[0051] Among them, H1:2 is the first matrix, and U, S, and V are the results of singular value decomposition.
[0052] In one embodiment, the rotation matrix of the camera is determined using the singular value decomposition result according to the following formula:
[0053]
[0054]
[0055] R=[m1 m2 m1×m2]
[0056] Among them, U and V are the results of singular value decomposition, and R is the rotation matrix of the camera.
[0057] In one embodiment, the camera translation vector is determined using the singular value decomposition result and the second matrix according to the following formula: Among them, h3 is the second matrix, s1 and s2 are the results of singular value decomposition, and t is the translation vector of the camera.
[0058] In an embodiment of the present invention,
[0059] In one embodiment, determining the first local pose data and the second local pose data according to the first initial pose data may be performed according to the following steps:
[0060] The first initial pose data is used as the iteration starting point, and the first local pose data is calculated using a preset iterative algorithm; the first local pose data is mirror-flipped to obtain the second initial pose data; the second initial pose data is used as the iteration starting point, and the second local pose data is calculated using a preset iterative algorithm.
[0061] In an embodiment of the present invention, the same iterative algorithm may be used to generate the first local pose data and the second local pose data, and the starting points of the two iterations are the first initial pose data and the second initial pose data, respectively.
[0062] In one embodiment, the first local pose data is mirror-flipped according to the following formula to obtain the second initial pose data:
[0063] R op =[r 1op r 2op r 3op ]
[0064]
[0065] R′=[r1′r2′r1′×r2′]
[0066] t′=t op
[0067] Among them, [R op ,t op ] is the first local pose data, [R′,t′] is the second initial pose data, r 1op , r 2op , r 3op YesR op Three column vectors of .
[0068] In one embodiment, the preset iterative algorithm is a Lie algebraic perturbation model iterative method or an orthogonal iterative algorithm.
[0069] In the embodiment of the present invention, the iterative algorithm actually used can be set according to actual needs, and the embodiment of the present invention does not make any specific limitation on this.
[0070] In one embodiment, the Lie algebraic perturbation model iterative method calculates the local pose data according to the following formula:
[0071]
[0072]
[0073]
[0074]
[0075]
[0076]
[0077]
[0078] Among them, [R t ,t t ] is the pose data before updating, [R t+1 ,t t+1 ] is the updated pose data, Δξ is the pose increment, N represents the number of feature points, J i is the Jacobian matrix corresponding to the ith feature point, is the reprojection error vector corresponding to the ith feature point, (x i y i ) T is the pixel coordinate of the feature point, P i is the three-dimensional coordinate of the feature point in the world coordinate system, f x and f y is the focal length of the camera, c x and c y are the coordinates of the camera's principal point.
[0079] In an embodiment of the present invention, It is the pixel coordinate obtained by reprojecting the three-dimensional coordinates of the feature point in the world coordinate system through the camera pose, which is a function of R and t. It should be noted that
[0080] Δξ=(Δρ Δφ) T =(Δρ θa) T =(Δρ1 Δρ2 Δρ3 θa1 θa2 θa3) TIt is the Lie algebra representation of the pose increment, where Δρ represents the change in camera position, Δφ represents the change in camera pose, θ represents the modulus of Δφ, a1, a2, a3 represent the direction of Δφ, is a unit vector, Δρ1, Δρ2, Δρ3 represent the components of Δφ, and Δξ is used to describe the change in camera pose. ^ Represents the mapping of Δξ from vector to matrix,
[0081] In one embodiment, determining the camera pose measurement result according to the first local pose data and the second local pose data may be performed according to the following steps:
[0082] A first reprojection error is calculated according to the first local pose data; a second reprojection error is calculated according to the second local pose data; and a camera pose measurement result is determined according to the first reprojection error and the second reprojection error.
[0083] In the embodiment of the present invention, the first reprojection error and the second reprojection error are compared, and the local pose data with the smaller error is selected as the camera pose measurement result.
[0084] In one embodiment, the reprojection error is calculated according to the following formula based on the local pose data:
[0085]
[0086]
[0087] P i =(X i Y i Z i ) T
[0088]
[0089] Among them, E is the reprojection error, N is the number of feature points, (x i y i ) T is the pixel coordinate of the feature point, P i is the three-dimensional coordinate of the feature point in the world coordinate system, [R t ,t t ] is the local pose data, f x and f y is the focal length of the camera, c x and c y are the coordinates of the camera's principal point.
[0090] The execution process of the method is described below with reference to a specific embodiment.
[0091] 1. Convert the pixel coordinates of N feature points on the image into normalized coordinates. Specifically, the conversion formula is as follows:
[0092]
[0093] Where (x i y i ) T is the pixel coordinate of the feature point, (u i v i ) T is the normalized coordinate of the feature point, f x and f y is the focal length of the camera, c x and c y are the coordinates of the camera's principal point.
[0094] 2. Establish a world coordinate system so that the plane where the feature points are located coincides with the XOY plane of the world coordinate system, such as Figure 2 As shown. Using the coordinates of N feature points on the XOY plane (X i Y i ) T , i = 1, ... N, and the normalized coordinates of N feature points (u i v i ) T , i = 1, ... N, calculate the homography matrix H from the feature point plane to the image plane. The calculation method uses DLT (direct linear transformation method). The obtained H can be expressed as:
[0095]
[0096] 3. Take the first two columns of H to form the matrix H 1:2 =[h1 h2], and for H 1:2 Perform singular value decomposition and get:
[0097] make:
[0098] Right now
[0099]
[0100] The first initial pose solution [R, t] of the camera is calculated using the following formula:
[0101] R=[m1 m2 m1×m2], Where R represents the rotation matrix of the camera and t represents the translation vector of the camera.
[0102] 4. Take the first initial posture solution [R, t] as the starting point of iteration, and use the Lie algebra perturbation model iteration method or orthogonal iteration algorithm to obtain the first local optimal solution [Rop ,t op ].
[0103] Specifically, in the Lie algebra perturbation model iterative method, the reprojection error is defined as:
[0104]
[0105] Where N represents the number of feature points, (x i y i ) T is the pixel coordinate of the feature point, It is the pixel coordinate of the feature point in the world coordinate system after the three-dimensional coordinate is reprojected by the camera pose, that is: in P i =(X i Y i Z i ) T is the three-dimensional coordinate of the feature point in the world coordinate system, [R t ,t t ] is the pose of the current iteration. x and f y is the focal length of the camera, c x and c y are the coordinates of the camera's principal point.
[0106] The Lie algebra perturbation model iteration method takes the initial pose solution [R, t] as the starting point and iteratively updates the pose so that the reprojection error E reaches a local minimum. The pose increment in each iteration is calculated as:
[0107]
[0108] Where, Δξ=(Δρ Δφ) T =(Δρ θa) T =(Δρ1 Δρ2 Δρ3 θa1 θa2 θa3) T is the Lie algebraic representation of the pose increment.
[0109] is the Jacobian matrix corresponding to the i-th feature point, is the reprojection error vector corresponding to the i-th feature point.
[0110] After obtaining the pose increment Δξ, the pose update method is:
[0111]
[0112] [R t+1 ,t t+1 ] is the updated pose, which serves as the current pose for the next iteration.
[0113]
[0114] in The iterative process is repeated until the modulus of Δξ is less than a given threshold, and the pose of the current iteration is taken as the first local optimal solution [R op ,t op ].
[0115] 5. Mirror the first local optimal solution and use the flipped solution as the second initial pose solution. op =[r 1op r 2op r 3op ], where r 1op , r 2op , r 3op YesR op The mirror flip method is: calculate r separately 1op and r 2op In t op The component in the direction is negated, while keeping other components unchanged, to obtain r1′ and r2′, as Figure 3 The formula is:
[0116]
[0117] Then the solution after mirror flipping is R′=[r1′ r2′ r1′×r2′], t′=t op . Take [R′,t′] as the second initial pose solution.
[0118] 6. Taking the second initial posture solution [R′, t′] as the starting point of iteration, the Lie algebra perturbation model iteration method or the orthogonal iteration algorithm is used to obtain the second local optimal solution [R′ op ,t′ op ]. Specifically, the Lie algebra perturbation model iteration method is the same as step 4.
[0119] 7. For the first local optimal solution [R op ,t op ] and the second local optimal solution [R′ op ,t′ op ] to make a judgment, and take the solution that makes the reprojection error E smaller as the final pose measurement result. Specifically, [R op ,t op ] and [R′ op ,t′ op ]Substitute into the calculation formula of the reprojection error E in step 4:
[0120]
[0121] Replace [R t ,t t ], get the corresponding reprojection error, and then make a size judgment to select the final measurement result.
[0122] The present invention provides a camera pose measurement method and device based on coplanar feature points. The method first uses the homography matrix between the feature point plane and the image plane to decompose and obtain a first initial pose solution, and uses the first initial pose solution as a starting point to obtain a first local optimal pose by an iterative method. Then, the first local optimal pose is mirror-flipped to obtain a second initial pose solution, and the second local optimal pose is obtained by an iterative method using the second initial pose solution as a starting point. Finally, one of the two local optimal poses that makes the reprojection error smaller is selected as the global optimal pose. The method takes into account the ambiguity problem of pose measurement when the feature points are coplanar, and guides the iterative process through two initial solutions, so that the final camera pose measurement result is more stable and accurate.
[0123] The present invention also provides a camera pose measurement device based on coplanar feature points in an embodiment, as described in the following embodiment. Since the principle of the device to solve the problem is similar to that of the camera pose measurement method based on coplanar feature points, the implementation of the device can refer to the implementation of the camera pose measurement method based on coplanar feature points, and the repeated parts will not be repeated. Figure 4 The structure block diagram of a camera pose measurement device based on coplanar feature points is shown, and the device includes:
[0124] An acquisition module 41 is used to acquire the normalized coordinates of the coplanar feature points and the first coordinates of the coplanar feature points on the target plane of the world coordinate system; the plane where the coplanar feature points are located coincides with the target plane; a matrix module 42 is used to calculate the homography matrix based on the normalized coordinates and the first coordinates; a posture module 43 is used to calculate the first initial posture data of the camera using the homography matrix; an iteration module 44 is used to determine the first local posture data and the second local posture data based on the first initial posture data; a measurement module 45 is used to determine the camera posture measurement result based on the first local posture data and the second local posture data.
[0125] In one embodiment, see Figure 5 Another structural block diagram of a camera pose measurement device based on coplanar feature points is shown, and the device also includes: a preprocessing module 46, which is used to: normalize the pixel coordinates of the coplanar feature points on the image to obtain normalized coordinates; the image is obtained by taking pictures with a camera; a world coordinate system is established; and the plane where the coplanar feature points are located coincides with a plane of the world coordinate system.
[0126] In one embodiment, the preprocessing module is specifically used to: normalize the pixel coordinates of the coplanar feature points on the image according to the following formula to obtain normalized coordinates: i=1,…N;where (x i y i ) T is the pixel coordinate of the feature point, (u i v i ) T is the normalized coordinate of the feature point, f x and f y is the focal length of the camera, c x and c y is the principal point coordinate of the camera, and N is the number of coplanar feature points.
[0127] In one embodiment, the matrix module is specifically used to calculate the homography matrix from the feature point plane to the image plane using a direct linear transformation method.
[0128] In one embodiment, see Figure 6 As shown in the block diagram of the posture module structure, the posture module includes: a data unit 61, used to determine the first matrix and the second matrix according to the homography matrix; a decomposition unit 62, used to perform singular value decomposition on the first matrix to obtain a singular value decomposition result; a first determination unit 63, used to determine the rotation matrix of the camera using the singular value decomposition result; a second determination unit 64, used to determine the translation vector of the camera using the singular value decomposition result and the second matrix; a posture unit 65, used to use the rotation matrix and the translation vector as the first initial posture data of the camera.
[0129] In one embodiment, the data unit is specifically used to determine the first matrix and the second matrix according to the following formula based on the homography matrix: H 1:2 =[h1 h2]; where H is the homography matrix, H 1:2 is the first matrix and h3 is the second matrix.
[0130] In one embodiment, the decomposition unit is specifically used to perform singular value decomposition on the first matrix according to the following formula to obtain a singular value decomposition result: Among them, H 1:2 is the first matrix, U, S, V are the results of singular value decomposition.
[0131] In one embodiment, the first determining unit is specifically configured to: determine the rotation matrix of the camera using the singular value decomposition result according to the following formula: R = [m1 m2 m1×m2], where U and V are the results of singular value decomposition, and R is the rotation matrix of the camera.
[0132] In one embodiment, the second determining unit is specifically configured to determine the camera translation vector using the singular value decomposition result and the second matrix according to the following formula: Among them, h3 is the second matrix, s1 and s2 are the results of singular value decomposition, and t is the translation vector of the camera.
[0133] In one embodiment, see Figure 7 The iterative module structural block diagram shown in the figure, the iterative module includes: a first iterative unit 71, used to take the first initial posture data as the iteration starting point, and calculate the first local posture data using a preset iterative algorithm; a mirror flip unit 72, used to perform mirror flip processing on the first local posture data to obtain second initial posture data; a second iterative unit 73, used to take the second initial posture data as the iteration starting point, and calculate the second local posture data using a preset iterative algorithm.
[0134] In one embodiment, the mirror flip unit is specifically used to perform mirror flip processing on the first local pose data according to the following formula to obtain the second initial pose data: op =[r 1op r 2op r 3op ]; R′=[r1′ r2′ r1′×r2′]; t′=t op ; Among them, [R op ,t op ] is the first local pose data, [R′,t′] is the second initial pose data, r 1op , r 2op , r 3op YesR op Three column vectors of .
[0135] In one embodiment, the preset iterative algorithm is a Lie algebraic perturbation model iterative method or an orthogonal iterative algorithm.
[0136] In one embodiment, the Lie algebraic perturbation model iterative method calculates the local pose data according to the following formula:
[0137]
[0138]
[0139] Among them, [R t ,t t ] is the pose data before updating, [R t+1 ,t t+1 ] is the updated pose data, Δξ is the pose increment, N represents the number of feature points, J iis the Jacobian matrix corresponding to the ith feature point, is the reprojection error vector corresponding to the ith feature point, (x i y i ) T is the pixel coordinate of the feature point, P i is the three-dimensional coordinate of the feature point in the world coordinate system, f x and f y is the focal length of the camera, c x and c y are the coordinates of the camera's principal point.
[0140] In one embodiment, the measurement module is used to: calculate a first reprojection error based on first local pose data; calculate a second reprojection error based on second local pose data; and determine a camera pose measurement result based on the first reprojection error and the second reprojection error.
[0141] In one embodiment, the measurement module is specifically used to calculate the reprojection error according to the local pose data according to the following formula:
[0142]
[0143]
[0144] P i =(X i Y i Z i ) T ;
[0145]
[0146] Among them, E is the reprojection error, N is the number of feature points, (x i y i ) T is the pixel coordinate of the feature point, P i is the three-dimensional coordinate of the feature point in the world coordinate system, [R t ,t t ] is the local pose data, f x and f y is the focal length of the camera, c x and c y are the coordinates of the camera's principal point.
[0147] The embodiment of the present invention also provides a computer device, see Figure 8 The computer device structure schematic block diagram shown in the figure includes a memory 81, a processor 82 and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any of the above-mentioned camera pose measurement methods based on coplanar feature points are implemented.
[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the computer device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0149] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned camera pose measurement methods based on coplanar feature points.
[0150] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0154] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. A camera pose measurement method based on coplanar feature points, characterized in that: include: Acquire normalized coordinates of the coplanar feature points and first coordinates of the coplanar feature points on the target plane of the world coordinate system; The plane where the coplanar feature points are located coincides with the target plane; Calculating a homography matrix according to the normalized coordinates and the first coordinates; Calculating first initial pose data of the camera using the homography matrix; Determine first local pose data and second local pose data according to the first initial pose data; Determine a camera pose measurement result according to the first local pose data and the second local pose data; Calculating first initial pose data of the camera using the homography matrix includes: Determine a first matrix and a second matrix according to the homography matrix; Performing singular value decomposition on the first matrix to obtain a singular value decomposition result; Determine the rotation matrix of the camera using the singular value decomposition result; Determine a translation vector of the camera using the singular value decomposition result and the second matrix; Using the rotation matrix and the translation vector as first initial pose data of the camera; Determining first local pose data and second local pose data according to the first initial pose data includes: Using the first initial posture data as the iteration starting point, and calculating the first local posture data using a preset iterative algorithm; Performing mirror flipping processing on the first local pose data to obtain second initial pose data; Using the second initial pose data as an iteration starting point, and calculating the second local pose data using a preset iterative algorithm; Determining a camera pose measurement result according to the first local pose data and the second local pose data includes: Calculating a first reprojection error according to the first local pose data; Calculating a second reprojection error according to the second local pose data; A camera pose measurement result is determined according to the first reprojection error and the second reprojection error.
2. The method according to claim 1, characterized in that Before obtaining the normalized coordinates of the coplanar feature points and the first coordinates of the coplanar feature points on the target plane of the world coordinate system, the method further includes: Normalizing the pixel coordinates of the coplanar feature points on the image to obtain normalized coordinates; the image is obtained by photographing the camera; A world coordinate system is established; the plane where the coplanar feature points are located coincides with a plane of the world coordinate system.
3. The method according to claim 1, characterized in that Calculating a homography matrix according to the normalized coordinates and the first coordinates includes: The direct linear transformation method is used to calculate the homography matrix from the feature point plane to the image plane.
4. The method according to claim 1, characterized in that include: The first local pose data is mirror-flipped according to the following formula to obtain the second initial pose data: R op =[r 1op r 2op r 3op ] t′=t op Among them, [R op ,t op ] is the first local pose data, [R′,t′] is the second initial pose data, r 1op , r 2op , r 3op YesR op Three column vectors of .
5. A camera pose measurement device based on coplanar feature points, characterized in that: include: An acquisition module, used for acquiring normalized coordinates of coplanar feature points and first coordinates of the coplanar feature points on a target plane in a world coordinate system; The plane where the coplanar feature points are located coincides with the target plane; A matrix module, used for calculating a homography matrix according to the normalized coordinates and the first coordinates; A posture module, used for calculating first initial posture data of the camera using the homography matrix; An iteration module, configured to determine first local pose data and second local pose data according to the first initial pose data; A measurement module, configured to determine a camera pose measurement result according to the first local pose data and the second local pose data; The posture module includes: a data unit, used to determine a first matrix and a second matrix according to a homography matrix; a decomposition unit, used to perform singular value decomposition on the first matrix to obtain a singular value decomposition result; a first determination unit, used to determine a rotation matrix of the camera using the singular value decomposition result; a second determination unit, used to determine a translation vector of the camera using the singular value decomposition result and the second matrix; a posture unit, used to use the rotation matrix and the translation vector as first initial posture data of the camera; The iteration module includes: a first iteration unit, used to use the first initial posture data as the iteration starting point, and use a preset iteration algorithm to calculate the first local posture data; a mirror flip unit, used to perform mirror flip processing on the first local posture data to obtain second initial posture data; a second iteration unit, used to use the second initial posture data as the iteration starting point, and use a preset iteration algorithm to calculate the second local posture data; Measurement modules, specifically for: Calculating a first reprojection error according to the first local pose data; Calculating a second reprojection error according to the second local pose data; A camera pose measurement result is determined according to the first reprojection error and the second reprojection error.
6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the camera pose measurement method based on coplanar feature points described in any one of claims 1 to 4 is implemented.
7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the camera pose measurement method based on coplanar feature points according to any one of claims 1 to 4 is implemented.
