Camera Fast and Robust Calibration Method, Device and Equipment for Outdoor Visual Measurement

Abstract

The present invention discloses a method, device and equipment for rapid and robust calibration of a camera for outdoor visual measurement. The method includes: obtaining a pinhole camera model corresponding to the camera to be calibrated, and constructing an initial linear equation in combination with preset construction information; performing transformation on the initial linear equation based on a preset transformation strategy to obtain a homogeneous equation, and performing zero-point analysis processing on the homogeneous equation based on a zero-point analysis strategy to obtain a zero-point analysis result set; screening the zero-point analysis result set based on a screening strategy to obtain a screened zero-point analysis result and determining a current initial translation vector; and performing iterative optimization on the current initial translation vector based on an iterative optimization strategy to obtain an optimized translation vector for calibrating the camera. The embodiment of the present invention can start from the pinhole camera model, directly construct constraints without relying on intermediate variables, thereby improving the robustness to the point distribution; moreover, by adopting a null space analysis framework based on rotation matrix constraints, the camera to be calibrated can be calibrated more quickly and with high precision.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration in machine vision, and particularly to a method, device and equipment for rapid and robust camera calibration for outdoor vision measurement. Background Art

[0002] Due to the advantages of high precision, non-contact, dynamic measurement, etc. of vision measurement technology, image-based vision measurement technology is widely used in large-scale structure deformation measurement. The calibration of the measurement system is the most important link in the development of the measurement system, and the calibration efficiency and accuracy of the measurement system directly affect the measurement efficiency and accuracy of the vision measurement system. Therefore, accurate and rapid calibration of the measurement system parameters is very important.

[0003] In the patent with the application number CN202410310560.0, a single camera calibration method and system are disclosed. This prior art uses a standard plane grid image to first determine the distortion center, then initialize external parameters such as the focal length, and determine the camera parameters by combining model-based and non-model-based methods. This technology requires the use of a calibration plane grid, that is, it can only process coplanar points, and it is often difficult to construct a coplanar configuration for outdoor scene space points. The space points are often non-coplanar, and under long-distance observation conditions, the distribution of space points along the depth of field direction is much smaller than the observation distance, which belongs to a singular configuration. In this configuration, this method cannot achieve camera parameter calibration, and at the same time, this technology requires multiple images to achieve calibration.

[0004] In the patent with the application number CN202410277095.5, a camera calibration method and system for decoupling distortion are disclosed. This prior art uses a checkerboard plane target image to correct the central corner point according to the adjacent four corner points of the initial central corner point to obtain the reference corner point, generates ideal corner points according to the reference corner point combined with set constraints, and calibrates the camera parameters according to the corresponding relationship between the ideal corner points and the actual corner points. Similarly, in this method, camera parameter calibration cannot be achieved in a singular configuration, and at the same time, this technology requires multiple images to achieve calibration.

[0005] That is, in the calibration of large field-of-view cameras in the wild, it often occurs that it is difficult to meet the scene constraints and motion constraints required for self-calibration, the strong stereo conditions required for stereo calibration, or the absolute coplanar conditions required for plane target calibration due to the overly simple scene. The outdoor calibration conditions are often singular, such as the singular spatial distribution of a small number of space points, long distance or near plane, etc. Under these conditions, the existing methods often fail. Summary of the Invention

[0006] The embodiments of the present invention provide a method, device, and equipment for fast and robust calibration of a camera for outdoor visual measurement, aiming to solve the problem in the prior art that when calibrating a camera with a large field of view in the wild, the calibration conditions are often singular, such as a small number of spatial points, a long distance, or a nearly flat plane, resulting in the inability to achieve fast and accurate calibration of camera parameters.

[0007] In a first aspect, the embodiments of the present invention provide a method for fast and robust calibration of a camera for outdoor visual measurement, which includes:

[0008] Obtain a pinhole camera model corresponding to the camera to be calibrated, and correspondingly construct an initial linear equation based on the pinhole camera model and preset construction information;

[0009] Convert the initial linear equation based on a preset conversion strategy to obtain a homogeneous equation;

[0010] Perform zero-point analysis processing on the homogeneous equation based on a preset zero-point analysis strategy to obtain a zero-point analysis result set including several pairs of rotation matrix elements and focal lengths; wherein, each zero-point analysis result in the zero-point analysis result set corresponds to a pair of rotation matrix elements and a focal length;

[0011] Screen the zero-point analysis result set based on a preset screening strategy to obtain a screened zero-point analysis result;

[0012] Determine the current initial translation vector based on the screened zero-point analysis result;

[0013] Iteratively optimize the current initial translation vector based on a preset iterative optimization strategy to obtain an optimized translation vector, so that the camera to be calibrated is calibrated based on the optimized translation vector.

[0014] In a second aspect, the embodiments of the present invention further provide a device for fast and robust calibration of a camera for outdoor visual measurement, which includes:

[0015] An initial linear equation construction unit, configured to obtain a pinhole camera model corresponding to the camera to be calibrated, and correspondingly construct an initial linear equation based on the pinhole camera model and preset construction information;

[0016] A homogeneous equation obtaining unit, configured to convert the initial linear equation based on a preset conversion strategy to obtain a homogeneous equation;

[0017] A zero-point analysis processing unit, configured to perform zero-point analysis processing on the homogeneous equation based on a preset zero-point analysis strategy to obtain a zero-point analysis result set including several pairs of rotation matrix elements and focal lengths; wherein, each zero-point analysis result in the zero-point analysis result set corresponds to a pair of rotation matrix elements and a focal length;

[0018] An initial value screening unit, configured to screen the zero-point analysis result set based on a preset screening strategy to obtain a screened zero-point analysis result;

[0019] An initial value obtaining unit, configured to determine a current initial translation vector based on the screened zero-point analysis result;

[0020] An iterative optimization unit, configured to iteratively optimize the current initial translation vector based on a preset iterative optimization strategy to obtain an optimized translation vector, so that the camera to be calibrated is calibrated based on the optimized translation vector.

[0021] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor. A computer program is stored on the memory, and when the processor executes the computer program, the method described in the first aspect above is implemented.

[0022] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium. The computer storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the method described in the first aspect above can be implemented.

[0023] An embodiment of the present invention provides a method, device, and equipment for fast and robust calibration of a camera for outdoor vision measurement. The method includes: obtaining a pinhole camera model corresponding to the camera to be calibrated, and correspondingly constructing an initial linear equation based on the pinhole camera model and preset construction information; converting the initial linear equation based on a preset conversion strategy to obtain a homogeneous equation; performing zero-point analysis processing on the homogeneous equation based on a preset zero-point analysis strategy to obtain a zero-point analysis result set including several pairs of rotation matrix elements and focal lengths; where each zero-point analysis result in the zero-point analysis result set corresponds to a pair of rotation matrix elements and a focal length; screening the zero-point analysis result set based on a preset screening strategy to obtain a screened zero-point analysis result; determining a current initial translation vector based on the screened zero-point analysis result; iteratively optimizing the current initial translation vector based on a preset iterative optimization strategy to obtain an optimized translation vector, so that the camera to be calibrated is calibrated based on the optimized translation vector. The embodiment of the present invention can start from the pinhole camera model, directly construct constraints without relying on intermediate variables, thereby improving the robustness to the point distribution; and adopting a null space analysis framework based on rotation matrix constraints, it has high operation efficiency and can calibrate the camera to be calibrated more quickly and accurately. Description of the Drawings

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0025] Figure 1 Schematic diagram of the application scenario of the camera rapid and robust calibration method for outdoor vision measurement provided by the embodiment of the present invention;

[0026] Figure 2 Schematic diagram of the process of the camera rapid and robust calibration method for outdoor vision measurement provided by the embodiment of the present invention;

[0027] Figure 3 Schematic diagram of the sub - process of the camera rapid and robust calibration method for outdoor vision measurement provided by the embodiment of the present invention;

[0028] Figure 4 Schematic diagram of the sub - process of the camera rapid and robust calibration method for outdoor vision measurement provided by the embodiment of the present invention;

[0029] Figure 5 Schematic diagram of the sub - process of the camera rapid and robust calibration method for outdoor vision measurement provided by the embodiment of the present invention;

[0030] Figure 6 Schematic block diagram of the camera rapid and robust calibration device for outdoor vision measurement provided by the embodiment of the present invention;

[0031] Figure 7 Schematic block diagram of the computer device provided by the embodiment of the present invention. Detailed implementation manners

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0033] It should be understood that when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0034] It should also be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0035] It should be further understood that the term "and / or" used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] Please refer to Figure 1 and Figure 2 , Figure 1 which is a schematic diagram of the scene of the camera rapid and robust calibration method for outdoor vision measurement according to the embodiment of the present invention. Figure 2 FIG. is a schematic flow chart of the camera rapid and robust calibration method for outdoor vision measurement provided by the embodiment of the present invention. This method is applied to the server 10 or the user terminal 20, and the server 10 or the user terminal 20 is communicatively connected to the camera 30 to be calibrated. As Figure 2 shown, this method includes the following steps S110-S160.

[0037] S110. Obtain a pinhole camera model corresponding to the camera to be calibrated, and correspondingly construct an initial linear equation based on the pinhole camera model and preset construction information.

[0038] In this embodiment, generally, due to the limited processing capacity of the camera to be calibrated itself, the rapid calibration of the camera to be calibrated is not performed locally on the camera to be calibrated, but the technical solution is described with the server or the user terminal as the execution subject. The process of calibrating the camera to be calibrated is to calculate the internal parameters and external parameters of the camera in a given camera model. When calibrating the camera to be calibrated, the server or the user terminal needs to first obtain a pinhole camera model corresponding to the camera to be calibrated, and then correspondingly construct an initial linear equation based on the pinhole camera model and preset construction information. Among them, specifically in implementation, the camera to be calibrated is a camera for outdoor vision measurement. For example, the camera takes images of large civil structures for further health monitoring of large civil structures.

[0039] In one embodiment, the preset construction information includes a rotation matrix, a translation vector, and a focal length;

[0040] In the corresponding construction of the initial linear equation based on the pinhole camera model and the preset construction information, the pinhole camera model is:

[0041] (1)

[0042] Among them, in Equation (1) corresponding to the initial linear equation, E represents the rotation matrix from the world coordinate system to the camera coordinate system of the camera to be calibrated, tt represents the translation vector from the world coordinate system to the camera coordinate system of the camera to be calibrated, represents the distance of the i-th point in the z-axis direction of the camera coordinate system, f represents the focal length of the camera to be calibrated, u c represents the coordinate value of the origin of the image physical coordinate system corresponding to the camera coordinate system on the u-axis of the image pixel coordinate system corresponding to the camera coordinate system, v c represents the coordinate value of the origin of the image physical coordinate system corresponding to the camera coordinate system on the v-axis of the image pixel coordinate system corresponding to the camera coordinate system, px i represents the coordinate of the i-th point in the image physical coordinate system corresponding to the camera coordinate system, PY i represents the coordinate of the i-th point in the world coordinate system corresponding to the camera coordinate system, and represents the first orientation vector to be calibrated in the rotation matrix E, and represents the second orientation vector to be calibrated in the rotation matrix E, and represents the third orientation vector to be calibrated in the rotation matrix E, r1 - r9 are values determined by the installation and configuration of the camera to be calibrated, and t1 - t3 are values determined by the calibration process of the camera to be calibrated.

[0043] PY i and px i can be regarded as 3D - 2D points, or can be understood as 3D - 2D points in the PnPf problem (the PnPf problem is also the PNP problem, which refers to calculating the pose of the camera under the premise of knowing the camera internal parameters through N pairs of matching image coordinates p i and their world coordinates P i where N is a positive integer greater than 2). By solving the PnPf problem, it can be used to estimate the rotation matrix E, the translation vector tt, and the focal length f. In the pinhole camera model as shown in Equation (1) above, the rotation matrix E is a 3*3 matrix, that is, it includes 9 parameters r1 - r9 and their respective values depend on the installation and configuration of the camera to be calibrated. Similarly, the translation vector tt includes 3 parameters t1 - t3 and their respective values are determined by the calibration process of the camera to be calibrated.

[0044] When constructing the initial linear equation based on the pinhole camera model and the preset construction information, the expression of can be obtained by using the third - row equation of Equation (1), and substituting it into the first two - row equations can eliminate , thus obtaining the following Equation (2):

[0045] (2)

[0046] Among them, ui represents the coordinate value of the i-th point on the u-axis in the image pixel coordinate system corresponding to the camera coordinate system, v i Represents the coordinate value of the i-th point on the v-axis in the image pixel coordinate system corresponding to the camera coordinate system.

[0047] Then divide both sides of the equation in equation (2) by f to obtain the following equation (3):

[0048] (3)

[0049] Then, define the two variables in the following formula (4) and :

[0050] (4)

[0051] in, , , substituting the two variables in formula (4) into formula (3), it can be converted into the following formula (5):

[0052] (5)

[0053] In formula (5), it is the constraint for the i-th pair of 3D-2D points, W i yes The coefficient matrix, V i yes The coefficient matrix of , when N pairs of 3D-2D point constraints are combined, the initial linear equation is obtained as follows (6):

[0054] (6)

[0055] In formula (6) and , and N is the total number of pairs of 3D-2D points, and W1 to W N Composed with The corresponding complete coefficient matrix W, V1 to V N Composed with The corresponding complete coefficient matrix V.

[0056] As can be seen, the above method constructs the initial linear equations without relying on constraints on intermediate variables. Instead, it directly constructs the initial linear equations for the rotation matrix elements and focal length based on the pinhole camera model. Furthermore, the resulting initial linear equations can be further processed and transformed, for example, to obtain the corresponding homogeneous equations.

[0057] S120: Convert the initial linear equation based on a preset conversion strategy to obtain a homogeneous equation.

[0058] In this embodiment, the obtained initial linear equation is not yet a homogeneous equation, and its solution space does not have a good linear structure, which is not convenient for solving. At this time, it can be linearly transformed based on a preset transformation strategy to obtain a homogeneous equation. Then, combined with the fact that the solution space of the homogeneous equation has a good linear structure, it can be solved more quickly.

[0059] In one embodiment, as Figure 3 shown, step S120 includes:

[0060] S121. Obtain a linear least squares criterion corresponding to the preset transformation strategy;

[0061] S122. Transform the initial linear equation based on the linear least squares criterion, and make the translation vector related to the rotation matrix elements and the focal length to obtain the homogeneous equation.

[0062] In this embodiment, the specifically adopted preset transformation strategy is the linear least squares criterion. When using the linear least squares criterion to transform the initial linear equation, since the initial linear equation is related to the rotation matrix elements and the focal length. Moreover, the rotation matrix constraint related to the rotation matrix elements is different from the method of using intermediate variables (collecting images by the camera to be calibrated first and determining multiple control points as intermediate variables) and partial constraints in the prior art, but directly based on the rotation matrix elements and making full use of the rotation matrix itself constraints for solving. The rotation matrix constraint is the orthogonality constraint of the rotation matrix, that is, any row and any column are unit vectors, and any two rows and any two columns are orthogonal.

[0063] For example, still referring to the example corresponding to the above formulas (1)-(6), the in formula (6) can be represented by the pseudoinverse, as shown in the following formula (7):

[0064] (7)

[0065] Substituting formula (7) back into formula (6) can obtain a homogeneous equation related only to , and the homogeneous equation can be specifically seen in the following formula (8):

[0066] (8)

[0067] Among them, in formula (8), K is a homogeneous coefficient matrix determined by the complete coefficient matrix W corresponding to and the complete coefficient matrix V corresponding to . After obtaining the homogeneous equation as in formula (8), its solution space has a good linear structure, which is convenient for more rapid solution.

[0068] S130. Perform zero - point analysis processing on the homogeneous equation based on a preset zero - point analysis strategy to obtain a zero - point analysis result set including several pairs of rotation matrix elements and focal lengths.

[0069] Among them, each zero - point analysis result in the zero - point analysis result set corresponds to a pair of rotation matrix elements and a focal length.

[0070] In this embodiment, when solving the homogeneous equation, a zero - point analysis strategy, that is, a null - space analysis method, can be adopted. Null - space analysis is a method for solving homogeneous linear equations. Specifically, the solution vector is expressed as a linear combination of null - space vectors. After specifically adopting the zero - point analysis strategy to perform zero - point analysis processing on the homogeneous equation, what is finally obtained is a zero - point analysis result set including several zero - point analysis results, and each zero - point analysis result corresponds to a pair of rotation matrix elements and a focal length. When performing zero - point analysis processing on the homogeneous equation based on the zero - point analysis strategy, it is also necessary to specifically consider the coplanarity of multiple control points corresponding to the camera to be calibrated. The specific processing process will be described in detail in the subsequent process.

[0071] In one embodiment, as Figure 4 shown, step S130 includes:

[0072] S131. Obtain multiple control points corresponding to the camera to be calibrated, and control - point coordinates corresponding to each control point respectively;

[0073] S132. If it is determined that the control - point coordinates corresponding to the multiple control points are not coplanar, then based on the ordinary configuration strategy in the zero - point analysis strategy and the orthogonality constraint of the rotation matrix, solve the null - space coefficients of the homogeneous equation to obtain the first null - space coefficient, and determine the zero - point analysis result set with the first null - space coefficient.

[0074] In this embodiment, if it is determined that the control - point coordinates corresponding to the multiple control points are not coplanar, it corresponds to the ordinary - point configuration situation. At this time, the null - space coefficients of the homogeneous equation can be solved based on the ordinary configuration strategy in the zero - point analysis strategy and the orthogonality constraint of the rotation matrix, so as to obtain the first null - space coefficient.

[0075] Among them, in the ordinary - point configuration situation, for the homogeneous linear equation in Equation (8), at least four cases of null - space m = 1, 2, 3, and 4 need to be considered, which can cover the situation where the total number of pairs of 3D - 2D points N≥4. , dim() is a function used to determine the dimension of an object, and rank() is a function for finding the rank, thus completely covering the minimum point configuration in the case of ordinary point configuration (i.e., it can be solved in the case of 4 points) and overdetermined configuration (i.e., it can also be solved in the case of more than 4 points). It can be seen that in the case of ordinary point configuration, null space analysis can directly obtain several first null space coefficients including the focal length and rotation matrix, and use the several first null space coefficients as the zero point analysis result set.

[0076] In one embodiment, as Figure 4 shown, after step S131, it further includes:

[0077] S133. If it is determined that the control point coordinates corresponding to multiple control points are coplanar, then based on the coplanar configuration strategy in the zero point analysis strategy and the orthogonality constraint of the rotation matrix, solve the null space coefficients of the homogeneous equation to obtain the second null space coefficients, and use the second null space coefficients to determine the zero point analysis result set.

[0078] In this embodiment, if it is determined that the control point coordinates corresponding to multiple control points are coplanar, then it corresponds to the case of planar point configuration. At this time, based on the coplanar configuration strategy in the zero point analysis strategy and the orthogonality constraint of the rotation matrix, the null space coefficients of the homogeneous equation can be solved to obtain the second null space coefficients.

[0079] Among them, in the case of planar point configuration, the null space method only outputs the solutions of the focal length and the first two columns c1 and c2 of the rotation matrix, and the third column still needs to be recovered by cross product . Due to the influence of noise, the output rotation matrix does not strictly satisfy the orthogonality constraint of the rotation matrix. According to Frobenius theory (i.e., the manifold topology theory proposed by Frobenius), the SVD decomposition (i.e., singular value decomposition) can be used to obtain the closest strict rotation matrix.

[0080] Moreover, in the case of planar point configuration, the homogeneous equation corresponding to equation (8) degenerates. A common preprocessing method in the case of planar point configuration is to perform a pre-(rigid) transformation on the world coordinate system so that the Z-axis of the new world coordinate system (i.e., the image physical coordinate system) is perpendicular to the plane where the 3D points (which can also be understood as control points) are located. Subsequently, solve based on the new world coordinate system, and finally compensate the pre-(rigid) transformation back to the solution result to obtain the true rotation matrix and translation vector.

[0081] Obviously, in the new world coordinate system, the 3D point coordinates are , after substituting it into equation (1), the third column elements of the rotation matrix E are eliminated. Therefore, in the case of planar point configuration, the vector remains unchanged, while the vector becomes a 6D vector and can be represented by the following equation (9):

[0082] (9)

[0083] Similarly, in the planar point configuration (5), the dimension of W i remains unchanged, while V i changes to the 2×6 matrix of the following formula (10):

[0084] (10)

[0085] The dimension of W in formula (6) remains unchanged, while V becomes a matrix. Similarly, for the 2N×6 under the planar point configuration, the homogeneous equation related only to can be obtained using the pseudoinverse and has the same form as formula (8), but and V therein need to adopt the forms provided in formula (9) and formula (10); among them, the 3D point coordinates PY i (i.e., ) in the new world coordinate system and x i in formula (10) i represent the abscissa of the 3D point corresponding to PY i in the image physical coordinate system, and y i represents the ordinate of the 3D point corresponding to PY

[0086] As shown in formula (9), under the planar point configuration contains 6 variables. At this time, two cases of m = 1 and m = 2 are considered respectively. This makes the solution method of the planar point configuration cover the case of N >= 4 (theoretically, when there are 4 corresponding points , additionally considering m = 2 can improve the noise robustness of the method). The above solution method in the case of the planar point configuration belongs to both the overdetermined configuration method and can solve the minimum point number configuration.

[0087] S140. Screen the zero-point analysis result set based on a preset screening strategy to obtain a screened zero-point analysis result.

[0088] In this embodiment, if the zero-point analysis result set is denoted as the output value , perform SVD decomposition on it (where each column vector in U is 's eigenvector, D represents the diagonal matrix composed of multiple eigenvalues of , and V in is the complete coefficient matrix V corresponding to ), then the rotation matrix closest to and strictly satisfying the rotation matrix orthogonality constraint is . In addition, there is An invalid solution (corresponding to a left-handed coordinate system) of (where det() represents a function for obtaining the determinant value of a matrix), and the invalid solution needs to be removed. After removing the invalid solutions in the zero-point analysis result set, the solution with the minimum reprojection error is selected as the initial value for subsequent iterative optimization.

[0089] In one embodiment, as Figure 5 shown, step S140 includes:

[0090] S141. Based on the screening strategy, for each rotation matrix corresponding to a zero-point analysis result in the zero-point analysis result set, delete the zero-point analysis result corresponding to the left-handed coordinate system of the rotation matrix from the zero-point analysis result set to obtain the zero-point analysis result set after the first-round screening;

[0091] S142. Obtain the reprojection error of each zero-point analysis result in the zero-point analysis result set after the first-round screening, and use the zero-point analysis result with the minimum reprojection error as the screened zero-point analysis result.

[0092] In this embodiment, specifically, the rotation matrix corresponding to each zero-point analysis result in the zero-point analysis result set by the screening strategy can be denoted as E. If the determinant value corresponding to the rotation matrix E of a zero-point analysis result is -1 (i.e., ), it means that it corresponds to a left-handed coordinate system rather than a right-handed coordinate system, and can be regarded as an invalid solution and deleted from the zero-point analysis result set. After all the zero-point analysis results with the determinant value of -1 corresponding to the rotation matrix E in the zero-point analysis result set are deleted from the zero-point analysis result set, the zero-point analysis result set after the first-round screening can be obtained.

[0093] Moreover, after determining the reprojection error of each zero-point analysis result in the first-round screened zero-point analysis result set based on the reprojection solution method, the zero-point analysis result with the minimum reprojection error can be selected as the screened zero-point analysis result. The reprojection error refers to the difference between pixel points calculated through two projection processes in computer vision. The first projection is to project three-dimensional space points onto an image, and the second projection is to re-project using the calculated camera pose and three-dimensional point coordinates, and the reprojection error can be determined by calculating the difference between the two projections. It can be seen that the above method essentially performs singular value decomposition on the coefficient matrix of each zero-point analysis result, and takes the right singular vector corresponding to the zero singular value as the null space vector. In the presence of noise caused by actual data, the singular value perturbation brought by the noise may cause the singular value of the correct right singular vector to be greater than that of the incorrect right singular vector, resulting in the incorrect right singular vector being selected as the null space vector. In the processing method adopted in this application, the restriction on the null space dimension is cancelled, and instead, various value cases are classified and discussed, so as to cover more null space situations, reduce the possibility of selecting the wrong null space vector, and improve the noise robustness of the method. Different from the existing method that solves based on intermediate variables (control point coordinates) and partial constraints (distance between control points), the processing method adopted in this application directly based on the rotation matrix elements and fully utilizes the self-constraint of the rotation matrix (i.e., the orthogonality constraint of the rotation matrix) to solve.

[0094] S150. Determine the current initial translation vector based on the screened zero-point analysis result.

[0095] In this embodiment, since the zero-point analysis result with the determinant value of 1 corresponding to the rotation matrix E and the minimum reprojection error is screened out from the zero-point analysis result set, the above-screened zero-point analysis result can be used to recover the translation vector, that is, first determine the current initial translation vector with the rotation matrix elements and focal length in the screened zero-point analysis result.

[0096] S160. Iteratively optimize the current initial translation vector based on a preset iterative optimization strategy to obtain an optimized translation vector, so that the camera to be calibrated is calibrated based on the optimized translation vector.

[0097] In this embodiment, the obtained current initial translation vector may not necessarily be used as the optimal camera calibration parameter, and the current initial translation vector can be further iteratively optimized based on the iterative optimization strategy to obtain an optimized translation vector, and finally the optimized translation vector is used as the optimal camera calibration parameter to calibrate the camera to be calibrated.

[0098] In one embodiment, step S160 includes:

[0099] Obtain the damped Newton iteration optimization strategy corresponding to the iterative optimization strategy, and perform iterative optimization on the current initial translation vector based on the damped Newton iteration optimization strategy until the iterative optimization stops when the least-squares error function of the current translation vector corresponding to the current initial translation vector has a minimum value, and obtain the current translation vector as the optimized translation vector.

[0100] In this embodiment, if the rotation matrix E corresponding to the current initial translation vector is represented by the following formula (11):

[0101] (11)

[0102] Among them, in formula (11), a, b, c, and d are the 4 elements in a preset quaternion. For example, a quaternion includes a real part and three imaginary parts, then a can be the real part, and b, c, and d each correspond to an imaginary part.

[0103] Construct the cost equation according to formula (8) as the following formula (12):

[0104] (12)

[0105] Among them, in formula (12), f still represents the focal length of the camera to be calibrated. Then use the damped Newton iteration optimization strategy to perform iterative optimization on the current initial translation vector. Let the value at the k-th time be , then it is updated at the (k + 1)-th iteration as:

[0106] (13)

[0107] In the above formula (13), λ is the step size, I5 is a 5-dimensional identity matrix, H(m k ) and g(m k ) are the Hessian matrix and gradient vector of the cost equation with respect to m k , respectively. Since the initial value accuracy of the current initial translation vector is high, usually only a few steps of iteration are required to reach the optimal accuracy, so as to obtain the current translation vector as the optimized translation vector, and finally use the optimized translation vector as the optimal camera calibration parameter to calibrate the camera to be calibrated.

[0108] It can be seen that the embodiments implementing this method can start from the pinhole camera model, directly construct constraints without relying on intermediate variables, thereby improving the robustness to the point distribution; and adopting a null space analysis framework based on rotation matrix constraints, it has high operation efficiency and can calibrate the camera to be calibrated more quickly and accurately.

[0109] Figure 6 is a schematic block diagram of a camera fast and robust calibration device for outdoor vision measurement provided by an embodiment of the present invention. AsFigure 6 As shown in Figure 6 , corresponding to the above-mentioned fast and robust calibration method of the camera for outdoor vision measurement, the present invention also provides a fast and robust calibration device 100 for the camera for outdoor vision measurement. The fast and robust calibration device for the camera for outdoor vision measurement includes: an initial linear equation construction unit 110, a homogeneous equation acquisition unit 120, a zero-point analysis and processing unit 130, an initial value screening unit 140, an initial value acquisition unit 150, and an iterative optimization unit 160.

[0110] The initial linear equation construction unit 110 is configured to obtain a pinhole camera model corresponding to the camera to be calibrated, and construct an initial linear equation based on the pinhole camera model and preset construction information.

[0111] In this embodiment, generally, due to the limited processing capacity of the camera to be calibrated, the fast calibration of the camera to be calibrated is not performed locally on the camera to be calibrated, but the technical solution is described with the server or the user terminal as the execution subject. The process of calibrating the camera to be calibrated is to calculate the internal parameters and external parameters of the camera in a given camera model. When calibrating the camera to be calibrated, the server or the user terminal needs to first obtain the pinhole camera model corresponding to the camera to be calibrated, and then construct an initial linear equation based on the pinhole camera model and preset construction information. Specifically, when implemented, the camera to be calibrated is a camera for outdoor vision measurement. For example, the camera is used to capture images of large civil structures for further health monitoring of large civil structures.

[0112] In one embodiment, the preset construction information includes a rotation matrix, a translation vector, and a focal length;

[0113] In the construction of the initial linear equation based on the pinhole camera model and preset construction information, the pinhole camera model is as shown in Equation (1); where, in Equation (1) corresponding to the initial linear equation, E represents the rotation matrix from the world coordinate system to the camera coordinate system of the camera to be calibrated, tt represents the translation vector from the world coordinate system to the camera coordinate system of the camera to be calibrated, λ i represents the distance of the i-th point in the z-axis direction of the camera coordinate system, f represents the focal length of the camera to be calibrated, u c represents the coordinate value of the origin of the image physical coordinate system corresponding to the camera coordinate system on the u-axis of the image pixel coordinate system corresponding to the camera coordinate system, v c represents the coordinate value of the origin of the image physical coordinate system corresponding to the camera coordinate system on the v-axis of the image pixel coordinate system corresponding to the camera coordinate system, px i represents the coordinate of the i-th point in the image physical coordinate system corresponding to the camera coordinate system, PY i represents the coordinate of the i-th point in the world coordinate system corresponding to the camera coordinate system. And represents the first vector to be calibrated in the rotation matrix E, And represents the second vector to be calibrated in the rotation matrix E, And represents the third vector to be calibrated in the rotation matrix E, r1-r9 are values determined by the installation and configuration of the camera to be calibrated, and t1-t3 are values determined by the calibration process of the camera to be calibrated.

[0114] PY i and px i It can be regarded as a 3D-2D point, or it can be understood as a PnPf problem (the PnPf problem is also called the PNP problem, which means that under the premise of knowing the camera internal parameters, N pairs of matching image coordinates p are matched. i and their world coordinates P i Calculate the camera pose, N is a positive integer greater than 2) in the 3D-2D point, and the principle diagram corresponding to the PnPf problem can be referred to Figure 3 By solving the PnPf problem, we can estimate the rotation matrix E, translation vector tt, and focal length f. In the pinhole camera model of equation (1), the rotation matrix E is a 3*3 matrix, which includes 9 parameters r1-r9, and the values of each are determined by the installation and configuration of the camera to be calibrated. Similarly, the translation vector tt includes 3 parameters t1-t3, and the values of each are determined by the calibration process of the camera to be calibrated.

[0115] When constructing the initial linear equation based on the pinhole camera model and the preset construction information, λ can be obtained using the third line of equation (1): i Substituting the expression into the first two lines of equations eliminates λ i , thus obtaining the above formula (2).

[0116] Then divide both sides of the equation in equation (2) by f to get equation (3). Then, define the two variables in equation (4) and ;in, , , Substituting the two variables in Equation (4) into Equation (3), it can be converted into Equation (5) above. In Equation (5), it is the constraint for the i-th pair of 3D-2D points, W i yes The coefficient matrix, V i yes The coefficient matrix of , when N pairs of 3D-2D point constraints are combined, the initial linear equation as shown in formula (6) is obtained. In formula (6) and , and N is the total number of pairs of 3D-2D points, and W1 to W N Composed with The corresponding complete coefficient matrices W, V1 to V N constitute the complete coefficient matrix V corresponding to the corresponding complete coefficient matrix V.

[0117] It can be seen that when constructing the initial linear equation in the above manner, without relying on the constraints of intermediate variables, it directly starts from the pinhole camera model and directly constructs the initial linear equation regarding the elements of the rotation matrix and the focal length. Moreover, the obtained initial linear equation can be further processed and transformed, for example, to obtain the corresponding homogeneous equation.

[0118] The homogeneous equation obtaining unit 120 is used to transform the initial linear equation based on a preset transformation strategy to obtain a homogeneous equation.

[0119] In this embodiment, the obtained initial linear equation is not yet a homogeneous equation, and its solution space does not have a good linear structure, which is not convenient for solving. At this time, it can be linearly transformed based on a preset transformation strategy to obtain a homogeneous equation. Then, combined with the fact that the solution space of the homogeneous equation has a good linear structure, it can be solved more quickly.

[0120] In one embodiment, the homogeneous equation obtaining unit 120 is specifically used for:

[0121] Obtain the linear least squares criterion corresponding to the preset transformation strategy;

[0122] Based on the linear least squares criterion, transform the initial linear equation and make the translation vector related to the elements of the rotation matrix and the focal length to obtain the homogeneous equation.

[0123] In this embodiment, the specifically adopted preset transformation strategy is the linear least squares criterion. When using the linear least squares criterion to transform the initial linear equation, since the initial linear equation is related to the elements of the rotation matrix and the focal length. Moreover, the rotation matrix constraint related to the elements of the rotation matrix is different from the way of using intermediate variables (collecting images by the camera to be calibrated first and determining multiple control points as intermediate variables) and partial constraints in the prior art for solving, but directly based on the elements of the rotation matrix and making full use of the self-constraint of the rotation matrix for solving. The rotation matrix constraint is the orthogonality constraint of the rotation matrix, that is, any row and any column are unit vectors, and any two rows and any two columns are orthogonal.

[0124] For example, still referring to the example corresponding to the above formulas (1)-(6), the in formula (6) can be represented by the pseudo-inverse, specifically as the above formula (7); substituting formula (7) back into formula (6) can obtain only related to The related homogeneous equation can be specifically seen in the above formula (8). After obtaining the homogeneous equation in formula (8), its solution space has a good linear structure, which is convenient for faster solution.

[0125] The zero-point analysis processing unit 130 is used to perform zero-point analysis processing on the homogeneous equation based on a preset zero-point analysis strategy, and obtain a zero-point analysis result set including several pairs of rotation matrix elements and focal lengths.

[0126] Among them, each zero-point analysis result in the zero-point analysis result set corresponds to a pair of rotation matrix elements and focal lengths.

[0127] In this embodiment, when solving the homogeneous equation, a zero-point analysis strategy, that is, a null space analysis method, can be adopted. Null space analysis is a method for solving homogeneous linear equations. Specifically, the solution vector is expressed as a linear combination form of null space vectors. After specifically adopting the zero-point analysis strategy to perform zero-point analysis processing on the homogeneous equation, what is finally obtained is a zero-point analysis result set including several zero-point analysis results, and each zero-point analysis result corresponds to a pair of rotation matrix elements and focal lengths. When performing zero-point analysis processing on the homogeneous equation based on the zero-point analysis strategy, it is also necessary to specifically consider the coplanarity of multiple control points corresponding to the camera to be calibrated. The specific processing process will be described in detail in the subsequent process.

[0128] In one embodiment, the zero-point analysis processing unit 130 is specifically used for:

[0129] Obtain multiple control points corresponding to the camera to be calibrated, and control point coordinates respectively corresponding to each control point;

[0130] If it is determined that the control point coordinates corresponding to the multiple control points are not coplanar, then based on the ordinary configuration strategy and rotation matrix orthogonality constraint in the zero-point analysis strategy, perform null space coefficient solution on the homogeneous equation to obtain a first null space coefficient, and determine the zero-point analysis result set with the first null space coefficient.

[0131] In this embodiment, if it is determined that the control point coordinates corresponding to the multiple control points are not coplanar, it corresponds to the ordinary point configuration situation. At this time, based on the ordinary configuration strategy and rotation matrix orthogonality constraint in the zero-point analysis strategy, null space coefficient solution can be performed on the homogeneous equation to obtain a first null space coefficient.

[0132] Among them, in the case of ordinary point configuration, the homogeneous linear equation in formula (8) needs to consider at least four cases of null space m = 1, 2, 3, and 4, which can cover the case where the total number of pairs of 3D-2D points N≥4 , dim() is a function used to determine the dimension of an object, and rank() is a function for finding the rank), thus completely covering the minimum point configuration and overdetermined configuration in the case of ordinary point configuration. It can be seen that in the case of ordinary point configuration, null space analysis can directly obtain a number of first null space coefficients including the focal length and rotation matrix, and use the number of first null space coefficients as the zero point analysis result set.

[0133] In one embodiment, the zero point analysis processing unit 130 is further specifically configured to:

[0134] If it is determined that the control point coordinates corresponding to multiple control points are coplanar, then based on the coplanar configuration strategy in the zero point analysis strategy and the orthogonality constraint of the rotation matrix, the null space coefficients of the homogeneous equation are solved to obtain the second null space coefficients, and the zero point analysis result set is determined by the second null space coefficients.

[0135] In this embodiment, if it is determined that the control point coordinates corresponding to multiple control points are coplanar, then corresponding to the case of planar point configuration, at this time, the null space coefficients of the homogeneous equation can be solved based on the coplanar configuration strategy in the zero point analysis strategy and the orthogonality constraint of the rotation matrix to obtain the second null space coefficients.

[0136] Among them, in the case of planar point configuration, the null space method only outputs the solutions of the first two columns c1 and c2 of the focal length and rotation matrix, and the third column needs to be recovered by cross product. . Due to the influence of noise, the output rotation matrix does not strictly satisfy the orthogonality constraint of the rotation matrix. According to the Frobenius theory (i.e., the manifold topology theory proposed by Frobenius), the SVD decomposition (i.e., singular value decomposition) can be used to obtain the closest strict rotation matrix.

[0137] Moreover, in the case of planar point configuration, the homogeneous equation corresponding to Equation (8) degenerates. A common preprocessing method in the case of planar point configuration is to perform a pre-(rigid) transformation on the world coordinate system so that the Z-axis of the new world coordinate system is perpendicular to the plane where the 3D points are located. Subsequently, the solution is obtained based on the new world coordinate system, and finally the pre-(rigid) transformation is compensated back into the solution result to obtain the true rotation matrix and translation vector.

[0138] Obviously, in the new world coordinate system, the 3D point coordinates are , after substituting it into Equation (1), the elements of the third column of the rotation matrix E are eliminated. Therefore, in the case of planar point configuration, the vector remains unchanged, while the vector becomes a 6D vector and can be expressed as the above Equation (9). Similarly, in the planar point configuration Equation (5), the dimension of W i remains unchanged, while the dimension of V iIt becomes a 2×6 matrix as shown in the above formula (10). The dimension of W in formula (6) remains unchanged, while V becomes a matrix. In the planar point configuration, 2N×6 can also use the pseudo-inverse to obtain a homogeneous equation related only to and has the same form as formula (8), but and V in it need to adopt the forms provided in formula (9) and formula (10).

[0139] As shown in formula (9), in the planar point configuration contains 6 variables. At this time, two cases of m = 1 and m = 2 are considered respectively. This makes the solution method for the planar point configuration cover the case of N >= 4 (theoretically, when there are 4 corresponding points , additionally considering m = 2 can improve the noise robustness of the method). The above solution method in the planar point configuration case belongs to both the overdetermined configuration method and can solve the minimum point number configuration.

[0140] The initial value screening unit 140 is used to screen the zero-point analysis result set based on a preset screening strategy to obtain the screened zero-point analysis result.

[0141] In this embodiment, if the zero-point analysis result set is denoted as the output value , perform SVD decomposition on it (where each column vector in U is 's eigenvector, D represents the diagonal matrix composed of multiple eigenvalues of , and V in is the complete coefficient matrix V corresponding to ), then the rotation matrix closest to and strictly satisfying the rotation matrix orthogonality constraint is . In addition, there are (det() represents the function to obtain the value of the matrix determinant) invalid solutions in the output E (corresponding to the left-handed coordinate system), and the invalid solutions need to be removed. After removing the invalid solutions in the zero-point analysis result set, select the solution with the smallest reprojection error as the initial value for subsequent iterative optimization.

[0142] In one embodiment, the initial value screening unit 140 is specifically used for:

[0143] Based on the screening strategy, for each rotation matrix corresponding to the zero-point analysis result in the zero-point analysis result set, and delete the zero-point analysis result corresponding to the left-handed coordinate system of the rotation matrix from the zero-point analysis result set to obtain the first-round screened zero-point analysis result set;

[0144] Obtain the reprojection error of each zero-point analysis result in the first-round screened zero-point analysis result set, and use the zero-point analysis result with the smallest reprojection error as the screened zero-point analysis result.

[0145] In this embodiment, specifically, the rotation matrix corresponding to each zero-point analysis result in the zero-point analysis result set for the screening strategy is denoted as E. If the determinant value corresponding to the rotation matrix E of a zero-point analysis result is -1 (i.e., ), it indicates that it corresponds to a left-handed coordinate system rather than a right-handed coordinate system, and can be regarded as an invalid solution and deleted from the zero-point analysis result set. After all the zero-point analysis results with the determinant value of -1 corresponding to the rotation matrix E in the zero-point analysis result set are deleted from the zero-point analysis result set, the zero-point analysis result set after the first-round screening can be obtained.

[0146] Moreover, after determining the reprojection error of each zero-point analysis result in the zero-point analysis result set after the first-round screening based on the reprojection solution method, the zero-point analysis result with the minimum reprojection error can be selected as the screened zero-point analysis result. The reprojection error refers to the difference between pixel points calculated through two projection processes in computer vision. The first projection projects a three-dimensional space point onto an image, and the second projection re-projects using the calculated camera pose and three-dimensional point coordinates, and the reprojection error can be determined by calculating the difference between the two projections. It can be seen that the above method essentially performs singular value decomposition on the coefficient matrix of each zero-point analysis result, and takes the right singular vector corresponding to the zero singular value as the null space vector. In the presence of noise in actual data, the singular value perturbation caused by the noise may cause the singular value of the correct right singular vector to be greater than that of the incorrect right singular vector, resulting in the incorrect right singular vector being selected as the null space vector. In the processing method adopted in this application, the restriction on the dimension of the null space is cancelled, but various value cases are classified and discussed, so as to cover more null space situations, reduce the possibility of selecting the wrong null space vector, and improve the noise robustness of the method. Different from the existing method that solves based on intermediate variables (control point coordinates) and partial constraints (distance between control points), the processing method adopted in this application directly based on the rotation matrix elements and makes full use of the self-constraint of the rotation matrix (i.e., the orthogonality constraint of the rotation matrix) for solving.

[0147] The initial value acquisition unit 150 is configured to determine the current initial translation vector based on the screened zero-point analysis result.

[0148] In this embodiment, since the zero-point analysis result with the determinant value of 1 corresponding to the rotation matrix E and the minimum reprojection error is screened out from the zero-point analysis result set, the above-screened zero-point analysis result can be used to recover the translation vector, that is, the current initial translation vector is determined first by the rotation matrix elements and the focal length in the screened zero-point analysis result.

[0149] An iterative optimization unit 160 is configured to iteratively optimize the current initial translation vector based on a preset iterative optimization strategy to obtain an optimized translation vector, so that the camera to be calibrated is calibrated based on the optimized translation vector.

[0150] In this embodiment, the obtained current initial translation vector may not necessarily be used as the optimal camera calibration parameter. The current initial translation vector can be further iteratively optimized based on the iterative optimization strategy to obtain an optimized translation vector. Finally, the optimized translation vector is used as the optimal camera calibration parameter to calibrate the camera to be calibrated.

[0151] In one embodiment, the iterative optimization unit 160 is specifically configured to:

[0152] Obtain a damped Newton iterative optimization strategy corresponding to the iterative optimization strategy, and iteratively optimize the current initial translation vector based on the damped Newton iterative optimization strategy until the least squares error function of the current translation vector corresponding to the current initial translation vector has a minimum value, and then stop the iterative optimization and obtain the current translation vector as the optimized translation vector.

[0153] In this embodiment, if the rotation matrix E corresponding to the current initial translation vector is represented in the above formula (11). The cost equation is constructed as the above formula (12) according to formula (8); the damped Newton iterative optimization strategy is used to iteratively optimize the current initial translation vector, and the value at the k-th time is set as , then at the (k + 1)-th iteration, it is updated and represented by the above formula (13).

[0154] In the above formula (13), λ is the step size, I5 is a 5-dimensional identity matrix, H(m k ) and g(m k ) are the Hessian matrix and the gradient vector of the cost equation with respect to m k respectively. Since the initial value accuracy of the current initial translation vector is high, usually only a few steps of iteration are required to reach the optimal accuracy, so as to obtain the current translation vector as the optimized translation vector, and finally the optimized translation vector is used as the optimal camera calibration parameter to calibrate the camera to be calibrated.

[0155] It can be seen that implementing the embodiment of this device can directly construct constraints without relying on intermediate variables starting from the pinhole camera model, thereby improving the robustness to the point distribution; moreover, adopting a null space analysis framework based on the rotation matrix constraint has high operation efficiency and can calibrate the camera to be calibrated more quickly and accurately.

[0156] The above camera fast and robust calibration device for outdoor vision measurement can be implemented in the form of a computer program, and this computer program can be stored in a storage medium such as Figure 7runs on the computer device shown.

[0157] Please refer to Figure 7 , Figure 7 which is a schematic block diagram of a computer device provided by an embodiment of the present invention. This computer device integrates any one of the camera fast and robust calibration devices for outdoor vision measurement provided by the embodiments of the present invention.

[0158] Refer to Figure 7 , the computer device 400 includes a processor 402, a memory, and a network interface 405 connected through a system bus 401. Among them, the memory may include a storage medium 403 and an internal memory 404.

[0159] The storage medium 403 can store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions. When the program instructions are executed, the processor 402 can be made to execute the above-mentioned camera fast and robust calibration method for outdoor vision measurement.

[0160] The processor 402 is used to provide computing and control capabilities to support the operation of the entire computer device.

[0161] The internal memory 404 provides an environment for the operation of the computer program 4032 in the storage medium 403. When the computer program 4032 is executed by the processor 402, the processor 402 can be made to execute the above-mentioned camera fast and robust calibration method for outdoor vision measurement.

[0162] The network interface 405 is used for network communication with other devices. Those skilled in the art can understand that Figure 7 the structure shown in [[ ]] is only a block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0163] Among them, the processor 402 is used to run the computer program 4032 stored in the memory to implement the above-mentioned camera fast and robust calibration method for outdoor vision measurement.

[0164] It should be understood that in the embodiments of the present invention, the processor 402 may be a central processing unit (CPU), and the processor 402 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0165] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, and the storage medium is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the above methods.

[0166] Therefore, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, where the computer program includes program instructions. When the program instructions are executed by the processor, the processor executes the camera fast and robust calibration method for outdoor vision measurement as described above.

[0167] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, an optical disc, or other computer-readable storage media that can store program codes.

[0168] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0169] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0170] The steps in the method embodiments of the present invention can be adjusted, combined, and deleted according to actual needs. The units in the device embodiments of the present invention can be combined, divided, and deleted according to actual needs. In addition, the functional units in each embodiment of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0172] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A fast and robust camera calibration method for outdoor visual measurement, characterized in that Including: Obtain a pinhole camera model corresponding to the camera to be calibrated, and correspondingly construct an initial linear equation based on the pinhole camera model and preset construction information; Convert the initial linear equation based on a preset conversion strategy to obtain a homogeneous equation; Perform zero-point analysis processing on the homogeneous equation based on a preset zero-point analysis strategy to obtain a zero-point analysis result set including several pairs of rotation matrix elements and focal lengths; wherein, each zero-point analysis result in the zero-point analysis result set corresponds to a pair of rotation matrix elements and a focal length; Filter the zero-point analysis result set based on a preset filtering strategy to obtain a filtered zero-point analysis result; Determine the current initial translation vector based on the filtered zero-point analysis result; Iteratively optimize the current initial translation vector based on a preset iterative optimization strategy to obtain an optimized translation vector, so that the camera to be calibrated is calibrated based on the optimized translation vector; If it is determined that the control point coordinates corresponding to multiple control points of the camera to be calibrated are coplanar, then solve for the null space coefficients of the homogeneous equation based on the coplanar configuration strategy and the rotation matrix orthogonality constraint in the zero-point analysis strategy to obtain second null space coefficients, and determine the zero-point analysis result set with the second null space coefficients; wherein, before executing the coplanar configuration strategy, it further includes making the Z-axis of the image physical coordinate system perpendicular to the plane where the control points are located; if it is determined that the control point coordinates corresponding to the multiple control points are coplanar, it corresponds to plane point configuration, and the preprocessing method corresponding to plane point configuration is to perform a rigid transformation on the world coordinate system to make the Z-axis of the image physical coordinate system perpendicular to the plane where the control points are located; the step of solving for the null space coefficients of the homogeneous equation based on the coplanar configuration strategy and the rotation matrix orthogonality constraint in the zero-point analysis strategy to obtain second null space coefficients is to solve for the second null space coefficients of the homogeneous equation in the image physical coordinate system by combining the coplanar configuration strategy and the rotation matrix orthogonality constraint, and compensate the rigid transformation to the second null space coefficients to determine the zero-point analysis result set.

2. The method according to claim 1, characterized in that, Converting the initial linear equation based on a preset conversion strategy to obtain a homogeneous equation, including: Obtain a linear least squares criterion corresponding to the preset conversion strategy; Convert the initial linear equation based on the linear least squares criterion and make the translation vector related to the rotation matrix elements and the focal length to obtain the homogeneous equation.

3. The method according to claim 1, wherein The performing zero-point analysis processing on the homogeneous equation based on a preset zero-point analysis strategy to obtain a zero-point analysis result set including several pairs of rotation matrix elements and focal lengths, including: Obtain multiple control points corresponding to the camera to be calibrated, and control point coordinates respectively corresponding to each control point; If it is determined that the control point coordinates corresponding to the multiple control points are not coplanar, then solve for the null space coefficients of the homogeneous equation based on the normal configuration strategy and the rotation matrix orthogonality constraint in the zero-point analysis strategy to obtain first null space coefficients, and determine the zero-point analysis result set with the first null space coefficients.

4. The method according to claim 1, characterized in that, Screening the zero-point analysis result set based on a preset screening strategy to obtain a screened zero-point analysis result, including: Based on the screening strategy, for each rotation matrix corresponding to the zero-point analysis result in the zero-point analysis result set, and deleting the zero-point analysis result corresponding to the left-handed coordinate system of the rotation matrix from the zero-point analysis result set to obtain a zero-point analysis result set after the first round of screening; Obtain the reprojection error of each zero-point analysis result in the zero-point analysis result set after the first round of screening, and use the zero-point analysis result with the smallest reprojection error as the screened zero-point analysis result.

5. The method according to claim 1, characterized in that, Iteratively optimizing the current initial translation vector based on a preset iterative optimization strategy to obtain an optimized translation vector, including: Obtain a damped Newton iterative optimization strategy corresponding to the iterative optimization strategy, and iteratively optimize the current initial translation vector based on the damped Newton iterative optimization strategy until the least squares error function of the current translation vector corresponding to the current initial translation vector has a minimum value, and then stop the iterative optimization, and obtain the current translation vector as the optimized translation vector.

6. A camera rapid and robust calibration device for outdoor visual measurement, characterized in that, Including: An initial linear equation construction unit, configured to obtain a pinhole camera model corresponding to the camera to be calibrated, and construct an initial linear equation based on the pinhole camera model and preset construction information; A homogeneous equation obtaining unit, configured to convert the initial linear equation based on a preset conversion strategy to obtain a homogeneous equation; A zero-point analysis processing unit, configured to perform zero-point analysis processing on the homogeneous equation based on a preset zero-point analysis strategy to obtain a zero-point analysis result set including several pairs of rotation matrix elements and focal lengths; wherein, each zero-point analysis result in the zero-point analysis result set corresponds to a pair of rotation matrix elements and a focal length; An initial value screening unit, configured to screen the zero-point analysis result set based on a preset screening strategy to obtain a screened zero-point analysis result; An initial value obtaining unit, configured to determine a current initial translation vector based on the screened zero-point analysis result; An iterative optimization unit, configured to iteratively optimize the current initial translation vector based on a preset iterative optimization strategy to obtain an optimized translation vector, so that the camera to be calibrated is calibrated based on the optimized translation vector; If it is determined that the control point coordinates corresponding to multiple control points of the camera to be calibrated are coplanar, then based on the coplanar configuration strategy and the rotation matrix orthogonality constraint in the zero-point analysis strategy, the null space coefficients of the homogeneous equation are solved to obtain the second null space coefficients, and the zero-point analysis result set is determined with the second null space coefficients; wherein, before executing the coplanar configuration strategy, it further includes making the Z-axis of the image physical coordinate system perpendicular to the plane where the control points are located; if it is determined that the control point coordinates corresponding to the multiple control points are coplanar, it corresponds to the planar point configuration, and the preprocessing method corresponding to the planar point configuration is to perform a rigid transformation on the world coordinate system to make the Z-axis of the image physical coordinate system perpendicular to the plane where the control points are located; the step of solving the null space coefficients of the homogeneous equation based on the coplanar configuration strategy and the rotation matrix orthogonality constraint in the zero-point analysis strategy to obtain the second null space coefficients is to solve the null space coefficients of the homogeneous equation in the image physical coordinate system by combining the coplanar configuration strategy and the rotation matrix orthogonality constraint to obtain the second null space coefficients, and compensating the rigid transformation to the second null space coefficients to determine the zero-point analysis result set.

7. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the fast and robust camera calibration method for outdoor vision measurement according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by the processor, the fast and robust camera calibration method for outdoor vision measurement according to any one of claims 1-5 can be implemented.

Images