Camera calibration method, device and readable storage medium

Abstract

The application provides a camera calibration method, device and readable storage medium, the camera calibration method comprises the following steps: S1, obtaining N groups of image sequences respectively rotated by a camera around N rotation axes, wherein N >= 1; S2, tracking original pixel tracks of effective feature points on the N groups of image sequences; S3, solving principal points and distortion parameters of the camera by fitting a quadratic curve of the original pixel tracks for optimization. The camera calibration method provided by the application does not rely on various calibration tools such as checkerboards, can simplify the camera calibration process, improve the camera calibration accuracy, and is suitable for parameter calibration of various lenses such as long-focus lenses and zoom lenses in various complex and harsh environments.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a camera calibration method, device, and readable storage medium. Background Art

[0002] In fields such as machine vision, camera calibration is the most fundamental step in many visual applications. Through camera calibration, important functions such as visual measurement, 3D reconstruction, and SLAM can be realized.

[0003] Camera calibration is mainly used to solve the camera's internal parameters, including focal length (f x ,f y ), principal point (c x ,c y ) and lens distortion parameters (k1, k2, k3). The radial distortion model of the lens is as follows:

[0004]

[0005] Among them, (u,v) is the original image coordinate, is the transformed image coordinate, r=‖(u,v)-(c x ,c y )‖2 represents the distance from the image coordinate to the center of distortion, and D(r) represents the radial distortion model of the lens. The radial distortion model of the lens includes but is not limited to the polynomial model and the division model. The polynomial model is as follows:

[0006] D(r)=1+k1r 2 +k2r 4 +k3r 6 (2)

[0007] The division model method is as follows:

[0008]

[0009] The academic paper "A Flexible New Technique for Camera Calibration" describes a classic camera calibration method that estimates the camera's intrinsic parameters by capturing multiple images of a regular checkerboard-like calibration pattern. However, many complex field environments do not allow for calibration using various calibration tools like checkerboards. Furthermore, the camera's intrinsic parameters can be affected by external forces, impacts, and even environmental temperature and humidity during use. Therefore, a more convenient method is needed to regularly calibrate and update at least some of the camera's parameters. Summary of the Invention

[0010] The object of the present invention is to provide a camera calibration method, device and readable storage medium, which at least solve one of the technical problems existing in the prior art.

[0011] In order to achieve the above object, as a first aspect of the present invention, a camera calibration method is provided, comprising the following steps:

[0012] S1. Obtain N sets of image sequences of the camera rotating around N rotation axes, where N ≥ 1;

[0013] S2, tracking the original pixel trajectories of the valid feature points on the N groups of image sequences;

[0014] S3. Optimizing and solving the principal point and distortion parameters of the camera by fitting a quadratic curve to the original pixel trajectory.

[0015] Optionally, the S2 specifically includes:

[0016] S21, the kth group of image sequences of the N groups of image sequences includes I k Extract feature points from each image, where k = 1, 2, 3...N;

[0017] S22, tracking the I k The same feature points in the image are recorded, and the same feature points appear in the image. k pixel coordinates in the image;

[0018] S23 , selecting the same feature points whose number of pixel coordinates exceeds a set threshold as valid feature points on the kth group of image sequences, and using the pixel coordinates of the valid feature points as original pixel trajectories of the valid feature points on the kth group of image sequences.

[0019] Optionally, the S3 specifically includes:

[0020] S31, constructing a first optimization variable using the principal point and distortion parameters of the camera, and performing distortion correction on the original pixel trajectories of the effective feature points on the N groups of image sequences using a radial distortion model of the lens, to obtain corrected pixel trajectories of the effective feature points on the N groups of image sequences under the first optimization variable;

[0021] S32, performing quadratic curve fitting on the corrected pixel trajectories of the effective feature points on the N groups of image sequences to obtain fitting quadratic curves of the effective feature points on the N groups of image sequences;

[0022] S33, calculating a first fitting error between a corrected pixel trajectory of the effective feature points on the N groups of image sequences and a fitting quadratic curve of the effective feature points on the N groups of image sequences;

[0023] S34, calculating a first collinearity error based on a principal axis collinearity error of a fitting quadratic curve of valid feature points of each image sequence in the N image sequences;

[0024] S35. Define a first optimization objective function using the first fitting error and the first collinearity error, use an optimization algorithm to make the first optimization objective function reach a global minimum, and use the optimal solution of the first optimization variable as the principal point and distortion parameter of the camera.

[0025] Optionally, the fitting quadratic curve is a circle, an ellipse, a hyperbola or a parabola.

[0026] Optionally, the camera calibration method further includes:

[0027] S4, performing distortion correction on the original pixel trajectory using the solved principal point and distortion parameters of the camera to obtain an undistorted pixel trajectory;

[0028] S5. Optimizing and solving the focal length and rotation axis direction vector of the camera by projecting and transforming the undistorted pixel trajectory and fitting a circular curve.

[0029] Optionally, the S5 specifically includes:

[0030] S51, constructing a second optimization variable using the camera focal length f and the direction vectors of N rotation axes, calculating N projection transformation rotation vectors corresponding to the N rotation axes, and converting the N projection transformation rotation vectors into N projection transformation rotation matrices using Rodrigues transform;

[0031] S52, performing a projective transformation on the undistorted pixel trajectories of the valid feature points on the N groups of image sequences using the focal length f of the camera, the principal point of the camera, and the N projective transformation rotation matrices, to obtain projective transformed pixel trajectories of the valid feature points on the N groups of image sequences;

[0032] S53, performing circular curve fitting on the pixel trajectory of the projection transformation of the effective feature points on the N groups of image sequences to obtain a fitting circular curve of the projection transformation of the effective feature points on the N groups of image sequences;

[0033] S54, calculating the pixel trajectory of the projected transformation of the effective feature points on the N groups of image sequences and the second fitting error of the fitted circular curve;

[0034] S55, calculating a second collinearity error based on the collinearity error of the centers of fitted circular curves of the projective transformation of the effective feature points of each of the N groups of image sequences;

[0035] S56. Define a second optimization objective function based on a second fitting error of the fitting circular curve of the projective transformation of the valid feature points on the N groups of image sequences and a second collinearity error of the fitting circular curve of the projective transformation of the valid feature points on the N groups of image sequences, adopt an optimization algorithm to make the second optimization objective function reach a global minimum, obtain an optimal solution for the second optimization variable, and use the optimal solution of the second optimization variable as the camera focal length and the N rotation axis direction vectors.

[0036] Based on the same inventive concept, as a second aspect of the present invention, another camera calibration method is provided, comprising the following steps:

[0037] S100, obtaining N sets of image sequences of the camera rotating around N rotation axes, where N ≥ 1;

[0038] S200, tracking original pixel trajectories of valid feature points on the N groups of image sequences;

[0039] S300, performing distortion correction on the original pixel trajectory according to known camera principal points and distortion parameters to obtain an undistorted pixel trajectory;

[0040] S400 , optimizing and solving the focal length and rotation axis direction vector of the camera by projecting and transforming the undistorted pixel trajectory and fitting a circular curve.

[0041] Optionally, the S200 specifically includes:

[0042] S201, the kth group of image sequences of the N groups of image sequences includes I k Extract feature points from each image, where k = 1, 2, 3...N;

[0043] S202, tracking the I k The same feature points in the image are recorded, and the same feature points appear in the image. k pixel coordinates in the image;

[0044] S203 , selecting the same feature points whose number of pixel coordinates exceeds a set threshold as valid feature points on the kth group of image sequences, and using the pixel coordinates of the valid feature points as original pixel trajectories of the valid feature points on the kth group of image sequences.

[0045] Optionally, the S400 specifically includes:

[0046] S401, constructing a third optimization variable using the camera focal length f and the direction vectors of N rotation axes, calculating N projection transformation rotation vectors corresponding to the N rotation axes, and converting the N projection transformation rotation vectors into N projection transformation rotation matrices using Rodrigues transform;

[0047] S402, performing a projective transformation on the undistorted pixel trajectories of the valid feature points on the N groups of image sequences using the focal length f of the camera, the principal point of the camera, and the N projective transformation rotation matrices to obtain projective transformed pixel trajectories of the valid feature points on the N groups of image sequences;

[0048] S403, performing circular curve fitting on the pixel trajectory of the projection transformation of the effective feature points on the N groups of image sequences to obtain a fitting circular curve of the projection transformation of the effective feature points on the N groups of image sequences;

[0049] S404, calculating the pixel trajectory of the projected transformation of the effective feature points on the N groups of image sequences and the third fitting error of the fitted circular curve;

[0050] S405, calculating three collinearity errors based on the collinearity errors of the centers of fitted circular curves of the projective transformation of the effective feature points of each image sequence in the N image sequences;

[0051] S406. Define a third optimization objective function based on a third fitting error of the fitted circular curve of the projective transformation of the valid feature points on the N groups of image sequences and a third collinearity error of the fitted circular curve of the projective transformation of the valid feature points on the N groups of image sequences, adopt an optimization algorithm to globally minimize the third optimization objective function, obtain an optimal solution for the third optimization variable, and use the optimal solution for the third optimization variable as the camera focal length and the N rotation axis direction vectors.

[0052] Based on the same inventive concept, as a third aspect of the present invention, a camera calibration device is provided, comprising:

[0053] at least one imaging component, the imaging component comprising at least one camera capable of acquiring continuous images and a rotation axis with at least one degree of freedom, the camera being capable of rotating about the rotation axis with each degree of freedom;

[0054] The processor is configured to execute the camera calibration method as described above.

[0055] Based on the same inventive concept, as a fourth aspect of the present invention, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it can implement the camera calibration method described above.

[0056] The camera calibration method, device, and readable storage medium provided by the present invention do not rely on various calibration tools such as checkerboards, can simplify the camera calibration process, improve camera calibration accuracy, and are suitable for parameter calibration of various lenses such as telephoto lenses and zoom lenses in various complex and harsh environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Those skilled in the art will appreciate that the accompanying drawings are provided for a better understanding of the present invention and do not constitute any limitation on the scope of the present invention.

[0058] Figure 1 A diagram showing the steps of a camera calibration method provided by an embodiment of the present invention;

[0059] Figure 2 A diagram showing the steps of another camera calibration method provided by an embodiment of the present invention;

[0060] Figure 3 A schematic structural diagram of a camera calibration device provided in one embodiment of the present invention;

[0061] Figure 4 A schematic structural diagram of a single-degree-of-freedom rotating axis camera component provided by an embodiment of the present invention;

[0062] Figure 5 This is a schematic structural diagram of a three-degree-of-freedom rotating axis camera component provided by an embodiment of the present invention.

[0063] In the attached figure:

[0064] 1- Camera component; 2- Processor; 10- Camera; 11- Rotation axis. DETAILED DESCRIPTION

[0065] In order to make the objects, advantages and features of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are in a very simplified form and are not in precise proportions. They are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention, and are not used to limit the conditions for the implementation of the present invention. Therefore, they have no technical significance. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed by the present invention without affecting the efficacy and purpose of the present invention.

[0066] It should also be understood that, unless otherwise specified or indicated, the terms "first", "second", "third", etc. in the description are merely used to distinguish between the various components, elements, steps, etc. in the description, and are not used to indicate the logical relationship or sequential relationship between the various components, elements, steps, etc. Moreover, the terms "comprise", "include", or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0067] The core idea of ​​the present invention is to provide a camera calibration method, device and readable storage medium. Without relying on various calibration tools such as checkerboards, the calibration of camera internal parameters such as camera focal length, principal point and lens distortion parameters is achieved. This simplifies the camera calibration process and improves the camera calibration accuracy. The method is suitable for parameter calibration of various lenses such as telephoto lenses and zoom lenses in various complex and harsh environments.

[0068] The present invention provides a camera calibration method comprising the following steps:

[0069] S1. Obtain N sets of image sequences of the camera rotating around N rotation axes, where N ≥ 1;

[0070] S2, tracking the original pixel trajectories of the valid feature points on the N groups of image sequences;

[0071] S3. Optimizing and solving the principal point and distortion parameters of the camera by fitting a quadratic curve to the original pixel trajectory.

[0072] Another camera calibration method provided by the present invention comprises the following steps:

[0073] S100, obtaining N sets of image sequences of the camera rotating around N rotation axes, where N ≥ 1;

[0074] S200, tracking original pixel trajectories of valid feature points on the N groups of image sequences;

[0075] S300, performing distortion correction on the original pixel trajectory according to known camera principal points and distortion parameters to obtain an undistorted pixel trajectory;

[0076] S400 , optimizing and solving the focal length and rotation axis direction vector of the camera by projecting and transforming the undistorted pixel trajectory and fitting a circular curve.

[0077] The camera calibration device provided by the present invention includes:

[0078] at least one imaging component, the imaging component comprising at least one camera capable of acquiring continuous images and a rotation axis with at least one degree of freedom, the camera being capable of rotating about the rotation axis with each degree of freedom;

[0079] The processor is configured to execute the camera calibration method as described above.

[0080] The readable storage medium provided by the present invention stores a computer program thereon, and when the computer program is executed by a processor, it can implement the camera calibration method described above.

[0081] The following description is given with reference to the accompanying drawings.

[0082] Figure 1 A diagram showing the steps of a camera calibration method provided by an embodiment of the present invention; Figure 2 A diagram showing the steps of another camera calibration method provided by an embodiment of the present invention; Figure 3 A schematic structural diagram of a camera calibration device provided in one embodiment of the present invention; Figure 4 A schematic structural diagram of a single-degree-of-freedom rotating axis camera component provided by an embodiment of the present invention; Figure 5 This is a schematic structural diagram of a three-degree-of-freedom rotating axis camera component provided by an embodiment of the present invention.

[0083] Please refer to Figure 1 , this embodiment provides a camera calibration method, comprising the following steps:

[0084] S1. Obtain N sets of image sequences of the camera rotating around N rotation axes, where N ≥ 1;

[0085] S2, tracking the original pixel trajectories of the valid feature points on the N groups of image sequences;

[0086] S3. Optimizing and solving the principal point and distortion parameters of the camera by fitting a quadratic curve to the original pixel trajectory.

[0087] First, step S1 is performed to obtain N sets of image sequences of a camera rotating around N rotation axes. Specifically, the camera is rotated around N rotation axes, and the camera collects N sets of image sequences during the rotation process around the N rotation axes, where N≥1.

[0088] Then, step S2 is executed to track the original pixel trajectories of the effective feature points on the N groups of image sequences.

[0089] Furthermore, the S2 specifically includes:

[0090] S21, the kth group of image sequences of the N groups of image sequences includes I k Extract feature points from each image, where k = 1, 2, 3...N;

[0091] S22, tracking the I k The same feature points in the image are recorded, and the same feature points appear in the image. k pixel coordinates in the image;

[0092] S23, selecting the same feature points whose pixel coordinates exceed a set threshold as valid feature points on the k-th group of image sequences, and using the pixel coordinates of the valid feature points as the original pixel trajectory of the valid feature points on the k-th group of image sequences, denoted as Among them, k=1,2,3…N, m=1,2,3…Λ k , i=1,2,3…Γ km , Λ k is the number of effective feature points on the kth group of image sequences, Γ km is the number of pixel coordinates of the mth valid feature point in the kth group of image sequences.

[0093] Optionally, the image sequence of the kth group is tracked by feature descriptor matching or optical flow method. k The same feature points in the two images.

[0094] Optionally, feature points may be extracted using at least one of a FAST algorithm, an ORB algorithm, a SIFT algorithm, and a SURF algorithm.

[0095] Then, step S3 is executed to optimize and solve the principal point and distortion parameters of the camera by fitting the quadratic curve of the original pixel trajectory. Further, step S3 specifically includes:

[0096] S31, constructing a first optimization variable x1=(c x ,c y ,k1,k2,k3), using the radial distortion model of the lens to calculate the original pixel trajectory of the effective feature points on the N groups of image sequences Perform distortion correction to obtain the corrected pixel trajectory of the effective feature points on the N groups of image sequences under the first optimization variable

[0097] S32: Correcting pixel trajectories of effective feature points on the N groups of image sequences Perform quadratic curve fitting to obtain the fitting quadratic curve of the effective feature points on the N groups of image sequences

[0098] S33, calculating a first fitting error between a corrected pixel trajectory of the effective feature points on the N groups of image sequences and a fitting quadratic curve of the effective feature points on the N groups of image sequences;

[0099] S34, calculating a first collinearity error based on a principal axis collinearity error of a fitting quadratic curve of valid feature points of each image sequence in the N image sequences;

[0100] S35: define a first optimization objective function with the first fitting error and the first collinearity error, and use an optimization algorithm to make the first optimization objective function reach a global minimum. At this time, the optimal value of the first optimization variable is As the principal point of the camera and distortion parameters

[0101] Execute S31-S35 in sequence, wherein in S32, the fitting quadratic curve of the effective feature points on the k-th group of image sequences has Λ k The fitting quadratic curves of the effective feature points on the N groups of image sequences are , where Λ k represents the number of valid feature points in the kth group of image sequences.

[0102] In this embodiment, the fitting quadratic curve C km For a circle, ellipse, hyperbola or parabola, the expression for fitting the quadratic curve is defined as:

[0103] C km :a km x 2 +b km y 2 +c km xy+d km x+e km y+f km =0 (4)

[0104] Among them, a km 、b km 、c km d km 、e km 、f km is the polynomial coefficient of the fitting quadratic curve, a km and b km Not 0 at the same time.

[0105] In the S33, the first fitting error is defined as:

[0106]

[0107] in, The corrected pixel track of the mth valid feature point on the kth image sequence The shortest distance between the fitted conic C km The distance weight of the ith pixel coordinate of the corrected pixel track of the mth valid feature point on the kth image sequence, the farther the pixel coordinate is from the principal point of the camera, the greater the distance weight

[0108] In the S34, the first collinear error of the fitted conic is defined as:

[0109]

[0110] Wherein, The principal axis collinear error of the fitted conic of the valid feature point on the kth image sequence;

[0111] In the S35, the first optimization objective function is defined as:

[0112]

[0113] Wherein, The different error model combination of e1(x1) and l1(x1);

[0114] Assuming that x0=(w / 2, h / 2, 0, 0, 0) is the optimization initial value, wherein w and h are the length and width of the image respectively, the optimization algorithm is used to make the objective function reach the global minimum, and the optimal solution is obtained The principal point of the camera is The distortion parameter is

[0115] Optionally, the optimization algorithm is a nonlinear optimization algorithm such as GN, LM, or an evolutionary algorithm such as GA, PSO.

[0116] Optionally, the selection of the optimization initial value can sample k1 and k2 in a fixed range, calculate The corresponding first optimization objective function value e(x ij ), and select the x ij corresponding to the minimum first optimization objective function value e(x ij as the optimization initial value.

[0117] Further, the camera calibration method further comprises:

[0118] S4, performing distortion correction on the original pixel track by using the principal point and the distortion parameter obtained by solving to obtain a pixel track without distortion;

[0119] ​​S5. Optimizing and solving the focal length and rotation axis direction vector of the camera by projecting and transforming the undistorted pixel trajectory and fitting a circular curve.

[0120] After obtaining the principal point and distortion parameters of the camera, the focal length and rotation axis direction vector of the camera can be further obtained. Specifically, after executing S3, the principal point obtained by S3 is used. and distortion parameters The original pixel trajectory of the effective feature points on the N groups of image sequences is obtained by using the radial distortion model of the lens. Perform distortion correction to obtain the undistorted pixel trajectory of the effective feature points on the N groups of image sequences

[0121] Finally, S5 is executed to optimize and solve the focal length and rotation axis direction vector of the camera by projecting and transforming the undistorted pixel trajectory and fitting a circular curve. S5 specifically includes:

[0122] S51, with the camera focal length f and the direction vector n of N rotation axes k Construct the second optimization variable x2=(f,n1…n N ), where k = 1…N, ‖n k ‖2=1; calculate the N projection transformation rotation vectors n′ corresponding to the N rotation axes k =[0 0 1] T ×n k , use Rodrigues transformation to transform the N projection rotation vector n′ k Converted to N projection transformation rotation matrices R k =rotationVectionToMatrix(n′ k );

[0123] S52, using the focal length f of the camera and the principal point of the camera And the N projection transformation rotation matrices are used to calculate the pixel trajectory of the effective feature points on the N groups of image sequences without distortion. Perform projection transformation to obtain the pixel trajectory of the projection transformation of the effective feature points on the N groups of image sequences The projection transformation function proj is expressed as follows:

[0124]

[0125] S53: Pixel trajectory of the projection transformation of the effective feature points on the N groups of image sequences Perform circular curve fitting to obtain the fitting circular curve of the projection transformation of the effective feature points on the N groups of image sequences; wherein the fitting circular curve Circle of the projection transformation of the mth effective feature point on the kth group of image sequenceskm The expression is as follows:

[0126] Circle km :(xa km ) 2 +(yb km ) 2 -r km 2 =0 (9) Among them, a km 、b km is the center of the fitting circular curve, r km is the radius of the fitted circular curve;

[0127] S54, calculating the pixel trajectory of the projection transformation of the effective feature points on the N groups of image sequences and the fitted circular curve Circle km The second fitting error is defined as:

[0128]

[0129] in, is the pixel trajectory of the projected transformation Fitting the circle with the curve Circle km The shortest distance;

[0130] S55. Calculate a second collinearity error based on the collinearity error of the centers of the fitted circular curves of the projective transformation of the effective feature points of each of the N image sequences. The second collinearity error is defined as:

[0131]

[0132] in, is the collinearity error of the center of the fitted circular curve of the projection transformation of the effective feature points on the kth group of image sequences;

[0133] S56: Define a second optimization objective function based on a second fitting error of the circular curve fitted by the projective transformation of the effective feature points on the N groups of image sequences and a second collinearity error of the circular curve fitted by the projective transformation of the effective feature points on the N groups of image sequences. The second optimization objective function is as follows:

[0134]

[0135] in, is a combination of different error models of e2(x2) and l2(x2);

[0136] Use the optimization algorithm to make the second optimization objective function reach the global minimum and obtain the optimal solution of the second optimization variable The focal length of the camera is f opt , the N rotation axis direction vectors are Where k = 1,…,N.

[0137] Therefore, the camera calibration method provided by the present invention realizes the calibration of camera internal parameters such as camera focal length, principal point, and lens distortion parameters without relying on various calibration tools such as checkerboards, simplifies the camera calibration process, and improves the camera calibration accuracy. It is suitable for parameter calibration of various lenses such as telephoto lenses and zoom lenses in various complex and harsh environments.

[0138] Optionally, the optimization algorithm is a nonlinear optimization algorithm such as GN, LM, or an evolutionary algorithm such as GA, PSO, etc.

[0139] Based on the same invention concept, please refer to Figure 2 , an embodiment of the present invention further provides another camera calibration method, comprising the following steps:

[0140] S100, obtaining N sets of image sequences of the camera rotating around N rotation axes, where N ≥ 1;

[0141] S200, tracking original pixel trajectories of valid feature points on the N groups of image sequences;

[0142] S300, performing distortion correction on the original pixel trajectory according to known camera principal points and distortion parameters to obtain an undistorted pixel trajectory;

[0143] S400 , optimizing and solving the focal length and rotation axis direction vector of the camera by projecting and transforming the undistorted pixel trajectory and fitting a circular curve.

[0144] The difference from the previous camera calibration method that first solves the camera's principal point and distortion parameters is that in this method, the camera's principal point and distortion parameters are known, and then the camera's focal length and rotation axis direction vector are solved based on the known camera principal point and distortion parameters.

[0145] First, step S1 is performed to obtain N sets of image sequences of a camera rotating around N rotation axes. Specifically, the camera is rotated around N rotation axes, and the camera collects N sets of image sequences during the rotation process around the N rotation axes, where N≥1.

[0146] Then, step S2 is executed to track the original pixel trajectories of the effective feature points on the N groups of image sequences.

[0147] Furthermore, the S200 specifically includes:

[0148] S201, the kth group of image sequences of the N groups of image sequences includes Ik Extract feature points from each image, where k = 1, 2, 3...N;

[0149] S202, tracking the I k The same feature points in the image are recorded, and the same feature points appear in the image. k pixel coordinates in the image;

[0150] S203 , selecting the same feature points whose number of pixel coordinates exceeds a set threshold as valid feature points on the kth group of image sequences, and using the pixel coordinates of the valid feature points as original pixel trajectories of the valid feature points on the kth group of image sequences.

[0151] Optionally, the image sequence of the kth group is tracked by feature descriptor matching or optical flow method. k The same feature points in the two images.

[0152] Optionally, feature points may be extracted using at least one of a FAST algorithm, an ORB algorithm, a SIFT algorithm, and a SURF algorithm.

[0153] After executing S203, execute S300, with the known camera principal point and distortion parameters The original pixel trajectory of the effective feature points on the N groups of image sequences is obtained by using the radial distortion model of the lens. Perform distortion correction to obtain the undistorted pixel trajectory of the effective feature points on the N groups of image sequences

[0154] It should be understood that the known camera principal points and distortion parameters mentioned here can be understood as factory parameters of the camera or camera principal points and distortion parameters obtained using methods known in the prior art.

[0155] Finally, S400 is executed to optimize and solve the focal length and rotation axis direction vector of the camera by projecting and transforming the undistorted pixel trajectory and fitting a circular curve. Further, S400 specifically includes:

[0156] S401, using the camera focal length f and the direction vector n of N rotation axes k Construct the third optimization variable x3=(f,n1…n N ), where k = 1…N, ‖n k ‖2=1; calculate the N projection transformation rotation vectors n′ corresponding to the N rotation axes k =[0 0 1] T ×nk, using Rodrigues transformation to transform the N projections into rotation vector n′ k Converted to N projection transformation rotation matrices R k=rotationVectionToMatrix(n′ k );

[0157] S402, using the focal length f of the camera and the principal point of the camera And the N projection transformation rotation matrices are used to calculate the pixel trajectory of the effective feature points on the N groups of image sequences without distortion. Perform projection transformation to obtain the pixel trajectory of the projection transformation of the effective feature points on the N groups of image sequences The projection transformation function proj is expressed as follows:

[0158]

[0159] S403: Pixel trajectories of the projected transformation of the effective feature points on the N groups of image sequences Perform circular curve fitting to obtain the fitting circular curve of the projection transformation of the effective feature points on the N groups of image sequences; wherein the fitting circular curve Circle of the projection transformation of the mth effective feature point on the kth group of image sequences km The expression is as follows:

[0160] Circle km :(xa km ) 2 +(yb km ) 2 -r km 2 =0 (14)

[0161] Among them, a km 、b km is the center of the fitting circular curve, r km is the radius of the fitted circular curve;

[0162] S404: Calculate the pixel trajectory of the projection transformation of the effective feature points on the N groups of image sequences and the fitted circular curve Circle km The third fitting error is defined as:

[0163]

[0164] in, is the pixel trajectory of the projected transformation Fitting the circle with the curve Circle km The shortest distance;

[0165] S405: Calculate a third collinearity error based on the collinearity error of the centers of the fitted circular curves of the projective transformation of the effective feature points of each of the N image sequences. The third collinearity error is defined as:

[0166]

[0167] in, is the collinearity error of the center of the fitted circular curve of the projection transformation of the effective feature points on the kth group of image sequences;

[0168] S406: Define a third optimization objective function based on a third fitting error of the circular curve fitted by the projective transformation of the effective feature points on the N groups of image sequences and a third collinearity error of the circular curve fitted by the projective transformation of the effective feature points on the N groups of image sequences. The third optimization objective function is as follows:

[0169]

[0170] in, are different error model combinations of e3(x3) and l3(x3);

[0171] The optimization algorithm is used to make the third optimization objective function reach the global minimum and obtain the optimal solution of the third optimization variable. The focal length of the camera is f opt , the N rotation axis direction vectors are Where k = 1,…,N.

[0172] Therefore, the present invention provides a second camera calibration method that realizes the calibration of camera focal length without relying on various calibration tools such as a checkerboard, simplifies the camera calibration process, and improves the camera calibration accuracy. It is suitable for parameter calibration of various lenses such as telephoto lenses and zoom lenses in various complex and harsh environments.

[0173] Based on the same inventive concept, the embodiment of the present invention further provides a camera calibration device, Figure 3-Figure 5 , the camera calibration device comprises:

[0174] At least one imaging component 1, the imaging component 1 comprising at least one camera 10 capable of acquiring continuous images and a rotation axis 11 with at least one degree of freedom, the camera 10 being capable of rotating around the rotation axis 11 with each degree of freedom;

[0175] The processor 2 is configured to execute the camera calibration method as described above.

[0176] Based on the same inventive concept, an embodiment of the present invention further provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the camera calibration method described above can be implemented.

[0177] The readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The readable storage medium can be, for example but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched-tape, a holographic storage medium, or any suitable combination of the foregoing. A computer program product, can include a computer readable storage medium (or media) having stored thereon instructions that can be executed by a processing device (or multiple processing devices) to cause the processing device(s) to perform processes described herein. The computer program product can include packaged computer instructions, such as one or more modules of computer instructions encoded on a machine-readable medium, such as a machine-readable medium in a server, a client, or any machine in combination with a server or client. The computer program product can be distributed over network-coupled (and, vice versa) computer systems so that the computer program instructions can be executed from a server or client as a distributed algorithm, thus causing a combination of processing devices to perform the processes described herein. The computer program instructions can execute entirely on a user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0178] Various aspects of the present invention are described herein with reference to the flowcharts and / or block diagrams of the methods, systems, and computer program products according to embodiments of the present invention. It should be understood that each block of the flowcharts and / or block diagrams, as well as the combination of blocks in the flowcharts and / or block diagrams, can be implemented by computer programs. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these programs are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowcharts and / or block diagrams. These computer programs can also be stored in a readable storage medium, which causes the computer, programmable data processing device, and / or other device to operate in a specific manner, so that the readable storage medium storing the computer program comprises an article of manufacture comprising instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowcharts and / or block diagrams.

[0179] The computer program may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the computer program executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0180] In summary, the present invention provides a camera calibration method, device, and readable storage medium, which can achieve camera parameter calibration without relying on various calibration tools such as checkerboards, simplify the camera calibration process, and improve the camera calibration accuracy. It is suitable for parameter calibration of various lenses such as telephoto lenses and zoom lenses in various complex and harsh environments.

[0181] Furthermore, it should be recognized that although the present invention has been disclosed above with reference to preferred embodiments, the above embodiments are not intended to limit the present invention. Any person skilled in the art can utilize the above disclosed technical content to make many possible changes and modifications to the technical solution of the present invention, or modify it into equivalent embodiments with equivalent variations, without departing from the scope of the technical solution of the present invention. Therefore, any simple modifications, equivalent variations, and modifications made to the above embodiments based on the technical essence of the present invention, without departing from the content of the technical solution of the present invention, shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A camera calibration method, characterized in that: The following steps are involved: S1. Obtain N sets of image sequences of the camera rotating around N rotation axes, where N ≥ 1; S2, tracking the original pixel trajectories of the valid feature points on the N groups of image sequences; S3. Optimizing and solving the principal point and distortion parameters of the camera by fitting a quadratic curve to the original pixel trajectory.

2. The camera calibration method according to claim 1, wherein: The S2 specifically includes: S21, the kth group of image sequences of the N groups of image sequences includes I k Extract feature points from each image, where k = 1, 2, 3...N; S22, tracking the I k The same feature points in the image are recorded, and the same feature points appear in the image. k pixel coordinates in the image; S23 , selecting the same feature points whose number of pixel coordinates exceeds a set threshold as valid feature points on the kth group of image sequences, and using the pixel coordinates of the valid feature points as original pixel trajectories of the valid feature points on the kth group of image sequences.

3. The camera calibration method according to claim 1 or 2, wherein: The S3 specifically includes: S31, constructing a first optimization variable using the principal point and distortion parameters of the camera, and performing distortion correction on the original pixel trajectories of the effective feature points on the N groups of image sequences using a radial distortion model of the lens, to obtain corrected pixel trajectories of the effective feature points on the N groups of image sequences under the first optimization variable; S32, performing quadratic curve fitting on the corrected pixel trajectories of the effective feature points on the N groups of image sequences to obtain fitting quadratic curves of the effective feature points on the N groups of image sequences; S33, calculating a first fitting error between a corrected pixel trajectory of the effective feature points on the N groups of image sequences and a fitting quadratic curve of the effective feature points on the N groups of image sequences; S34, calculating a first collinearity error based on a principal axis collinearity error of a fitting quadratic curve of valid feature points of each image sequence in the N image sequences; S35. Define a first optimization objective function using the first fitting error and the first collinearity error, use an optimization algorithm to make the first optimization objective function reach a global minimum, and use the optimal solution of the first optimization variable as the principal point and distortion parameter of the camera.

4. The camera calibration method according to claim 3, wherein: The fitting quadratic curve is a circle, an ellipse, a hyperbola or a parabola.

5. The camera calibration method according to claim 1, wherein: The camera calibration method further includes: S4, performing distortion correction on the original pixel trajectory using the solved principal point and distortion parameters of the camera to obtain an undistorted pixel trajectory; S5. Optimizing and solving the focal length and rotation axis direction vector of the camera by projecting and transforming the undistorted pixel trajectory and fitting a circular curve.

6. The camera calibration method according to claim 5, wherein: The S5 specifically includes: S51, constructing a second optimization variable using the camera focal length f and the direction vectors of N rotation axes, calculating N projection transformation rotation vectors corresponding to the N rotation axes, and converting the N projection transformation rotation vectors into N projection transformation rotation matrices using Rodrigues transform; S52, performing a projective transformation on the undistorted pixel trajectories of the valid feature points on the N groups of image sequences using the focal length f of the camera, the principal point of the camera, and the N projective transformation rotation matrices, to obtain projective transformed pixel trajectories of the valid feature points on the N groups of image sequences; S53, performing circular curve fitting on the pixel trajectory of the projection transformation of the effective feature points on the N groups of image sequences to obtain a fitting circular curve of the projection transformation of the effective feature points on the N groups of image sequences; S54, calculating the pixel trajectory of the projected transformation of the effective feature points on the N groups of image sequences and the second fitting error of the fitted circular curve; S55, calculating a second collinearity error based on the collinearity error of the centers of fitted circular curves of the projective transformation of the effective feature points of each of the N groups of image sequences; S56. Define a second optimization objective function based on a second fitting error of the fitting circular curve of the projective transformation of the valid feature points on the N groups of image sequences and a second collinearity error of the fitting circular curve of the projective transformation of the valid feature points on the N groups of image sequences, adopt an optimization algorithm to make the second optimization objective function reach a global minimum, obtain an optimal solution for the second optimization variable, and use the optimal solution of the second optimization variable as the camera focal length and the N rotation axis direction vectors.

7. A camera calibration method, characterized in that: The following steps are involved: S100, obtaining N sets of image sequences of the camera rotating around N rotation axes, where N ≥ 1; S200, tracking original pixel trajectories of valid feature points on the N groups of image sequences; S300, performing distortion correction on the original pixel trajectory according to known camera principal points and distortion parameters to obtain an undistorted pixel trajectory; S400 , optimizing and solving the focal length and rotation axis direction vector of the camera by projecting and transforming the undistorted pixel trajectory and fitting a circular curve.

8. The camera calibration method according to claim 7, wherein: The S200 specifically includes: S201, the kth group of image sequences of the N groups of image sequences includes I k Extract feature points from each image, where k = 1, 2, 3...N; S202, tracking the I k The same feature points in the image are recorded, and the same feature points appear in the image. k pixel coordinates in the image; S203 , selecting the same feature points whose number of pixel coordinates exceeds a set threshold as valid feature points on the kth group of image sequences, and using the pixel coordinates of the valid feature points as original pixel trajectories of the valid feature points on the kth group of image sequences.

9. The camera calibration method according to claim 8, wherein: The S400 specifically includes: S401, constructing a third optimization variable using the camera focal length f and the direction vectors of N rotation axes, calculating N projection transformation rotation vectors corresponding to the N rotation axes, and converting the N projection transformation rotation vectors into N projection transformation rotation matrices using Rodrigues transform; S402, performing a projective transformation on the undistorted pixel trajectories of the valid feature points on the N groups of image sequences using the focal length f of the camera, the principal point of the camera, and the N projective transformation rotation matrices to obtain projective transformed pixel trajectories of the valid feature points on the N groups of image sequences; S403, performing circular curve fitting on the pixel trajectory of the projection transformation of the effective feature points on the N groups of image sequences to obtain a fitting circular curve of the projection transformation of the effective feature points on the N groups of image sequences; S404, calculating the pixel trajectory of the projected transformation of the effective feature points on the N groups of image sequences and the third fitting error of the fitted circular curve; S405, calculating a third collinearity error based on the collinearity error of the centers of fitted circular curves of the projective transformation of the effective feature points of each of the N groups of image sequences; S406. Define a third optimization objective function based on a third fitting error of the fitted circular curve of the projective transformation of the valid feature points on the N groups of image sequences and a third collinearity error of the fitted circular curve of the projective transformation of the valid feature points on the N groups of image sequences, adopt an optimization algorithm to globally minimize the third optimization objective function, obtain an optimal solution for the third optimization variable, and use the optimal solution for the third optimization variable as the camera focal length and the N rotation axis direction vectors.

10. A camera calibration device, characterized in that: include: at least one imaging component, the imaging component comprising at least one camera capable of acquiring continuous images and a rotation axis with at least one degree of freedom, the camera being capable of rotating about the rotation axis with each degree of freedom; A processor configured to execute the camera calibration method according to any one of claims 1 to 9.

11. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it can implement the camera calibration method according to any one of claims 1 to 9.

Images