Camera calibration method and device, electronic equipment and storage medium

Abstract

The invention provides a camera calibration method and device, electronic equipment and a storage medium, and relates to the field of camera calibration, and the method comprises the steps: obtaining a general parameter matrix of a target camera, the general parameter matrix comprises an initial internal parameter and an initial distortion parameter, and the general parameter matrix is determined by using a standard calibration method for the target camera; an initial external parameter matrix of the target camera is calculated based on the conversion relation between the physical coordinates of the angular points in the checkerboards and the pixel coordinates under the checkerboard photos under the checkerboard photos of at least two different view angles, and the initial external parameter matrix comprises initial external parameters corresponding to the checkerboard photos of at least two different view angles; constructing a target function based on the re-projection error of the angular points in the checkerboard; and solving an optimal solution for the target function, wherein a parameter corresponding to the optimal solution is a calibration result of the target camera. The technical problems of multiple operation procedures and long algorithm processing time caused by the fact that a large number of checkerboard pictures need to be shot for camera calibration in the prior art can be solved.

Description

Technical Field

[0001] The present application relates to the field of camera calibration, and in particular to a camera calibration method, device, electronic device and storage medium. Background Art

[0002] Camera calibration is a necessary step in 3D computer vision processing. The accuracy of calibration largely determines the accuracy of the algorithm system in extracting metric information from 2D images, and is a key step in ensuring the camera's imaging quality and geometric characteristics. In order to ensure the accuracy of the calibration parameters, the traditional camera calibration method must take more than a dozen chessboard photos, and then use algorithm optimization processing to obtain appropriate camera calibration parameters. Therefore, there are technical problems such as high process cost and long algorithm processing time. Summary of the invention

[0003] The embodiments of the present application provide a camera calibration method, device, electronic device and storage medium, which can solve the technical problem in the related art that a large number of chessboard photos need to be taken for camera calibration, resulting in many operating procedures and long algorithm processing time.

[0004] In a first aspect, an embodiment of the present application provides a camera calibration method, comprising:

[0005] Obtaining a universal parameter matrix of a target camera, wherein the universal parameter matrix includes initial internal parameters and initial distortion parameters, and the universal parameter matrix is ​​determined by using a standard calibration method for the target camera;

[0006] Under at least two checkerboard photos of different viewing angles, based on the conversion relationship between the physical coordinates of the corner points in the checkerboard and the pixel coordinates under the checkerboard photos, calculating the initial external parameter matrix of the target camera, the initial external parameter matrix including the initial external parameters corresponding to the at least two checkerboard photos of different viewing angles, the different viewing angles being used to indicate the difference in angle parameters in the pose of the target camera;

[0007] Constructing an objective function based on the reprojection errors of the corner points in the chessboard, wherein the optimization parameter matrix of the objective function includes the universal parameter matrix and the initial external parameter matrix;

[0008] An optimal solution is obtained for the objective function, and a parameter corresponding to the optimal solution is a calibration result of the target camera.

[0009] In some embodiments, the calculating the initial external parameter matrix of the target camera based on the conversion relationship between the physical coordinates of the corner points in the checkerboard and the pixel coordinates in the checkerboard photograph under at least two checkerboard photographs of different viewing angles includes:

[0010] Respectively extracting pixel coordinates of corner points in the at least two chessboard photos of different viewing angles;

[0011] For each checkerboard photo of each viewing angle, a homography transformation relationship is constructed based on the pixel coordinates of the corner points, the physical coordinates of the corner points and the corresponding homography transformation matrix, wherein the homography transformation matrix is ​​the product of the general parameter matrix and the corresponding initial external parameters;

[0012] Obtaining initial external parameters corresponding to the checkerboard photo of each viewing angle according to the homography transformation matrix;

[0013] The initial external parameter matrix is ​​determined according to the initial external parameters corresponding to the checkerboard photo of each viewing angle.

[0014] In some implementations, obtaining the initial external parameters corresponding to the checkerboard photo of each perspective according to the homography transformation matrix includes:

[0015] Constructing a linear equation group about the homography transformation matrix according to the pixel coordinates of the corner points and the physical coordinates of the corner points;

[0016] Solving the linear equations to obtain the homography transformation matrix;

[0017] According to the universal parameter matrix, the homography transformation matrix is ​​subjected to matrix decomposition to obtain the initial external parameters.

[0018] In some embodiments, the number of the at least two checkerboard photos from different viewing angles is two; and / or,

[0019] The standard calibration method includes the Zhang Zhengyou calibration method.

[0020] In some implementations, constructing an objective function based on the reprojection errors of the corner points in the chessboard includes:

[0021] The objective function is established based on the difference between the actual observed value of the corner point in the chessboard and the estimated value of the corner point, wherein the estimated value is calculated based on the optimization parameter matrix on the physical coordinates of the corner point.

[0022] In some implementations, finding an optimal solution to the objective function includes:

[0023] The objective function is iteratively solved using the LM algorithm, and the optimization parameter matrix is ​​adjusted to converge the solution value of the objective function.

[0024] In some implementations, finding the optimal solution for the objective function further includes:

[0025] Constructing incremental equations for adjustment amounts of parameters in the optimization parameter matrix based on the system matrix and the gradient;

[0026] According to the incremental equation, when the adjustment amount of the parameters in the optimization parameter matrix reaches a first preset threshold, and / or the solution value of the objective function reaches a second preset threshold, it is determined that the solution value of the objective function converges to obtain the optimal solution of the objective function.

[0027] In a second aspect, the present application provides a camera calibration device, comprising:

[0028] An acquisition unit, configured to acquire a universal parameter matrix of a target camera, wherein the universal parameter matrix includes initial internal parameters and initial distortion parameters, and the universal parameter matrix is ​​determined by using a standard calibration method for the target camera;

[0029] A calculation unit is used to calculate an initial external parameter matrix of the target camera based on a conversion relationship between physical coordinates of corner points in the checkerboard and pixel coordinates in the checkerboard photo under at least two checkerboard photos of different viewing angles, wherein the initial external parameter matrix includes initial external parameters corresponding to the at least two checkerboard photos of different viewing angles, and the different viewing angles are used to indicate differences in angle parameters in the pose of the target camera;

[0030] A construction unit, configured to construct an objective function based on the reprojection errors of the corner points in the chessboard, wherein the optimization parameter matrix of the objective function includes the universal parameter matrix and the initial external parameter matrix;

[0031] A solution unit is used to find an optimal solution for the objective function, and a parameter corresponding to the optimal solution is a calibration result of the target camera.

[0032] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0033] Memory for storing computer programs;

[0034] A processor is used to execute the computer program to implement the steps of any one of the methods in the first aspect.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the steps of any one of the methods described in the first aspect.

[0036] In summary, the camera calibration method provided in the embodiment of the present application determines relatively accurate initial internal parameters and initial distortion parameters through the standard calibration method, and according to the above two parameters, collects a small number of checkerboard photos to determine the initial external parameter matrix, and uses the above initial internal parameters, initial distortion parameters and initial external parameter matrix as optimization parameters to construct the objective function, and determines the calibration parameters of the target camera based on the convergence of the above objective function. Compared with the traditional camera calibration technology, which collects a large number of checkerboard photos for calibration algorithm processing, the method proposed in the embodiment of the present application reduces the number of collected photos, thereby reducing the operating cost of the calibration workers and improving the accuracy of the collected photos. In addition, since the accuracy of the initial internal parameters, initial distortion parameters and initial external parameters is high, and the amount of data of the initial external parameters corresponding to the collected checkerboard photos is small, the speed and accuracy of the algorithm convergence can be improved.

[0037] The technical effects obtained in the above-mentioned second to fourth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 A flowchart of the idea of ​​a camera calibration method proposed in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a camera calibration method according to an embodiment of the present application;

[0041] Figure 3 A schematic diagram of a flow chart of another camera calibration method proposed in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the structure of a camera calibration device provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0045] It should be understood that the "multiple" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0046] In addition, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0047] Camera calibration is a key technology in the field of computer vision and machine vision. Its purpose is to determine the internal parameters, external parameters and distortion parameters of the camera. In related technologies, in order to obtain accurate camera calibration parameters, a large number of chessboard photos (at least more than a dozen) at various angles must be collected, and algorithms are used to optimize and converge various parameters to achieve the desired calibration accuracy. However, due to the large number of collected photos, the operation process is complicated. In addition, due to the large number of collected photos, it depends on the production line workers to manually place and adjust the chessboard. Some placement positions and angles may not meet the requirements, resulting in calibration accuracy not meeting expectations. There are also technical problems such as too many iterative parameters, which leads to a long convergence time for the algorithm.

[0048] In view of this, an embodiment of the present application proposes a camera calibration method to at least solve the above technical problem.

[0049] First, the internal parameters, external parameters and distortion parameters involved in this application are introduced:

[0050] Intrinsic parameters and external parameters are used to describe the transformation relationship between the camera physical coordinate system and the image pixel coordinate system:

[0051]

[0052] Among them, Z c [u,v,1] T Represents the image pixel coordinates, [X W ,Y W ,Z W ,1] T represents physical coordinates, fx, fy, u0, v0 represent internal parameters, R 3×3 ,T 3×1 Represents external parameters.

[0053] The distortion parameters describe the transformation relationship between the ideal camera imaging image and the actual camera imaging image (taking the third-order model as an example):

[0054]

[0055] Among them, x and y represent the coordinates after distortion correction, and represents the distorted coordinates in the original image, r 2 Represents the square distance from the pixel to the center of the image, k i Represents the distortion parameter.

[0056] Figure 1 A flow chart of the idea of ​​a camera calibration method proposed in an embodiment of the present application, such as Figure 1 As shown: The method proposed in the embodiment of the present application divides the camera calibration into two major calibration steps: laboratory calibration and production line calibration. The laboratory calibration includes steps A110 to A130, and the production line calibration includes steps A140 to A160. For example:

[0057] A110, a group of lenses can be selected from a batch of cameras with the same product specifications as calibration objects.

[0058] A120, calibrate the selected lens using a standard calibration method to obtain the internal parameters and distortion parameters of the lens.

[0059] A130, save the above internal parameters and distortion parameters into the parameter file.

[0060] Through steps A110 to A130, laboratory calibration is completed, that is, the initial values ​​of the camera internal parameters and distortion parameters are determined. Further, based on the obtained camera internal parameters and distortion parameters, production line calibration is performed through steps A140 to A160, and finally the internal parameters, external parameters and distortion parameters of the camera lens are determined.

[0061] The core idea of ​​the method proposed in the embodiment of the present application is to use the internal parameters and distortion parameters of camera calibration as a laboratory calibration process, and the external parameters as a production line calibration process. The laboratory calibration process selects the lens of the camera from the same batch specification as the debugging target, and obtains the internal parameters and distortion parameters of the camera lens through the standard calibration method, and then performs production line calibration based on the internal parameters and distortion parameters. Since the internal parameters and distortion parameters are extracted from the cameras selected from the same batch specification, and the standard calibration method is also adopted, they have a certain degree of accuracy. Therefore, in the production line calibration, there is no need to collect more chessboard photos of different perspectives to calculate the external parameters, and relatively accurate initial camera calibration parameters can be obtained for convergence calculation, thereby reducing the workload of calibration workers in collecting photos during production line calibration, improving the accuracy of the collected photos, and since the initial value in the algorithm calculation has high accuracy and small amount of data, it can improve the convergence speed and ensure accuracy.

[0062] Figure 2 A schematic diagram of a camera calibration method proposed in an embodiment of the present application is shown in FIG. Figure 2 As shown, the camera calibration method proposed in the embodiment of the present application includes steps S110 to S140:

[0063] S110, obtaining a universal parameter matrix of the target camera, where the universal parameter matrix includes initial internal parameters and initial distortion parameters, and the universal parameter matrix is ​​determined by using a standard calibration method for the target camera.

[0064] Exemplarily, this step can be performed by Figure 1 For example, a group of camera lenses are selected from a batch of camera lenses with the same product specifications as calibration objects, namely the target cameras mentioned above, and the selected lenses are calibrated by the standard calibration method to obtain the initial internal parameters and initial distortion parameters of the lenses, and a universal parameter matrix is ​​constructed based on the parameters.

[0065] In some examples, the above standard calibration method may adopt the Zhang Zhengyou calibration method, which is a very mature, high-precision and widely used calibration method. Adopting the Zhang Zhengyou calibration method can improve the accuracy of the acquired parameters.

[0066] It should be noted that this step can obtain highly accurate internal parameters and distortion parameters by selecting a specific target camera and calibrating it using a standard calibration method.

[0067] The following steps S120 to S140 are for Figure 1 Expanded description of production line calibration involved in A140 to A160:

[0068] S120, under checkerboard photos of at least two different perspectives, based on the conversion relationship between the physical coordinates of the corner points in the checkerboard and the pixel coordinates in the checkerboard photo, calculate the initial external parameter matrix of the target camera, the initial external parameter matrix including the initial external parameters corresponding to the checkerboard photos of at least two different perspectives, and different perspectives are used to indicate the difference in angular parameters in the pose of the target camera.

[0069] Exemplarily, the angle parameters in the position and posture of the target camera are adjusted to obtain at least two checkerboard photos. Based on the general parameter matrix obtained in step S110, a conversion relationship between the initial external parameters, internal parameters, distortion parameters and physical coordinates of the corner points in the checkerboard photos at different perspectives and the corresponding pixel coordinates at the perspective can be established, as shown in the above formula (1). Based on the conversion relationship, the initial external parameters at the corresponding perspective can be obtained, and the initial external parameters at different perspectives can be further constructed into an initial external parameter matrix.

[0070] It should be noted that the above corner points are marking points with key features in the chessboard.

[0071] In some examples, step S120 may be performed as follows:

[0072] Pixel coordinates of corner points in at least two chessboard photos of different viewing angles are extracted respectively.

[0073] Exemplarily, the pixel coordinates of each corner point are extracted from chessboard photos at different viewing angles.

[0074] For each checkerboard photo of each perspective, a homography transformation relationship is constructed based on the pixel coordinates of the corner points, the physical coordinates of the corner points, and the corresponding homography transformation matrix. The homography transformation matrix is ​​the product of the general parameter matrix and the corresponding initial external parameters.

[0075] For example, the homography transformation relationship constructed for the chessboard photo of each perspective is shown in the following formula:

[0076]

[0077] where p′=[u,v,1] T Represents the pixel coordinates under each chessboard photo; P = [XY 1] T represents the physical coordinates of the corner points; A represents the general parameter matrix mentioned above; [R1 R2 t] represents the initial external parameters corresponding to the checkerboard photo of each perspective, R1 and R2 represent rotation vectors, and t represents the translation vector; H represents the homography transformation matrix mentioned above, and the homography transformation matrix H corresponding to the checkerboard photo of each perspective is composed of the dot product of the general parameter matrix A and the corresponding initial external parameters [R1 R2 t].

[0078] The initial external parameters corresponding to the checkerboard photo of each perspective are obtained according to the homography transformation matrix.

[0079] Exemplarily, based on the above formula (3), the initial external parameters corresponding to the checkerboard photos at each viewing angle can be obtained.

[0080] In some examples, the initial external parameters corresponding to the checkerboard photos at each perspective can be obtained in the following way:

[0081] A linear equation group about the homography transformation matrix is ​​constructed according to the pixel coordinates of the corner points and the physical coordinates of the corner points.

[0082] Exemplarily, by constructing a linear equation group to form equations for each element in the homography transformation matrix, the corresponding elements in the homography transformation matrix can be obtained.

[0083] For example, the DLT (Direct Linear Transform) method can be used to construct a linear equation system, substituting the physical coordinates of the corner points in the chessboard and the pixel coordinates under the corresponding chessboard photo, as shown below:

[0084]

[0085] Among them, H ij Represents each element in the above homography transformation matrix H.

[0086] Solve the linear equations to obtain the homography transformation matrix.

[0087] For example, according to formula (4), the physical coordinates and pixel coordinates of the corner points are substituted into the formula to obtain H ij .

[0088] According to the universal parameter matrix, the homography transformation matrix is ​​decomposed to obtain the initial external parameters.

[0089] For example, according to formula (4):

[0090] A[R1 R2 t]=H (5)

[0091] Since the elements in the general parameter matrices A and H are known, the initial external parameters [R1 R2 t] can be obtained by matrix decomposition.

[0092] An initial external parameter matrix is ​​determined according to initial external parameters corresponding to the checkerboard photo of each viewing angle.

[0093] Exemplarily, the initial parameter matrix mentioned above can be constructed according to the initial external parameters [R1 R2 t] of each viewing angle obtained in the above steps.

[0094] It should be noted that, through the above steps S110 and S120, the embodiment of the present application obtains the initial internal parameters, initial distortion parameters and initial external parameters of the target camera. Since the initial internal parameters and initial distortion parameters are obtained through a standard calibration method, they have a certain degree of accuracy. Based on the initial internal parameters and initial distortion parameters, the corresponding initial external parameters are obtained by collecting a small number of checkerboard photos. Since the number of collected checkerboard photos is small, the operation process of the production line workers can be reduced, thereby improving the quality of the collected checkerboard photos, and then improving the accuracy of the initial external parameters derived based on the photos. Furthermore, since the accuracy of the initial internal parameters and initial distortion parameters is high, there is no need for too many initial external parameters corresponding to the checkerboard photos, and a relatively accurate initial value can be obtained for subsequent algorithm iterations.

[0095] S130, constructing an objective function based on the reprojection errors of the corner points in the chessboard, wherein the optimization parameter matrix of the objective function includes a general parameter matrix and an initial external parameter matrix.

[0096] Exemplarily, based on the optimization model of maximum likelihood estimation, the objective function is constructed to minimize the reprojection error of the checkerboard corner point set, as shown below:

[0097]

[0098] Among them, E represents the value of the objective function, M and N represent the number of horizontal and vertical grids on the chessboard, and p i,j represents the minimum projection of the coordinates of the corner point set in the world coordinate system to the pixel coordinate system, p(A,Rt,K,P j ) represents the coordinates of the corresponding corner point set calculated by parameters in the pixel coordinate system.

[0099] The optimization parameter matrix of the objective function is

[0100] m=(a,w0,w1,......w i ) (7)

[0101] Wherein, m represents the above optimization parameter matrix, a represents the above general parameter matrix, w0,w1,......w i Medium i represents the initial external parameters corresponding to the chessboard photos of each perspective, and together constitutes the above initial parameter moment. Specifically, in the above formula (7),

[0102]

[0103] Wherein, A represents the above-mentioned initial internal parameters, including the vector [fx, fy, u0, v0], and K represents the above-mentioned initial distortion parameters, including k0, k1, and k2.

[0104]

[0105] Among them, R i represents the rotation vector in the initial external parameters, including the vector [ρ x,i ,ρ y,i ,ρ z,i ], t i represents the translation vector in the initial external parameters, including the vector [t x,i ,t y,i ,t z,i ].

[0106] S140, finding an optimal solution for the objective function, where the parameters corresponding to the optimal solution are the calibration results of the target camera.

[0107] Exemplarily, the minimum value of the numerical value E of the objective function can be used as a target to adjust the optimization parameter matrix m. When the number of iterations is reached or the numerical value E reaches a preset value, it indicates that the objective function converges. In this case, the parameters corresponding to the optimization parameter matrix m are the camera calibration parameters.

[0108] In some examples, step S140 may include: using the LM algorithm to iteratively solve the objective function, and adjusting the optimization parameter matrix so that the solution value of the objective function converges.

[0109] It should be noted that the LM algorithm is a nonlinear optimization algorithm that combines the advantages of the Gauss-Newton method and the gradient descent method, and has both convergence speed and stability.

[0110] Furthermore, when using the LM algorithm to solve the objective function, the convergence speed and stability can be further guaranteed by constructing an incremental equation. For example:

[0111] An incremental equation for the adjustment amount of the parameters in the optimization parameter matrix is ​​constructed based on the system matrix and the gradient.

[0112] Exemplarily, the increment equation is as follows:

[0113] (S+λD T D)Δx k =g (10)

[0114] Where S represents the above system matrix, g represents the above gradient, D represents the unit matrix, λ represents the damping factor, Δx k Represents the adjustment amount of the parameters in the current iteration step, that is, the adjustment amount of the parameters in the above-mentioned optimization parameter matrix m.

[0115] It should be noted that the above system matrix is ​​an approximation of the second-order Hessian matrix of the above objective function.

[0116] According to the incremental equation, when the adjustment amount of the parameters in the optimization parameter matrix reaches a first preset threshold, and / or the solution value of the objective function reaches a second preset threshold, it is determined that the solution value of the objective function converges to obtain the optimal solution of the objective function.

[0117] For example, in the process of iteratively solving the above objective function, a double stop iteration criterion can be set. In the above incremental equation, Δx k When the first preset threshold is reached and the solution value E of the objective function reaches the second preset threshold, the iteration is stopped, indicating that the current objective function converges. Alternatively, when one of the two conditions is met, the current objective function is considered to converge. After convergence, the parameters in the optimized parameter matrix m in the objective function are the calibration parameters of the target camera.

[0118] It should be noted that, through steps S130 and S140, the embodiment of the present application constructs the objective function with the reprojection error of the checkerboard corner points, and takes the initial internal parameters, initial distortion parameters and initial external parameters obtained in steps S110 and S120 as optimization objects for optimization. When converged, the corresponding optimization parameters are the calibration parameters of the target camera. Since the number of collected checkerboard photos is small, the variables of the above-mentioned initial external parameters are small, so the calculation amount of the objective function is small, which is convenient for convergence, and the accuracy of the initial internal parameters, initial distortion parameters and initial external parameters is also relatively good, so the convergence speed is further improved while also having accuracy.

[0119] In summary, the camera calibration method provided in the embodiment of the present application determines relatively accurate initial internal parameters and initial distortion parameters through the standard calibration method, and determines the initial external parameters based on the above two parameters by collecting a small number of checkerboard photos, and constructs the objective function with the above initial internal parameters, initial distortion parameters and initial external parameters as optimization parameters, and determines the calibration parameters of the target camera based on the convergence of the above objective function. Compared with the traditional camera calibration technology, which collects a large number of checkerboard photos for calibration algorithm processing, the method proposed in the embodiment of the present application reduces the number of collected photos, thereby reducing the operating cost of the calibration workers and improving the accuracy of the collected photos. In addition, since the accuracy of the initial internal parameters, initial distortion parameters and initial external parameters is high, and the amount of data of the initial external parameters corresponding to the collected checkerboard photos is small, the speed and accuracy of the algorithm convergence can be improved.

[0120] Based on the camera calibration method proposed in the present application, the present application proposes a preferred embodiment to further illustrate the method. Figure 3 A schematic diagram of a camera calibration method according to an embodiment of the present application is shown in FIG. Figure 3 Take the example to illustrate the preferred embodiment proposed in this application. Figure 3 As shown:

[0121] A210, take two photos of the checkerboard.

[0122] A220, extracts corner points in the chessboard.

[0123] A230, calculate the homography transformation matrices H0 and H1 corresponding to the two checkerboard photos.

[0124] Exemplarily, the embodiment of the present application takes two checkerboard photos from different perspectives, and obtains the corresponding homography transformation matrices H0 and H1 based on the above formulas (3), (4), and (5), respectively.

[0125] A240, performing matrix decomposition on the homography transformation matrices H0 and H1 to obtain corresponding initial external parameters Rt0 and Rt1.

[0126] A250, with the goal of minimizing the reprojection error, builds a maximum likelihood model of the calibration parameters.

[0127] Exemplarily, the initial internal parameter A0 and the initial distortion parameter K obtained in the above step S110, as well as the initial external parameters Rt0 and Rt1 are combined as optimization parameters, and an objective function is constructed with the goal of minimizing the reprojection error.

[0128] A260 to A280 can be performed in the manner of steps S130 to S140 described above, which will not be described in detail.

[0129] In the embodiment of the present application, two checkerboard photos with different perspectives are collected, and the initial external parameters corresponding to the two checkerboard photos are used as the initial solution for algorithm optimization. Under the premise of ensuring the accuracy of camera calibration, the number of checkerboard photos that can be collected is minimized, the process cost is reduced, and because the data volume of the optimization parameters is also minimized, the convergence speed of the objective function is also improved.

[0130] The method embodiment according to the present application is described above by way of example. Based on the camera calibration method provided in the embodiment of the present application, the embodiment of the present application provides a camera calibration device. Figure 4 A schematic diagram of the structure of a camera calibration device provided in an embodiment of the present application. Figure 4 , the device includes the following units.

[0131] An acquisition unit 410 is used to acquire a universal parameter matrix of a target camera, wherein the universal parameter matrix includes initial internal parameters and initial distortion parameters, and the universal parameter matrix is ​​determined by using a standard calibration method for the target camera;

[0132] A calculation unit 420 is used to calculate an initial external parameter matrix of the target camera based on a conversion relationship between physical coordinates of corner points in the checkerboard and pixel coordinates in the checkerboard photo under at least two checkerboard photos of different viewing angles, wherein the initial external parameter matrix includes initial external parameters corresponding to the at least two checkerboard photos of different viewing angles, and the different viewing angles are used to indicate differences in angle parameters in the pose of the target camera;

[0133] A construction unit 430 is used to construct an objective function based on the reprojection errors of the corner points in the chessboard, wherein the optimization parameter matrix of the objective function includes the general parameter matrix and the initial external parameter matrix;

[0134] The solving unit 440 is used to find the optimal solution for the objective function, and the parameters corresponding to the optimal solution are the calibration results of the target camera.

[0135] The above describes the device embodiments of the present application. For detailed descriptions of the specific execution processes of data, terms, nouns, steps, technical issues and effects, alternative methods and combinations, please refer to the descriptions in the method embodiments, which will not be repeated here.

[0136] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program runs on a computer or a processor, the computer or the processor executes the steps of any one of the methods described in the first aspect.

[0137] An embodiment of the present application further provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any one of the methods in the first aspect.

[0138] For example, Figure 5 As shown, the electronic device includes a processor 510, at least one communication bus 520, a user interface 530, at least one external communication interface 540 and a memory 550. The communication bus 520 is configured to realize the connection and communication between these components. The user interface 530 may include a display screen, and the external communication interface 540 may include a standard wired interface and a wireless interface. The memory 550 stores a computer program. The processor 510 is used to execute the computer program stored in the memory 550.

[0139] The description of the above computer program product, computer-readable storage medium, and electronic device is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the computer program product, computer-readable storage medium, and electronic device of this application, please refer to the description of the method embodiment of this application for understanding.

[0140] The sequence of serial numbers or introduction of the embodiments of the present application is for description only and does not represent the superiority or inferiority of the embodiments.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0143] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0144] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiment of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0145] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the scene data of the current frame in the three-dimensional virtual scene, the device information of the client, and the scene interaction information involved in the embodiments of this application are all obtained with full authorization.

[0146] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A camera calibration method, characterized in that: include: Obtaining a universal parameter matrix of a target camera, wherein the universal parameter matrix includes initial internal parameters and initial distortion parameters, and the universal parameter matrix is ​​determined by using a standard calibration method for the target camera; Under at least two checkerboard photos of different viewing angles, based on the conversion relationship between the physical coordinates of the corner points in the checkerboard and the pixel coordinates under the checkerboard photos, calculating the initial external parameter matrix of the target camera, the initial external parameter matrix including the initial external parameters corresponding to the at least two checkerboard photos of different viewing angles, the different viewing angles being used to indicate the difference in angle parameters in the pose of the target camera; Constructing an objective function based on the reprojection errors of the corner points in the chessboard, wherein the optimization parameter matrix of the objective function includes the general parameter matrix and the initial external parameter matrix; An optimal solution is obtained for the objective function, and a parameter corresponding to the optimal solution is a calibration result of the target camera.

2. The method according to claim 1, characterized in that The calculating of the initial external parameter matrix of the target camera based on the conversion relationship between the physical coordinates of the corner points in the chessboard and the pixel coordinates in the chessboard photos under at least two different perspectives includes: Respectively extracting pixel coordinates of corner points in the at least two chessboard photos of different viewing angles; For each checkerboard photo of each viewing angle, a homography transformation relationship is constructed based on the pixel coordinates of the corner points, the physical coordinates of the corner points and the corresponding homography transformation matrix, wherein the homography transformation matrix is ​​the product of the general parameter matrix and the corresponding initial external parameters; Obtaining initial external parameters corresponding to the checkerboard photo of each viewing angle according to the homography transformation matrix; The initial external parameter matrix is ​​determined according to the initial external parameters corresponding to the checkerboard photo of each viewing angle.

3. The method according to claim 2, characterized in that The step of obtaining the initial external parameters corresponding to the checkerboard photo of each viewing angle according to the homography transformation matrix includes: Constructing a linear equation group about the homography transformation matrix according to the pixel coordinates of the corner points and the physical coordinates of the corner points; Solving the linear equations to obtain the homography transformation matrix; According to the universal parameter matrix, the homography transformation matrix is ​​subjected to matrix decomposition to obtain the initial external parameters.

4. The method according to any one of claims 1 to 3, characterized in that The number of the at least two checkerboard photos from different perspectives is two; and / or, The standard calibration method includes the Zhang Zhengyou calibration method.

5. The method according to any one of claims 1 to 3, characterized in that The constructing of the objective function based on the reprojection error of the corner points in the chessboard comprises: The objective function is established based on the difference between the actual observed value of the corner point in the chessboard and the estimated value of the corner point, wherein the estimated value is calculated based on the optimization parameter matrix on the physical coordinates of the corner point.

6. The method according to claim 5, characterized in that The finding of the optimal solution for the objective function comprises: The objective function is iteratively solved using the LM algorithm, and the optimization parameter matrix is ​​adjusted to converge the solution value of the objective function.

7. The method according to claim 6, characterized in that The finding of the optimal solution for the objective function further comprises: Constructing incremental equations for adjustment amounts of parameters in the optimization parameter matrix based on the system matrix and the gradient; According to the incremental equation, when the adjustment amount of the parameters in the optimization parameter matrix reaches a first preset threshold, and / or the solution value of the objective function reaches a second preset threshold, it is determined that the solution value of the objective function converges to obtain the optimal solution of the objective function.

8. A camera calibration device, characterized in that: include: An acquisition unit, configured to acquire a universal parameter matrix of a target camera, wherein the universal parameter matrix includes initial internal parameters and initial distortion parameters, and the universal parameter matrix is ​​determined by using a standard calibration method for the target camera; A calculation unit is used to calculate an initial external parameter matrix of the target camera based on a conversion relationship between physical coordinates of corner points in the checkerboard and pixel coordinates in the checkerboard photo under at least two checkerboard photos of different viewing angles, wherein the initial external parameter matrix includes initial external parameters corresponding to the at least two checkerboard photos of different viewing angles, and the different viewing angles are used to indicate differences in angle parameters in the pose of the target camera; A construction unit, configured to construct an objective function based on the reprojection errors of the corner points in the chessboard, wherein the optimization parameter matrix of the objective function includes the universal parameter matrix and the initial external parameter matrix; A solution unit is used to find an optimal solution for the objective function, and a parameter corresponding to the optimal solution is a calibration result of the target camera.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 7.

Images