A camera calibration method, electronic device and storage medium

Abstract

This application provides a camera calibration method, electronic device, and storage medium. The method includes: acquiring the first coordinates of each grid point in a calibration board image in a pixel coordinate system and the second coordinates in a world coordinate system, wherein the calibration board image is acquired by the camera to be calibrated; dividing the calibration board image radially into a first preset number of intervals based on a preset center point; wherein each interval includes multiple grid points; extracting a second preset number of sample grid points from each interval, and processing the first and second coordinates of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated. This method avoids the local overfitting phenomenon of the radial distortion coefficient caused by uneven radial distribution of grid points, thus improving the overall calibration accuracy of the camera.

Description

Technical Field

[0001] This application relates to the field of camera calibration, and more specifically, to a camera calibration method, electronic device, and storage medium. Background Technology

[0002] A camera can be broadly divided into three parts: the lens, the image sensor, and the processing circuitry. When light passes through the lens, it forms an "image" of an object on the image sensor, a process similar to the pinhole camera principle. After a series of processing steps, it becomes an electronic image. Of course, this imaging process introduces certain errors (such as radial distortion of the image). Therefore, to eliminate or correct these errors, camera calibration is usually performed before the camera leaves the factory to obtain camera parameters (such as the radial distortion coefficient) used to eliminate or correct these errors.

[0003] However, the inventors discovered in practice that the radial distortion coefficients calculated by the current camera calibration process can suffer from local overfitting, which makes the estimated radial distortion coefficients unreasonable overall and affects the camera calibration results. Summary of the Invention

[0004] The purpose of this application is to provide a camera calibration method, electronic device, and storage medium to improve the local overfitting of the radial distortion coefficient during camera calibration and improve the accuracy of camera calibration.

[0005] This invention is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a camera calibration method, comprising: acquiring the first coordinates of each grid point in a calibration board image in a pixel coordinate system and the second coordinates in a world coordinate system, wherein the calibration board image is acquired by a camera to be calibrated; dividing the calibration board image radially into a first preset number of intervals based on a preset center point; wherein each interval includes multiple grid points; extracting a second preset number of sample grid points from each interval, and processing the first and second coordinates of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated.

[0007] In this embodiment, by dividing the pixel coordinates and world coordinates of the grid points into equal parts radially, and then extracting the same number of sample grid points from each radially divided interval to calculate the radial distortion coefficient, the local overfitting phenomenon of the radial distortion coefficient caused by the uneven radial distribution of grid points can be avoided, thereby improving the overall calibration accuracy of the camera.

[0008] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the step of extracting a second preset number of sample grid points from each of the intervals and processing the first and second coordinates of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated includes: S1: extracting the second preset number of sample grid points from each of the intervals; S2: processing the first and second coordinates of the sample grid points to obtain the radial distortion coefficient corresponding to the sample grid points; S3: based on the radial distortion coefficient corresponding to the sample grid points, determining the projection error of the remaining grid points in each of the intervals, comparing the projection error of the remaining grid points with a preset threshold, and counting the number P of remaining grid points whose projection error is less than the preset threshold. inlier ; wherein, the remaining grid points are the remaining grid points in each interval excluding the sample grid points; S4: if P inlier >P max Then let P max =P inlier Among them, P max The initial value is the preset value, P max S5: Repeat steps S1 to S4 until the preset iteration condition is met, then output the iterated P. max And based on the iterative P max The corresponding radial distortion coefficient is used to determine the radial distortion coefficient of the camera to be calibrated.

[0009] Because grid point extraction may result in inaccurate grid points due to factors such as lighting, aberrations, and dirt, these points can introduce random errors. While uniformizing the grid points, such as through interval sampling, reduces the number of grid points needed to calculate the radial distortion coefficient, this approach utilizes the concept of random sampling consistency. It repeatedly extracts the same number of sample grid points from each radially divided interval to calculate the radial distortion coefficient. After meeting preset conditions, it outputs the determined radial distortion coefficient of the camera to be calibrated. This method filters out sample grid points with significant errors, reducing random error interference and further improving the overall estimation accuracy and reliability of the radial distortion coefficient.

[0010] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the step of basing the solution on the iterated P... max The corresponding radial distortion coefficients are used to determine the radial distortion coefficients of the camera to be calibrated, including: based on the iterative P... max The corresponding radial distortion coefficient is obtained from the P. maxAll corresponding target grid points; wherein, the target grid points are those corresponding to P after the iteration. max The remaining grid points whose projection error is less than the preset threshold under the corresponding radial distortion coefficient; the first and second coordinates of the target grid points are processed to obtain the radial distortion coefficient of the camera to be calibrated.

[0011] In this embodiment of the application, when obtaining P max After identifying all target grid points, the radial distortion coefficient of the camera to be calibrated is calculated based on all target grid points. This method can reduce the random errors introduced by calculating the radial distortion coefficient through uniform sampling.

[0012] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the iterated P max The corresponding radial distortion coefficient is the radial distortion coefficient of the camera to be calibrated.

[0013] In this embodiment of the application, the iterated P is directly used max The corresponding radial distortion coefficient is determined to be the radial distortion coefficient of the camera to be calibrated, which can improve computational efficiency and reduce the processing pressure on electronic equipment.

[0014] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the preset iteration conditions include: the number of iterations reaching a preset number, or the number P of remaining grid points where the projection error is less than the preset threshold. inlier The ratio of the number of remaining grid points to the number of grid points is greater than a preset ratio value.

[0015] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, after determining the radial distortion coefficient of the camera to be calibrated, the method further includes: correcting the distortion of the image acquired by the camera to be calibrated based on the radial distortion coefficient of the camera to be calibrated.

[0016] Secondly, embodiments of this application provide a camera calibration device, comprising: a first acquisition module, configured to acquire the first coordinates of each grid point of a calibration board image in a pixel coordinate system and the second coordinates in a world coordinate system, wherein the calibration board image is acquired by a camera to be calibrated; a first processing module, configured to divide the calibration board image radially into a first preset number of intervals based on a preset center point; wherein each interval includes multiple grid points; extract a second preset number of sample grid points from each interval, and process the first and second coordinates of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated.

[0017] Thirdly, a computer program product includes computer program instructions that, when read and executed by a processor, perform the method provided in the first aspect of the embodiments described above.

[0018] Fourthly, embodiments of this application provide an electronic device, including: a processor and a memory, the processor and the memory being connected; the memory being used to store a program; the processor being used to invoke the program stored in the memory to execute the method provided in the first aspect of the embodiments described above.

[0019] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, the computer program executing the method provided in the first aspect of the embodiments described above when run by a processor. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of a calibration board image used in the existing camera calibration process.

[0022] Figure 2 This is a module block diagram of an electronic device provided in an embodiment of this application.

[0023] Figure 3 A flowchart illustrating the steps of a first camera calibration method provided in this application embodiment.

[0024] Figure 4 This is a flowchart illustrating the steps of an imaging method using a camera, as provided in an embodiment of this application.

[0025] Figure 5 This is a block diagram of a camera calibration device provided in an embodiment of this application.

[0026] Figure 6 This is a block diagram of an imaging device for a camera provided in an embodiment of this application.

[0027] Icons: 100 - Electronic device; 110 - Processor; 120 - Memory; 200 - Camera calibration device; 210 - First acquisition module; 220 - First processing module; 300 - Camera imaging device; 310 - Second acquisition module; 320 - Second processing module; 330 - Distortion correction module. Detailed Implementation

[0028] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0029] With the development of smart devices, cameras have become an indispensable component. A camera can be broadly divided into three parts: lens, image sensor, and processing circuitry. When light passes through the lens, it forms an "image" of an object on the image sensor, a process similar to pinhole imaging. After a series of processing steps, it becomes an electronic image. Of course, this imaging process introduces certain errors (such as radial distortion). Therefore, to eliminate or correct these errors, camera calibration is usually performed before the camera leaves the factory to obtain camera parameters (such as the radial distortion coefficient) used to eliminate or correct these errors.

[0030] However, the inventors discovered in practice that the radial distortion coefficients calculated using current camera calibration processes suffer from local overfitting, leading to an overall unreasonable estimation of the radial distortion coefficients and affecting the camera calibration results. Research revealed that while the grid points in the currently captured calibration board images are uniformly distributed along the horizontal (X-axis) and vertical (Y-axis) directions, they are not uniformly distributed radially. Radial distribution can be understood as the distribution along a radius (r) centered on a point in the image. Due to this non-uniform radial distribution of grid points, calculating the radial distortion coefficients using all grid points leads to local overfitting. For example, as... Figure 1 As shown, when the image center is taken as the center, the projection error (0.4) of each grid point near the center is significantly greater than the projection error (0.2) of each grid point far from the center. Because the projection error is uneven in the radial direction from the image center, it is easy to cause local overfitting of the radial distortion coefficient.

[0031] The discovery of the above-mentioned problems was the result of the inventors' practice and careful research. Therefore, the discovery process of the above-mentioned problems and the solutions proposed by the inventors in the embodiments of the present invention below should be considered as contributions made by the inventors to the present invention.

[0032] Please see Figure 2 This application provides a schematic structural block diagram of an electronic device 100 that applies the camera calibration method or camera imaging method described below. In this application embodiment, the electronic device 100 can be any device including a camera, such as, but not limited to, a smartphone, tablet computer, SLR camera, camcorder, etc. Of course, the electronic device 100 can also be any device connected to a camera, such as a personal computer connected to a camera.

[0033] Structurally, the electronic device 100 may include a processor 110 and a memory 120.

[0034] The processor 110 is electrically connected directly or indirectly to the memory 120 to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The camera calibration device or camera imaging device described later includes at least one software module that can be stored in the memory 120 or embedded in the operating system (OS) of the electronic device 100 in the form of software or firmware. The processor 110 is used to execute the executable module stored in the memory 120. The processor 110 can execute a computer program after receiving an execution instruction.

[0035] The processor 110 can be an integrated circuit chip with signal processing capabilities. The processor 110 can also be a general-purpose processor, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a discrete gate or transistor logic device, or a discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. Furthermore, the general-purpose processor can be a microprocessor or any conventional processor.

[0036] The memory 120 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM). The memory 120 is used to store programs, which the processor 110 executes upon receiving an execution instruction.

[0037] It should be noted that, Figure 2 The structure shown is for illustrative purposes only. The electronic device 100 provided in this embodiment may also have a structure that is more advanced than the one shown in the diagram. Figure 2 Fewer or more components, or having the same Figure 2 The different configurations shown. Furthermore... Figure 2 The components shown can be implemented through software, hardware, or a combination thereof.

[0038] Please see Figure 3 , Figure 3 This is a flowchart illustrating the steps of a camera calibration method provided in an embodiment of this application. The method is applied to... Figure 2 The electronic device 100 shown. It should be noted that the camera calibration method provided in this embodiment is not based on... Figure 3 The following order is a limitation, and the method includes: steps S101-S103.

[0039] Step S101: Obtain the first coordinates of each grid point in the pixel coordinate system and the second coordinates in the world coordinate system of the calibration plate image, wherein the calibration plate image is acquired by the camera to be calibrated.

[0040] Specifically, the operator needs to keep the camera and calibration plate stationary in a fixed position. To directly determine the camera's radial distortion coefficient, at least three calibration plates should be used, and these plates should be at different folding angles. For example, this can be achieved using... Figure 1 The placement of the four calibration plates is described. After fixing the calibration plates and camera, the calibration plates are photographed to obtain an image. Typically, the calibration plates can be placed in the exact center of the image. Furthermore, to obtain a high-quality image of the calibration plates, proper focus and appropriate lighting are required when taking the photograph.

[0041] After obtaining the calibration board image, it can be processed. Processing methods include image filtering, gradient image calculation, and second-derived image processing. Image filtering removes noise, while gradient and second-derived image processing facilitate the extraction of the first coordinates of each grid point in the pixel coordinate system. Furthermore, when determining the grid points in the calibration board image, they can be determined sequentially from left to right and from top to bottom, eliminating erroneously acquired points with adjacent distances less than a preset value. To further improve the accuracy of the coordinates, sub-pixelization can be performed on the obtained coordinates.

[0042] It should be noted that the second coordinates of each grid point in the calibration board image in the world coordinate system are determined by the placement angle of the calibration board. The first coordinates of each grid point in the calibration board image in the pixel coordinate system are the coordinates of consecutive points in that image. The second coordinates of each grid point in the calibration board image in the world coordinate system are related to the world coordinate system defined by each calibration board. For example, each calibration board uses the top left corner grid of its own board body as the origin of the world coordinate system, the plane on which the board body is located corresponds to the z-axis of the world coordinate system as zero, the x-axis of the world coordinate system is the direction of the width of the calibration board, and the y-axis of the world coordinate system is the direction of the height of the calibration board. In addition, the dimensions of the calibration board and the relevant information of the world coordinate system in the calibration board are all pre-configured in the electronic equipment by the staff.

[0043] Step S102: Based on a preset center point, the calibration plate image is divided into a first preset number of intervals in the radial direction; wherein each interval includes multiple grid points.

[0044] Then, the grid points on the calibration board image are divided radially. The preset center point can be the exact center of the image or the coordinates of the camera's intrinsic principal point. The first preset number can be determined according to actual needs. For example, in this embodiment, the first preset number is 6. Of course, the first preset number can also be 4, 8, 10, etc., and this application does not limit it.

[0045] For example, the exact center of the calibration board image is used as the preset center, and then the calibration board image is divided into 6 equal intervals radially. That is, any two adjacent intervals in the 6 intervals have the same length in the radial direction. Through this interval division method, all grid points are assigned to different intervals. Each grid point in each interval includes its own first coordinate in the pixel coordinate system and its second coordinate in the world coordinate system.

[0046] Step S103: Extract a second preset number of sample grid points from each interval, and process the first and second coordinates of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated.

[0047] After dividing the radial direction into intervals, the same number of sample grid points are extracted from each interval. Then, the radial distortion coefficients of the camera to be calibrated are calculated based on Zhang's algorithm and the preset distortion algorithm.

[0048] The second preset number can be determined according to actual needs. For example, in this embodiment, the second preset number can be 2, that is, 2 sample grid points are taken in each interval. Of course, the second preset number can also be 4, 8, 10, etc.

[0049] It should be noted that the second preset number needs to be less than the number of grid points in any interval. Therefore, as one implementation method, before extracting the second preset number of sample grid points from each interval, it is necessary to determine the number of grid points in each interval, and then determine the second preset number based on the number of grid points in each interval.

[0050] For example, the second preset number is set to 8. After dividing the intervals radially, the number of grid points in each interval is determined. If the number of grid points in all intervals is greater than 8, then 8 sample grid points are extracted from each interval. If the number of grid points in one interval is less than 8, for example, if the number of grid points in that interval is 6, then any number of grid points less than the number of grid points in that interval can be used as the second preset number. Alternatively, the number of grid points in that interval can be reduced by 1 as the determined second preset number. This application does not impose any limitations on this.

[0051] The specific process of Zhang's algorithm described above is as follows: Based on the relationship between the world coordinate system and the camera's own camera coordinate system, determine the camera's extrinsic parameters; based on the relationship between the camera coordinate system and the pixel coordinate system, determine the camera's intrinsic parameter matrix; based on the first and second coordinates of each grid point in the calibration board image, the camera's extrinsic parameters, and the camera's intrinsic parameter matrix, fit the initial value of the camera's radial distortion coefficient.

[0052] It should be noted that Zhang's algorithm involves the transformation of four coordinate systems during calibration. These four coordinate systems are: world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system.

[0053] The world coordinate system is a coordinate system defined by the operator in the three-dimensional world, introduced to describe the position of the target object in the real world. The camera coordinate system is a coordinate system established on the camera itself, defined to describe the object's position from the camera's perspective; it serves as an intermediary between the world coordinate system and the image / pixel coordinate system. The image coordinate system is introduced to describe the projection and transmission relationship of the object from the camera coordinate system to the image coordinate system during the imaging process, facilitating the further deriving of coordinates in the pixel coordinate system. The pixel coordinate system is introduced to describe the coordinates of the image points of the object after imaging on the digital image (photograph); it is the coordinate system where the information actually read from the camera resides.

[0054] Zhang's algorithm can accurately determine the initial values ​​of the camera's extrinsic and intrinsic parameter matrices, as well as the camera's radial distortion coefficients.

[0055] Since the specific calculation process of Zhang's algorithm is well known in this field, it will not be explained in detail.

[0056] The preset distortion algorithm can be either the Rational Model algorithm or the Brown-Conrady Model algorithm.

[0057] The expression for the Rational Model algorithm is as follows:

[0058]

[0059]

[0060] In formulas (1) and (2), x d and y d Represents the distorted coordinates in the image coordinate system; x u and y u k1 represents the undistorted coordinates in the image coordinate system; k1 to k6 represent the radial distortion coefficients. p1 and p2 represent the tangential distortion coefficients.

[0061] Since the values ​​of p1 and p2 are usually small, and tangential distortion is related to camera manufacturing and assembly, we will temporarily disregard tangential distortion. Therefore, formulas (1) and (2) can be transformed into:

[0062]

[0063]

[0064] The expression for the Brown-Conrady Model algorithm is as follows:

[0065]

[0066]

[0067] It should be noted that the difference between the Brown-Conrady Model algorithm and the Rational Model algorithm is that the Brown-Conrady Model algorithm only includes radial distortion coefficients k1 to k3, while the Rational Model algorithm also includes radial distortion coefficients k4 to k6. All other parameters are the same, and will not be repeated here.

[0068] Neglecting tangential distortion, equations (5) and (6) can be transformed into:

[0069] x d =x u (1+k1r 2 +k2r4 +k3r 6 (7)

[0070] y d =y u (1+k1r 2 +k2r 4 +k3r 6 (8)

[0071] The following describes the specific process of calculating the radial distortion coefficient of the camera to be calibrated based on the sample grid points in each interval. The process specifically includes steps S11 to S14.

[0072] Step S11: Determine the estimated projection point of each grid point based on the second coordinates of the sample grid points in the world coordinate system, the initial value of the radial distortion coefficient of the camera, the intrinsic and extrinsic parameters of the camera, and the preset distortion algorithm; wherein, the estimated projection point is the coordinate point in the pixel coordinate system.

[0073] Step S12: Calculate the error between the estimated projection point of each sample grid point and the first coordinate of each grid point in the pixel coordinate system.

[0074] Step S13: Optimize the error value to obtain the optimized radial distortion coefficient.

[0075] Step S14: Substitute the optimized radial distortion coefficients into the initial values ​​of the radial distortion coefficients in steps S11 to S13 to determine the radial distortion coefficients after further optimization; repeat this process until the cutoff condition is met, and then output the optimized radial distortion coefficients corresponding to the latest error value; wherein, the optimized radial distortion coefficients corresponding to the latest error value are the radial distortion coefficients of the camera to be calibrated.

[0076] The aforementioned cutoff condition can refer to the minimum error value, the error value no longer changing, or a fixed number of iterations, such as ten, twenty, one hundred, etc. This application does not impose any limitations.

[0077] Taking the Rational Model algorithm as an example of the preset distortion algorithm, the specific process of the above steps is as follows: First, based on the camera's extrinsic parameters and the second coordinates of the sample grid points of the calibration board image in the world coordinate system, determine the distortion-free coordinates of the sample grid points of the calibration board image in the image coordinate system. The expression for calculating the distortion-free coordinates is:

[0078]

[0079] In formula (9), [*] represents a matrix, (X, Y, Z) represents the second coordinate of the sample grid point in the world coordinate system; R and t are the extrinsic parameters of the camera, R is a 3*3 vector and t is a 3*1 vector; s is a scaling factor of any value.

[0080] The distortion-free coordinates (x, y) of the sample grid points in the image coordinate system are calculated. u y u Then, based on the RationalModel algorithm, i.e., formulas (3) and (4), the distorted coordinates (x, y, x) of the sample grid points in the image coordinate system are calculated. d y d Then, the estimated projection point for each sample grid point is calculated. The expression for calculating the estimated projection point is:

[0081]

[0082] In formula (10), u p and v p This represents the estimated projection points of the sample grid points; Let f represent the intrinsic parameter matrix, where f represents the focal length, dx represents the pixel size in the x-direction, dy represents the pixel size in the y-direction, and x0 and y0 represent the principal point coordinates.

[0083] Then, the first error value between the estimated projection point of each sample grid point and the first coordinate of its corresponding grid point in the pixel coordinate system is calculated, and all the first error values ​​are stacked to obtain the error value.

[0084] The expression for calculating the first error value is as follows:

[0085]

[0086] In formula (11), e x and e y Let u and v represent the first set of error values, where u and v represent the first coordinates of the sample grid points in the pixel coordinate system. Since the stacking of the above vectors is well known in this field, it will not be elaborated upon here.

[0087] Finally, the optimizer is invoked to optimize the error values, thereby obtaining the optimized radial distortion coefficients. It should be noted that the optimization algorithm used by the optimizer can include, but is not limited to, the LM (Levenberg–Marquardt) algorithm, gradient descent, etc. Since such optimization algorithms are well-known in this field, they will not be discussed in detail here.

[0088] Then, the optimized radial distortion coefficients are substituted into steps S11 to S13 to replace the initial values ​​of the radial distortion coefficients, and the radial distortion coefficients are optimized again to determine the radial distortion coefficients. This process is repeated until the cutoff condition is met, and then the optimized radial distortion coefficients corresponding to the latest error value are output as the radial distortion coefficients k1 to k6 of the camera.

[0089] Similarly, when the distortion algorithm is the Brown-Conrady Model algorithm, the above example can be used as a reference, and this application will not elaborate further.

[0090] In summary, in this embodiment of the application, by dividing the pixel coordinates and world coordinates of the grid points into equal parts in the radial direction, and then extracting the same number of sample grid points from each of the equally divided intervals in the radial direction to calculate the radial distortion coefficient, the phenomenon of local overfitting of the radial distortion coefficient caused by the uneven distribution of grid points in the radial direction can be avoided, thereby improving the overall calibration accuracy of the camera.

[0091] Because grid point extraction may result in inaccurate grid points due to factors such as lighting, aberrations, and dirt, these points can introduce random errors. While uniformizing the grid points, such as through interval sampling, reduces the number of grid points needed to calculate the radial distortion coefficient, this also mitigates the impact of random errors on the calculation results. Therefore, this application also provides a method for determining the radial distortion coefficient of a calibrated camera using the concept of random sampling consistency. Specifically, step S103 includes steps S1 to S5.

[0092] S1: Extract a second preset number of sample grid points from each interval.

[0093] S2: Process the first and second coordinates of the sample grid points to obtain the radial distortion coefficients corresponding to the sample grid points.

[0094] In this process, Zhang's algorithm and a preset distortion algorithm are used to process the first and second coordinates of the sample grid points. It should be noted that the process of calculating the radial distortion coefficient based on Zhang's algorithm and the preset distortion algorithm has already been described in the previous embodiments and will not be repeated here; the same parts can be referred to interchangeably.

[0095] S3: Based on the radial distortion coefficients corresponding to the sample grid points, determine the projection error of the remaining grid points in each interval, compare the projection error of the remaining grid points with a preset threshold, and count the number P of remaining grid points whose projection error is less than the preset threshold. inlier .

[0096] The remaining grid points are the grid points in each interval excluding the sample grid points.

[0097] After calculating the corresponding radial distortion coefficient based on the sample grid points, the projection error of the remaining grid points is calculated using this distortion coefficient. That is, the projection error of the remaining grid points is calculated by formulas (3), (4), (9), (10), and (11), or by formulas (7), (8), (9), (10), and (11).

[0098] Then, the projection error of the remaining grid points is compared with a preset threshold ε, and the number P of remaining grid points with projection errors less than the preset threshold ε is counted. inlier The value of the preset threshold ε can be set according to actual needs, and this application does not impose any restrictions.

[0099] S4: If P inlier >P max Then let P max =P inlier Among them, P max The initial value is the preset value, P max This indicates the maximum number of remaining grid points whose projection error is less than a preset threshold.

[0100] After calculating P inlier Then, it was compared with P. max Compare them. If P inlier >P max Then let P max =P inlier If P inlier <P max Then P max No processing is required. The above preset values ​​can be 0 or 1, or can be set according to requirements; this application does not impose any restrictions.

[0101] S5: Repeat steps S1 to S4 until the preset iteration conditions are met, then output the iterated P. max And based on the iterative P max The corresponding radial distortion coefficient is used to determine the radial distortion coefficient of the camera to be calibrated.

[0102] The preset iteration conditions may include: the number of iterations reaches a preset number.

[0103] The preset number of times can be 50 times, 100 times, etc., and this application does not limit it.

[0104] The preset iteration conditions may also include: the number P of remaining grid points with projection error less than a preset threshold. inlier The ratio of the number of remaining grid points to the number of grid points is greater than the preset ratio value.

[0105] The preset ratio can be 50%, 80%, etc., and this application does not limit it.

[0106] As one implementation method, based on the iterative P max The corresponding radial distortion coefficients, determining the radial distortion coefficients of the camera to be calibrated, can specifically include: based on the iterative P... max The corresponding radial distortion coefficient is obtained from P. max All corresponding target grid points; where the target grid point is the P after iteration. max The remaining grid points with projection errors less than a preset threshold under the corresponding radial distortion coefficient; the first and second coordinates of the target grid points are processed to obtain the radial distortion coefficient of the camera to be calibrated.

[0107] In this process, Zhang's algorithm and a pre-defined distortion algorithm are used to process the first and second coordinates of the target grid points.

[0108] That is, after determining P max Then, search for P during the iteration. max Corresponding P inlier and obtain the P inlier The corresponding target grid points are then used. Based on all target grid points, the radial distortion coefficients are recalculated using Zhang's algorithm and a pre-defined distortion algorithm. The final calculated radial distortion coefficients are then used as the radial distortion coefficients for the camera to be calibrated. This method reduces the random errors introduced by uniform sampling when calculating the radial distortion coefficients.

[0109] As another implementation method, the iterated P max The corresponding radial distortion coefficient is the radial distortion coefficient of the camera to be calibrated.

[0110] That is, after determining P max Then, search for P during the iteration. max Corresponding P inlier Then directly with P inlier (Each iteration is marked with the calculated radial distortion coefficient, i.e., the radial distortion coefficient corresponding to the sample grid point.) The corresponding radial distortion coefficient is the radial distortion coefficient of the camera to be calibrated. This method can improve computational efficiency and reduce the processing load on electronic equipment.

[0111] The above treatment process can effectively improve Figure 1 The problem of uneven projection error is shown. That is, through the above processing, uneven projection error can be effectively suppressed.

[0112] In summary, the embodiments of this application utilize the concept of random sampling consistency to repeatedly extract the same number of sample grid points from each radially equally divided interval to calculate the radial distortion coefficient. After meeting preset conditions, the determined radial distortion coefficient of the camera to be calibrated is output. In this way, sample grid points with large errors can be filtered out, random error interference can be reduced, and the overall estimation accuracy and reliability of the radial distortion coefficient can be further improved.

[0113] Optionally, after determining the radial distortion coefficients of the camera to be calibrated, the method may further include: correcting the distortion of the image acquired by the camera to be calibrated based on the radial distortion coefficients of the camera to be calibrated.

[0114] Please see Figure 4 Based on the same inventive concept, this application also provides an imaging method for a camera, which can also be applied to... Figure 2 The electronic device 100 shown. It should be noted that the imaging method of the camera provided in this embodiment is not based on... Figure 4 The following sequence is a limitation, and the method includes steps S201-S204.

[0115] Step S201: Obtain the first coordinates of each grid point in the pixel coordinate system and the second coordinates in the world coordinate system of the calibration plate image, wherein the calibration plate image is acquired by the camera to be calibrated.

[0116] Step S202: Based on a preset center point, the calibration plate image is divided into a first preset number of intervals in the radial direction; wherein each interval includes multiple grid points.

[0117] Step S203: Extract a second preset number of sample grid points from each interval, and process the first and second coordinates of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated.

[0118] Since steps S201 to S203 are the same as steps S101 to S103 in the previous embodiments, they will not be repeated here to avoid redundancy. The same parts can be referred to each other.

[0119] Step S204: Correct the distortion of the image acquired by the camera to be calibrated based on the radial distortion coefficient of the camera to be calibrated.

[0120] Finally, the camera can correct the distortion of the image it acquires based on the radial distortion coefficients. For example, if the improved Rational Model algorithm is used to obtain the radial distortion coefficients k1 to k6 of the camera, and then the estimated projection points are calculated based on these coefficients, these estimated projection points are the actual points after distortion correction. Alternatively, if the improved Brown-Conrady Model algorithm is used to obtain the radial distortion coefficients k1 to k3 of the camera, and then the estimated projection points are calculated based on these coefficients, these estimated projection points are also the actual points after distortion correction.

[0121] Please see Figure 5 Based on the same inventive concept, embodiments of this application provide a camera calibration device 200, comprising:

[0122] The first acquisition module 210 is used to acquire the first coordinates of each grid point in the calibration board image in the pixel coordinate system and the second coordinates in the world coordinate system, wherein the calibration board image is acquired by the camera to be calibrated.

[0123] The first processing module 220 is used to divide the calibration plate image radially into a first preset number of intervals based on a preset center point; wherein each interval includes multiple grid points; extract a second preset number of sample grid points from each interval, and process the first coordinate and second coordinate of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated.

[0124] Please see Figure 6 This application provides an imaging device 300 for a camera, comprising:

[0125] The second acquisition module 310 is used to acquire the first coordinates of each grid point in the calibration board image in the pixel coordinate system and the second coordinates in the world coordinate system, wherein the calibration board image is acquired by the camera to be calibrated.

[0126] The second processing module 320 is used to divide the calibration plate image radially into a first preset number of intervals based on a preset center point; wherein each interval includes multiple grid points; extract a second preset number of sample grid points from each interval, and process the first coordinate and second coordinate of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated.

[0127] The distortion correction module 330 is used to correct the distortion of the image acquired by the camera to be calibrated based on the radial distortion coefficient of the camera to be calibrated.

[0128] It should be noted that, as those skilled in the art will clearly understand, for the sake of convenience and brevity, the specific working processes of the systems, devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0129] Based on the same inventive concept, this application also provides a computer program product, including computer program instructions, which are read and executed by a processor to perform the method provided in the above embodiments.

[0130] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which executes the methods provided in the above embodiments when the computer program is run.

[0131] The storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0132] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0133] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0135] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0136] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A camera calibration method, characterized in that, include: The calibration board image is obtained by acquiring the first coordinates of each grid point in the pixel coordinate system and the second coordinates in the world coordinate system, wherein the calibration board image is acquired by the camera to be calibrated. Based on a preset center point, the calibration board image is divided into a first preset number of intervals in the radial direction; wherein each interval includes multiple grid points; Extract a second preset number of sample grid points from each of the intervals, and process the first and second coordinates of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated; The step of extracting a second preset number of sample grid points from each of the intervals, and processing the first and second coordinates of the sample grid points to determine the radial distortion coefficient of the camera to be calibrated includes: S1: Extract the second preset number of sample grid points from each of the intervals; S2: Process the first and second coordinates of the sample grid points to obtain the radial distortion coefficients corresponding to the sample grid points; S3: Based on the radial distortion coefficients corresponding to the sample grid points, determine the projection error of the remaining grid points in each interval, compare the projection error of the remaining grid points with a preset threshold, and count the number of remaining grid points whose projection error is less than the preset threshold. The remaining grid points are the remaining grid points in each interval excluding the sample grid points. S4: If > Then let = ;in, The initial value is the preset value. This represents the maximum number of remaining grid points whose projection error is less than the preset threshold. S5: Repeat steps S1 to S4 until the preset iteration conditions are met, then output the iterated result. And based on the iteration The corresponding radial distortion coefficient is used to determine the radial distortion coefficient of the camera to be calibrated.

2. The method according to claim 1, characterized in that, The basis and the iteration The corresponding radial distortion coefficient, determining the radial distortion coefficient of the camera to be calibrated, includes: Based on the iteration The corresponding radial distortion coefficient is obtained from the... All corresponding target grid points; wherein, the target grid points are those obtained after the iteration. The remaining grid points whose projection error is less than the preset threshold under the corresponding radial distortion coefficient; The first and second coordinates of the target grid points are processed to obtain the radial distortion coefficients of the camera to be calibrated.

3. The method according to claim 1, characterized in that, The iterative The corresponding radial distortion coefficient is the radial distortion coefficient of the camera to be calibrated.

4. The method according to claim 1, characterized in that, The preset iteration conditions include: the number of iterations reaches a preset number, or The number of remaining grid points whose projection error is less than the preset threshold The ratio of the number of remaining grid points to the number of grid points is greater than a preset ratio value.

5. The method according to any one of claims 1-4, characterized in that, After determining the radial distortion coefficient of the camera to be calibrated, the method further includes: The distortion of the image acquired by the camera to be calibrated is corrected based on the radial distortion coefficient of the camera to be calibrated.

6. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-5.

7. An electronic device, characterized in that, include: A processor and a memory, wherein the processor and the memory are connected; The memory is used to store programs; The processor is used to run a program stored in the memory and perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a computer, performs the method as described in any one of claims 1-5.

Images