Camera internal reference calibration method, device, equipment, medium and product

Abstract

The embodiment of the invention provides a camera internal reference calibration method and device, equipment, a medium and a product, and relates to the field of camera calibration. The method comprises the following steps: shooting a photo of a calibration plate comprising a plurality of checkerboards which are different in specification and are not shielded by each other through a to-be-calibrated target camera, then obtaining a plurality of calculation internal reference groups according to each checkerboard, and finally performing fusion calculation on each calculation internal reference group according to a target weight value of the corresponding checkerboard, so as to obtain a plurality of calculation internal reference groups. And determining a target internal reference group. Through the mode, the time and energy consumed by shooting the calibration plate for multiple times are reduced, and the internal reference calibration efficiency of the camera is greatly improved; the internal reference calibration of the camera can be realized only by positioning once and accommodating all checkerboards in the calibration plate into the field of view, and the relative position between the camera and the calibration plate does not need to be continuously changed to meet the field of view requirement, so that the universality of the camera internal reference calibration method provided by the invention is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of camera calibration, in particular to a camera intrinsic parameter calibration method, device, equipment, medium and product. BACKGROUND

[0002] Many functions of current intelligent vehicles cannot be realized without the assistance of vehicle internal and external cameras. Whether it is the development of basic data acquisition functions or the warning of blind spots, cameras are needed to accurately obtain real-time information. Therefore, the accuracy and efficiency of camera calibration have become the focus of the industry.

[0003] Camera intrinsic parameters (referred to as intrinsic parameters) are inherent properties of a camera, which include camera focal length, scale factor, principal point coordinates, and radial distortion parameters and tangential distortion parameters. Traditional intrinsic parameter calibration is performed by identifying valid checkerboard corner points multiple times, accumulating valid calibration point pairs, and implementing intrinsic parameter calibration based on multiple frames of point pairs. However, this method requires multiple shots to obtain pictures at different angles that cover valid checkerboard corner points. Due to multiple shots, the amount of data to be processed is large, which requires a lot of time and effort.

[0004] Therefore, an accurate and efficient camera intrinsic parameter calibration method is urgently needed. SUMMARY

[0005] To address the above technical problems, the embodiments of the present application provide a camera intrinsic parameter calibration method, device, equipment, medium and product, aiming to efficiently calibrate camera intrinsic parameters.

[0006] The first aspect of the embodiments of the present application provides a camera intrinsic parameter calibration method, which comprises:

[0007] Taking a photo of a calibration board by a target camera to be calibrated, the calibration board comprising a plurality of different specifications and non-occluded checkerboards, and the field of view of the camera covering each of the checkerboards;

[0008] Calculating each calculation intrinsic parameter group of the target camera according to each checkerboard;

[0009] Fusing each calculation intrinsic parameter group according to the target weight value of each corresponding checkerboard to determine the target intrinsic parameter group of the target camera.

[0010] Optionally, the method further comprises:

[0011] Obtaining the weight parameters of each checkerboard;

[0012] Calculating the target weight value of each checkerboard according to the weight parameters of each checkerboard.

[0013] Optionally, the target weight value of each chessboard grid is calculated according to the weight parameter of each chessboard grid, comprising:

[0014] Under the first accuracy requirement, the first weight value of each chessboard grid is calculated according to the first parameter group, and the second weight value of each chessboard grid is calculated according to the second parameter group, wherein the first parameter group at least includes the first weight parameter and the second weight parameter, and the second parameter group at least includes the third weight parameter;

[0015] The target weight value is determined according to the first weight value and the second weight value;

[0016] Under the second accuracy requirement, the first weight value of each chessboard grid is calculated according to the first parameter group, and the third weight value of each chessboard grid is calculated according to the third parameter group, wherein the third parameter group at least includes the third weight parameter and the fourth weight parameter;

[0017] The target weight value is determined according to the first weight value and the third weight value;

[0018] The second accuracy requirement is higher than the first accuracy requirement.

[0019] Optionally, under the first accuracy requirement, the second weight value of each chessboard grid is calculated according to the second parameter group, comprising:

[0020] When the sizes of the squares in each chessboard grid are inconsistent, the second parameter group further includes the fifth weight parameter.

[0021] Optionally, after the target intrinsic parameter group of the target camera is determined, when the error of the target intrinsic parameter group does not meet the requirement, the method further comprises:

[0022] The chessboard grid with the smallest size among the chessboard grids is replaced by a chessboard grid with a larger size and different from the sizes of other chessboard grids, and the chessboard grid with the smallest size is the chessboard grid containing the least number of corner points among the chessboard grids;

[0023] The intrinsic parameter calibration of the target camera is performed again according to the calibration board after the chessboard grid is replaced;

[0024] The chessboard grid with the smallest size among the chessboard grids is repeatedly replaced until the error of the target intrinsic parameter group of the target camera meets the requirement.

[0025] Optionally, after the target intrinsic parameter group of the target camera is determined, when the error of the target intrinsic parameter group does not meet the requirement, the method further comprises:

[0026] The brightness of each chessboard grid region in the calibration board is detected;

[0027] Adjust the scene light source to make the brightness of each checkerboard region uniform, the brightness uniformity refers to the gray value of the black grid in each checkerboard region is in the first range, the gray value of the white grid is in the second range, wherein the first range is different from the second range;

[0028] The photo of the calibration board is grayed, and the target camera is calibrated after adjusting the brightness of the photo.

[0029] The second aspect of the embodiment of the application provides a camera intrinsic parameter calibration device, the device comprises:

[0030] The photo shooting module is configured to shoot a photo of a calibration board by a target camera to be calibrated, the calibration board comprises a plurality of different specifications and non-occluded checkerboards, and a field of view range of the camera covers each of the checkerboards.

[0031] The calculation intrinsic parameter group determination module is configured to calculate each calculation intrinsic parameter group of the target camera according to each checkerboard.

[0032] The target intrinsic parameter group determination module is configured to fuse the calculation intrinsic parameter groups according to target weight values of the corresponding checkerboards to determine a target intrinsic parameter group of the target camera.

[0033] Optionally, the camera intrinsic parameter calibration device further comprises:

[0034] The weight parameter acquisition module is configured to acquire weight parameters of the checkerboards.

[0035] The target weight value calculation module is configured to calculate target weight values of the checkerboards according to the weight parameters of the checkerboards.

[0036] Optionally, the target weight value calculation module comprises:

[0037] The first precision weight value calculation submodule is configured to calculate first weight values of the checkerboards according to a first parameter group and second weight values of the checkerboards according to a second parameter group under a first precision requirement, the first parameter group at least comprises a first weight parameter and a second weight parameter, and the second parameter group at least comprises a third weight parameter.

[0038] The first target weight value calculation submodule is configured to determine the target weight values according to the first weight values and the second weight values.

[0039] a second precision weight value calculation sub-module, configured to calculate the first weight value of each checkerboard according to the first parameter set under a second precision requirement, and calculate a third weight value of each checkerboard according to a third parameter set, the third parameter set comprising at least a third weight parameter and a fourth weight parameter;

[0040] a second target weight value calculation sub-module, configured to determine the target weight value according to the first weight value and the third weight value

[0041] Optionally, the first precision weight value calculation sub-module comprises:

[0042] a second parameter set calculation unit, configured to, when the sizes of the squares in each checkerboard are inconsistent, the second parameter set further comprising a fifth weight parameter.

[0043] Optionally, the camera intrinsic parameter calibration device further comprises:

[0044] a checkerboard replacement module, configured to replace the checkerboard with the smallest specification in the plurality of checkerboards with a checkerboard with a larger specification and different from the specifications of the other checkerboards, the checkerboard with the smallest specification being the checkerboard with the least number of corner points in the plurality of checkerboards;

[0045] a secondary calibration module, configured to perform intrinsic parameter calibration on the target camera again according to the calibration board after the checkerboard is replaced;

[0046] an error satisfaction confirmation module, configured to repeatedly replace the checkerboard with the smallest specification in the plurality of checkerboards until the error of the target intrinsic parameter set of the target camera satisfies the requirement.

[0047] Optionally, the camera intrinsic parameter calibration device further comprises:

[0048] a brightness detection module, configured to perform brightness detection on the plurality of checkerboard regions in the calibration board;

[0049] a brightness adjustment module, configured to adjust the scene light source to make the brightness of the plurality of checkerboard regions uniform, the brightness uniformity being that the gray value of the black squares in the plurality of checkerboard regions is within a first range, and the gray value of the white squares is within a second range, wherein the first range is different from the second range;

[0050] a graying processing module, configured to perform graying processing on the photo of the calibration board, and perform intrinsic parameter calibration on the target camera after adjusting the brightness of the photo.

[0051] The third aspect of the embodiments of the present application provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is executed by the processor to implement the camera intrinsic parameter calibration method according to the first aspect of the embodiments of the present application.

[0052] The fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the camera intrinsic parameter calibration method according to the first aspect of the embodiments of the present application.

[0053] The fifth aspect of the embodiments of the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the camera intrinsic parameter calibration method according to the first aspect of the embodiments of the present application.

[0054] According to the camera intrinsic parameter calibration method of the embodiments of the present application, first, a target camera to be calibrated captures a photo of a calibration board, wherein the calibration board is different from the calibration board in the related art which only contains one checkerboard, the captured calibration board contains multiple checkerboards of different specifications and does not occlude each other, and all the checkerboards in the calibration board should be covered in the field of view of the camera; then the target camera is calibrated according to each checkerboard to obtain multiple calculation intrinsic parameter groups calculated according to different checkerboards; finally, each calculation intrinsic parameter group is fused and calculated according to the target weight value of the corresponding checkerboard, and the target intrinsic parameter group of the target camera is finally fused and calculated according to the calculation intrinsic parameter group of each checkerboard, so as to complete the intrinsic parameter calibration of the target camera.

[0055] In the present application, unlike the previous method of camera intrinsic parameter calibration by multiple shooting of one checkerboard, only one designed calibration board is needed, and multiple calculation results are obtained by one shooting according to the multiple checkerboards of different specifications contained in the calibration board, and then the calibration of the target camera is completed by fusion calculation according to the weight of each checkerboard. This method reduces the time and effort consumed by multiple shooting of the calibration board, greatly improves the efficiency of camera intrinsic parameter calibration; and since the calibration board used contains multiple checkerboards of different sizes, only one positioning is needed to contain all the checkerboards in the calibration board in the field of view during the intrinsic parameter calibration of the camera with different field of view sizes, so as to realize the intrinsic parameter calibration of the camera without constantly changing the relative position between the camera and the calibration board to meet the field of view range requirement, greatly improving the universality of the camera intrinsic parameter calibration method proposed in the present application. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description only some of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor under the premise of these drawings.

[0057] Figure 1 is a flow chart of a camera intrinsic parameter calibration method according to an embodiment of the present application;

[0058] Figure 2 is a schematic diagram of a calibration board photograph according to an embodiment of the present application;

[0059] Figure 3 is a structural block diagram of a camera intrinsic parameter calibration device according to an embodiment of the present application;

[0060] Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0062] In the drawings, sometimes the size of the constituent elements, the thickness of the layers or the regions are exaggerated for the sake of clarity, and therefore, any one of the implementations of the present disclosure is not necessarily limited to the size shown in the drawings. The shapes and the sizes of the components in the drawings do not reflect the actual proportions of the components. Moreover, the drawings schematically show ideal examples, and any one of the implementations of the present disclosure is not limited to the shapes or the values shown in the drawings.

[0063] In the related art, in the process of calibrating the intrinsic parameters of a camera, a calibration board containing a checkerboard is usually used as a reference, wherein there is only one checkerboard on the calibration board, and by taking multiple photographs of the calibration board by the camera to be calibrated at different angles, photographs containing all the corner points on the checkerboard in the calibration board are obtained, and the effective checkerboard corner points are identified multiple times, and the pixel corner point pairs in the effective photographs corresponding to the actual corner points are accumulated, and based on the multiple point pairs in the multiple frames of the calibration images at different angles or positions, the intrinsic parameter calibration of the camera is realized. However, it can be obviously found through this process that since the calibration board needs to be photographed multiple times, and each photographed image needs to be processed, a lot of time and effort of the relevant personnel will be consumed to complete the calibration of one camera.

[0064] Therefore, in order to at least partially solve one or more of the above-mentioned problems and other potential problems, this application proposes a camera intrinsic parameter calibration method, which can not only quickly and efficiently complete the intrinsic parameter calibration of the target camera, but also meet the accuracy of the intrinsic parameter calibration. The method proposed in this application also has a certain degree of versatility in the case of calibrating cameras with different pixels.

[0065] Please refer to the details. Figure 1 , Figure 1 This is a flowchart illustrating a camera intrinsic parameter calibration method according to an embodiment of this application. Figure 1 As shown, the method may include steps S101 to S103:

[0066] Step S101: Take a picture of the calibration board with the target camera to be calibrated. The calibration board contains multiple chessboard squares of different sizes that do not obstruct each other. The field of view of the camera covers each of the chessboard squares.

[0067] Step S102: Calculate each set of computational intrinsic parameters of the target camera according to each chessboard grid;

[0068] Step S103: The calculated intrinsic parameter groups are fused according to the target weight values ​​of the corresponding chessboard squares to determine the target intrinsic parameter group of the target camera.

[0069] In this embodiment, multiple checkerboard patterns of different sizes are set on the same calibration board, and each pattern does not obstruct the others. A target camera takes a single image of the calibration board, encompassing all the checkerboard patterns within its field of view. This results in a single photograph containing multiple checkerboard patterns suitable for camera intrinsic parameter calibration. Next, the intrinsic parameters of the target camera are calibrated for each checkerboard pattern, calculating the results corresponding to each checkerboard pattern, i.e., each set of calculated intrinsic parameters. Finally, based on the target weight values ​​of the checkerboard patterns corresponding to each set of calculated intrinsic parameters, the calculation results are fused, and the multiple sets of calculated intrinsic parameters are ultimately merged into a target intrinsic parameter set, thereby achieving the calibration of the target camera.

[0070] In an alternative embodiment, after the calibration of the camera is completed, the calibration result can also be verified by an error calculation algorithm to determine whether it is accurate, for example, by using a re-projection method to calculate the error of the calibration result. The real-world coordinates of any non-corner point on any checkerboard are converted into pixel coordinates according to the calibrated camera intrinsic parameters, and the Euclidean distance between the pixel coordinates and the true pixel coordinates measured by manual or other high-precision methods (such as laser trackers, etc.) is calculated. The smaller the Euclidean distance between the two, the smaller the error and the more accurate the calibration result. By verifying the calibration result in this way, the accuracy of the calibration result is ensured, and when the result deviates, appropriate adjustments can be made in time.

[0071] Step S101: Take a photo of the calibration board by the target camera to be calibrated, the calibration board containing a plurality of different specifications and non-occluded checkerboards, and the field of view of the camera covering each of the checkerboards.

[0072] In the embodiments of the present application, the selected calibration board needs to be designed in advance according to the actual application scenario and needs to determine the size and specification of the checkerboard in the calibration board. It not only needs to meet the selection of different specifications of the checkerboard in the calibration board, but also needs to ensure that the size of the checkerboard is appropriate. At the same time, it also needs to ensure that the complete image of all checkerboards is included in a photo. Moreover, there needs to be a certain distance between the checkerboards in the calibration board to ensure that the checkerboards do not occlude each other, avoiding the confusion of corner point recognition results due to adjacent checkerboards, thereby causing calculation errors. In order to make the subsequent recognition result more accurate, the checkerboard needs to ensure that the black and white square boundaries are clear and the square size is accurate, so as to facilitate the detection of the corner points in the checkerboard.

[0073] Before taking a photo of the calibration board, the calibration board needs to be placed or fixed on a certain plane to ensure the stability of the position of the calibration board and to make each checkerboard in the calibration board as horizontal as possible to facilitate the shooting. Before shooting, the distance and angle between the target camera and the calibration board need to be adjusted so that the photo can cover all the images of each checkerboard in the calibration board, ensuring that the photo clearly contains all the corner points of each checkerboard to facilitate the detection of the corner points of the checkerboard in the calibration board.

[0074] In an alternative embodiment, for the purpose of improving the detection efficiency, the calculation time of the calibration result can be saved by shooting the calibration board directly in front of the camera to be calibrated, for example, placing the calibration board on a horizontal plane and shooting the calibration board vertically to the horizontal plane by the camera to be calibrated. In this way, during the calculation according to the calibration algorithm, the coordinate change in the z-axis direction does not need to be considered, thereby saving the calculation time and further improving the efficiency of the internal parameter calibration of the target camera.

[0075] Step S102: calculating each set of internal parameters of the target camera according to each checkerboard.

[0076] In the embodiment of the present application, after the shooting of the calibration board photo is completed, the calibration board photo containing a plurality of checkerboard images is obtained, and a set of internal parameter calculation groups of the target camera can be correspondingly calculated according to each checkerboard. Specifically, as shown in Figure 2 Figure 2 is a schematic diagram of a calibration board photo according to an embodiment of the present application, which contains five checkerboards of different sizes. During the processing, first, an image processing software such as OpenCV (cross-platform computer vision library), MATLAB (matrix laboratory) and the like is used to process the photo, and a robust and accurate corner detection algorithm such as find Chessboard Corners (a function for finding the inner corner points of the checkerboard image) in OpenCV is used to identify the corner points of the checkerboard in the image, i.e., the junction points of the black and white squares in each checkerboard. These corner points have a determined real-world coordinate when the calibration board is designed, and the real-world coordinate can also be determined by a high-precision measuring tool such as a caliper after the calibration board is designed and manufactured. Then, the pixel coordinates of each detected corner point in the photo are extracted, so as to determine the real-world coordinate and the pixel coordinate of the same corner point for calculating the internal parameters of the camera. In an alternative embodiment, a plurality of lists can be set according to the number of corner points in different checkerboards, the length of the list is the number of corner points in the checkerboard + 1, so as to refer to the corresponding checkerboard, and the real-world coordinate and the pixel coordinate of each checkerboard are stored in the world coordinate list and the pixel coordinate list respectively, which is convenient for subsequent calculation.

[0077] After the coordinates of each corner point are determined, the internal parameter group of the target camera also needs to be calculated by a calibration algorithm according to the coordinates of the corner points. In an alternative embodiment, Zhang Zhengyou calibration method can be selected, and the homography transformation from the two-dimensional plane to the image plane is used to solve each internal parameter of the camera by the photo of the calibration board shot by the target camera.

[0078] ​Step S103: fusing the respective calculation intrinsic parameter sets according to the target weight values of the respective checkerboards to determine a target intrinsic parameter set of the target camera.

[0079] In the embodiments of the present application, after the plurality of calculation intrinsic parameter sets are calculated according to the respective checkerboards, the plurality of calculation intrinsic parameter sets can be fused according to the target weight values of the respective checkerboards to obtain a fused result, i.e., the target intrinsic parameter set of the target camera, thereby realizing the intrinsic parameter calibration of the target camera. In this way, the plurality of checkerboards in one calibration board are used to replace the way of taking pictures of the same checkerboard in the same calibration board for multiple times in the related art, and only one taking can obtain the calculation results of the plurality of target camera intrinsic parameter sets, and then the fusion of the calculation results is realized according to the relationship between the respective checkerboards on the calibration board, thereby realizing the purpose of efficiently and accurately calibrating the intrinsic parameters of the target camera.

[0080] In combination with the above embodiments, in an implementation manner, the present application further provides a camera intrinsic parameter calibration method, and the method further includes the following contents:

[0081] Firstly, the weight parameters of the respective checkerboards are obtained.

[0082] In the embodiments of the present application, when the weights of the checkerboards in the calibration board are set, two aspects are mainly considered. The first aspect is the proportion of the area of the image of each checkerboard in the photo taken by the target camera. Generally, the greater the area of the image of the checkerboard in the photo, the greater the influence of the checkerboard on the intrinsic parameter calibration of the camera, and the greater the weight of the checkerboard. Conversely, the smaller the area of the image of the checkerboard in the photo, the smaller the influence of the checkerboard on the intrinsic parameter calibration of the camera, and the smaller the weight of the checkerboard. The second aspect is the distance between the center point of the photo, i.e., the imaging center point, and each checkerboard. Generally, the farther the image from the imaging center point, the greater the distortion of the image. Therefore, the farther the image of the checkerboard from the imaging center point, the greater the weight of the checkerboard, and the closer the image of the checkerboard to the imaging center point, the smaller the weight of the checkerboard.

[0083] For the above two aspects, a plurality of weight parameters are required to be obtained for calculating the weight values of the respective checkerboards. Specifically, the weight parameters can include but are not limited to: an imaging ratio, i.e., a ratio between an area occupied by each checkerboard and a total area of an image in a photo used for target camera intrinsic parameter detection; a number of corner points, i.e., a total number of corner points contained in each checkerboard; a maximum corner point distance, i.e., a distance between a maximum corner point and an imaging center, the maximum corner point being a corner point farthest from the imaging center in a checkerboard; a square size, i.e., an actual length size of black and white squares in each checkerboard; and a maximum corner point size, i.e., a minimum size of each checkerboard that can contain all corner points.

[0084] Then, according to the weight parameters of the respective checkerboards, target weight values of the respective checkerboards are calculated.

[0085] In the embodiments of the present application, after the plurality of weight parameters used for calculating the weight values of the respective checkerboards are obtained, the target weight values of the respective checkerboards can be calculated according to the weight parameters. The target weight values of the respective checkerboards are a measure of the importance of the calculated intrinsic parameter sets according to the respective checkerboards, and comprehensively consider the importance of the areas occupied by the respective checkerboards in the photo and the positions of the respective checkerboards. By adding the condition of the target weight values, the calculated intrinsic parameter sets corresponding to the plurality of checkerboards in the same photo are reasonably combined with each other, and the intrinsic parameter calibration of the target camera is realized by taking a photo of the plurality of checkerboards at one time.

[0086] In combination with the above embodiments, in an implementation manner, the present application further provides a camera intrinsic parameter calibration method, which calculates target weight values of the respective checkerboards according to weight parameters of the respective checkerboards, and specifically includes the following contents:

[0087] First, first weight values of the respective checkerboards are calculated according to a first parameter group, and second weight values of the respective checkerboards are calculated according to a second parameter group under a first precision requirement, the first parameter group at least including a first weight parameter and a second weight parameter, and the second parameter group at least including a third weight parameter.

[0088] In the embodiments of the present application, after the weight parameters of each checkerboard are determined, the target weight values of each checkerboard need to be calculated according to the weight parameters. Since the consideration of the weight values of the checkerboard is divided into two aspects, the weight values of the two aspects are calculated respectively. Specifically, the first weight value considering the area occupied by the checkerboard needs to be calculated according to the first parameter group, which includes the first weight parameter and the second weight parameter. In an optional embodiment, the first weight parameter can be the number of corner points, denoted as q, and the second weight parameter can be the imaging ratio, denoted as r. Specifically, the scale factor representing the distribution of corner points in the checkerboard is calculated from the square root of the number of corner points. The calculation of the first weight value can be shown in the following formula:

[0089]

[0090] wherein w pr is the first weight value, k represents the number of the checkerboard, and takes the value of 1 to n, n is the total number of checkerboards in the calibration board, q(k) is the number of corner points of the kth checkerboard, and r(k) is the imaging ratio of the kth checkerboard.

[0091] The second weight value considering the size distance of the checkerboard and the imaging center needs to be calculated according to the second parameter group, which includes the third weight parameter. In an optional embodiment, the third weight parameter can be the maximum corner point distance, denoted as d. The calculation of the second weight value can be shown in the following formula:

[0092] w ex1 =d(k)

[0093] wherein w ex1 is the second weight value, k represents the number of the checkerboard, and takes the value of 1 to n, n is the total number of checkerboards in the calibration board, d(k) is the maximum corner point distance of the kth checkerboard.

[0094] Then, the target weight value is determined according to the first weight value and the second weight value.

[0095] In the embodiments of the present application, after the first weight value and the second weight value corresponding to each checkerboard are determined, the target weight value of each checkerboard can be calculated and determined according to the first weight value and the second weight value. In an optional embodiment, the target weight value of each checkerboard is determined by the product of the first weight value and the second weight value. The calculation of the target weight value can be shown in the following formula:

[0096] w=w pr ×w ex1

[0097] wherein w is the target weight value, w pr is the first weight value, and w ex1 is the second weight value.

[0098] Finally, under a second accuracy requirement, the first weight value of each chessboard is calculated according to the first parameter set, and the third weight value of each chessboard is calculated according to a third parameter set, the third parameter set at least including a third weight parameter and a fourth weight parameter; the target weight value is determined according to the first weight value and the third weight value; wherein the second accuracy requirement is higher than the first accuracy requirement.

[0099] In the embodiments of the present application, in the case of higher accuracy requirement for the calculation result, the calculation result of the internal parameter set corresponding to each chessboard can be more accurately weighted and fused by introducing more weight parameters in the calculation process, so as to realize more accurate calculation according to the photographed photo, and to improve the accuracy of the target internal parameter set determined finally. Specifically, under a second accuracy requirement higher than the first accuracy requirement, when the second weight is considered, the third weight value introducing the fourth weight parameter is used to replace the previous first weight value for the final target weight value calculation. In an optional embodiment, the fourth weight parameter can be the maximum corner size, denoted as l, and the calculation of the third weight value can be as follows:

[0100] w ex2 =d(k) / l(k)

[0101] wherein w ex2 is the third weight value, k represents the number of the chessboard, taking the value of 1 to n, n is the total number of the chessboards in the calibration board, d(k) is the maximum corner distance of the kth chessboard, and l(k) is the maximum corner size of the kth chessboard.

[0102] After the first weight value and the third weight value corresponding to each chessboard are determined, the target weight value of each chessboard can be calculated and determined according to the first weight value and the third weight value. In an optional embodiment, the target weight value of each chessboard is determined by the product of the first weight value and the third weight value, and the calculation of the target weight value can be as follows:

[0103] w=w pr ×w ex2

[0104] wherein w is the target weight value, w pr is the first weight value, and w ex2 is the third weight value.

[0105] In the embodiments of the present application, after the target weight values of the chessboard grids are determined, the target intrinsic parameter group of the target camera can be determined according to the target weight values of the chessboard grids and the calculated intrinsic parameter groups of the chessboard grids. In an optional embodiment, the calculation process of the target intrinsic parameter group can be shown in the following formula:

[0106]

[0107] wherein, P mean is the target intrinsic parameter group, k represents the number of the chessboard grid, and takes a value of 1 to n, n is the total number of the chessboard grids in the calibration board, w is the target weight value, and p(k) is the calculated intrinsic parameter group of the kth chessboard grid.

[0108] In combination with the above embodiments, in an implementation, the present application further provides a camera intrinsic parameter calibration method, under a first precision requirement, a second weight value of each chessboard grid is calculated according to a second parameter group, which specifically includes the following contents:

[0109] When the sizes of the squares in the chessboard grids are inconsistent, the second parameter group further includes a fifth weight parameter.

[0110] In the embodiments of the present application, there are also situations that the side lengths of the black and white squares in each chessboard grid for intrinsic parameter calibration in the calibration board are inconsistent. When the side lengths of the black and white squares in each chessboard grid are inconsistent, the calculation process of the second weight value considering the size distance of the chessboard grid and the imaging center will be affected to a certain extent when calculating the target weight value of each chessboard grid, and the weight parameter representing the side length of the black and white squares, i.e., the fifth weight parameter, needs to be considered. For the third parameter group, the fourth weight parameter is already included, and the fourth weight parameter can replace the fifth weight parameter in the process of calculating the weight value, so only the calculation of the second parameter group is considered in this case. In an optional embodiment, the fifth weight parameter can be the square size, represented by s, and the calculation of the second weight value can be shown in the following formula:

[0111] w ex1 = d(k) / s(k)

[0112] wherein, w ex1 is the second weight value, k represents the number of the chessboard grid, and takes a value of 1 to n, n is the total number of the chessboard grids in the calibration board, d(k) is the maximum angular point distance of the kth chessboard grid, and s(k) is the square size of the kth chessboard grid.

[0113] In combination with the above embodiments, in an implementation, the present application further provides a camera intrinsic parameter calibration method, after the target intrinsic parameter group of the target camera is determined, when the error of the target intrinsic parameter group does not meet the requirement, specifically including the following contents:

[0114] Firstly, the chessboard with the smallest size in the various chessboards is replaced by a chessboard with a larger size and different from the sizes of the other chessboards, and the chessboard with the smallest size is the chessboard with the least number of corner points in the various chessboards.

[0115] In the embodiments of the present application, after the target camera is determined, if it is found that the error of the target intrinsic parameter group calculated does not meet the use requirements, there are some corresponding optimization solutions to improve the accuracy of the target intrinsic parameter group calculated. Specifically, the optimization can be performed by adjusting the size of the chessboard. As described above, the image of the chessboard with a larger area in the photographed picture has a larger weight ratio. From the same angle, when the error of the calibration result does not meet the requirements, the smaller chessboard in the calibration plate is replaced by a chessboard with a larger size and different from the sizes of the other chessboards, so as to increase the weight of the relatively more accurate calculation intrinsic parameter group, thereby improving the accuracy of the target intrinsic parameter group finally fused. In the present application, the chessboard with the smallest size in the various chessboards is replaced by a chessboard with a larger size and different from the sizes of the other chessboards, wherein the chessboard with the smallest size refers to the chessboard with the least number of corner points in the various chessboards. As described above, the calibration plate contains a plurality of chessboards with different sizes. When the replacement is performed, the replacement is started from the chessboard with the smallest size. For example, as shown in the above-mentioned case, the calibration plate contains five chessboards with sizes of 5x8, 7x7, 6x9, 9x11 and 10x10. When the replacement is needed, the chessboard with the smallest size, i.e., the chessboard with the size of 5x8, is replaced by a chessboard with a larger size. For example, the chessboard with the size of 8x8 can be replaced. If the error of the target intrinsic parameter group obtained subsequently still does not meet the use requirements, the chessboard with the smallest size at this time, i.e., the chessboard with the size of 7x7, is replaced by a chessboard with a larger size, such as the chessboard with the size of 7x9, until the error of the target intrinsic parameter group meets the requirements. Figure 2

[0116] Then, the intrinsic parameter calibration of the target camera is performed again according to the calibration plate after the replacement of the chessboard.

[0117] In the embodiments of the present application, after the chessboard in the calibration plate is replaced, the intrinsic parameter calibration of the target camera is performed again according to the method proposed in the present application, the calculation of the calculation intrinsic parameter group corresponding to each chessboard is performed by photographing the picture, and finally the target intrinsic parameter group is determined according to the target weight value fusion. Whether the result of the intrinsic parameter calibration this time meets the error requirements of the user is determined, and if it meets the requirements, the calibration result this time is taken as the final calibration result.

[0118] ​Finally, the chessboard grid with the smallest size is repeatedly replaced until the error of the target intrinsic parameter group of the target camera meets the requirement.

[0119] In the embodiments of the present application, if the error requirement is still not met after the replacement of the chessboard grid, the smallest chessboard grid in the calibration board is replaced again, and calibration is performed again. Through the repeated replacement, the error of the target intrinsic parameter group meets the error requirement of the user, and the calibration result is taken as the final calibration result.

[0120] In combination with the above embodiments, in an implementation, the present application further provides a camera intrinsic parameter calibration method. After the target intrinsic parameter group of the target camera is determined, if the error of the target intrinsic parameter group does not meet the requirement, the method specifically includes the following contents:

[0121] First, the brightness of each chessboard grid region in the calibration board is detected.

[0122] In the embodiments of the present application, if the error of the target intrinsic parameter group does not meet the requirement, the error of the calibration can be reduced by adjusting the environmental factors. Specifically, since the calibration of the camera is performed by photographing the target image, the ambient light source will affect the detection result. Specifically, since the calibration of the target camera is achieved by calculating each chessboard grid in the calibration board, the brightness of each chessboard grid region in the calibration board needs to be detected to observe whether the brightness of a certain chessboard grid region is too dark or too bright to affect the detection result.

[0123] Then, the scene light source is adjusted to make the brightness of each chessboard grid region uniform. The uniform brightness means that the gray value of the black grid in each chessboard grid region is within a first range, and the gray value of the white grid is within a second range. The first range is different from the second range.

[0124] In the embodiment of the present application, when taking a photo of the calibration board by the target camera to be calibrated, the light distribution in the taken photo should be as consistent as possible to avoid local over-brightness or over-darkness affecting the calibration result. Specifically, when recognizing the corner points in the photo by a corner detection algorithm, if there is an over-bright area, the white squares in the checkerboard will be saturated and the corner point boundary will be blurred. If there is an over-dark area, the black squares in the checkerboard will be difficult to distinguish from the background, resulting in missing detection of the corner points. Therefore, after detecting the brightness of each checkerboard region, the brightness of each checkerboard region can be adjusted to be uniform and appropriate by adjusting the scene light source. The uniform brightness means that the gray value of the black squares in each checkerboard region is within a first range, and the gray value of the white squares is within a second range, wherein the first range is different from the second range. Specifically, the gray value of pure black is 0, and the gray value of pure white is 255. Therefore, if the gray value of the black squares in each checkerboard is within the same range, such as 0 to 50, and the gray value of the white squares is within another range, such as 200 to 255, it is considered that the brightness of the checkerboard region is uniform and appropriate. The values of the first range and the second range are determined according to the actual situation, which is not required in the present application. For the adjustment of the scene light source, specifically, the adjustment methods include but are not limited to adjusting the direction of the light source, adjusting the brightness of the light source, adjusting the number of light sources, etc., so that the brightness difference of all regions of each checkerboard is controlled within a small range, such as a difference of no more than 10% of the gray value, and there is no reflection caused by direct light from the light source, and no shadow caused by the light source being blocked, etc.

[0125] Finally, the photo of the calibration board is subjected to a gray-scale processing, and the target camera is subjected to an intrinsic parameter calibration after adjusting the brightness of the photo.

[0126] In the embodiment of the present application, in addition to adjusting the scene light source in the calibration scene, a certain image processing can also be performed on the result taken by the target camera. Specifically, after the photo of the calibration board taken by the target camera is subjected to a gray-scale processing, the brightness is adjusted to make the checkerboard in the image more clear and visible, and the boundary more obvious. The adjusted image can also be used for the intrinsic parameter calibration of the target camera to achieve the purpose of reducing the error of the final calibration result.

[0127] Based on the same design concept, an embodiment of the present application provides a camera intrinsic parameter calibration device. Referring to Figure 3 , Figure 3 is a structural block diagram of the camera intrinsic parameter calibration device provided by an embodiment of the present application.

[0128] As shown in Figure 3 , the device comprises:

[0129] a photographing module configured to photograph a photograph of a calibration board by a target camera to be calibrated, the calibration board comprising a plurality of different specifications and non-occluded checkerboards, and a field of view of the camera covering each of the checkerboards;

[0130] a calculation intrinsic parameter group determination module configured to calculate each of calculation intrinsic parameter groups of the target camera according to each of the checkerboards, respectively;

[0131] a target intrinsic parameter group determination module configured to fuse the calculation intrinsic parameter groups according to target weight values of the corresponding checkerboards to determine a target intrinsic parameter group of the target camera.

[0132] Optionally, the camera intrinsic parameter calibration device further comprises:

[0133] a weight parameter acquisition module configured to acquire weight parameters of the checkerboards;

[0134] a target weight value calculation module configured to calculate target weight values of the checkerboards according to the weight parameters of the checkerboards.

[0135] Optionally, the target weight value calculation module comprises:

[0136] a first precision weight value calculation submodule configured to calculate first weight values of the checkerboards according to a first parameter group and second weight values of the checkerboards according to a second parameter group under a first precision requirement, the first parameter group comprising at least a first weight parameter and a second weight parameter, and the second parameter group comprising at least a third weight parameter;

[0137] a first target weight value calculation submodule configured to determine the target weight values according to the first weight values and the second weight values;

[0138] a second precision weight value calculation submodule configured to calculate the first weight values of the checkerboards according to the first parameter group and third weight values of the checkerboards according to a third parameter group under a second precision requirement, the third parameter group comprising at least a third weight parameter and a fourth weight parameter;

[0139] a second target weight value calculation submodule configured to determine the target weight values according to the first weight values and the third weight values

[0140] Optionally, the first precision weight value calculation submodule comprises:

[0141] a second parameter group calculation unit configured to, when sizes of the squares in the checkerboards are inconsistent, the second parameter group further comprising a fifth weight parameter.

[0142] Optionally, the camera intrinsic parameter calibration device further comprises:

[0143] The chessboard grid replacement module is configured to replace the chessboard grid with the smallest specification among the chessboard grids with the chessboard grid with a larger specification and different from the specifications of the other chessboard grids, the chessboard grid with the smallest specification being the chessboard grid containing the least number of corner points among the chessboard grids.

[0144] The secondary calibration module is configured to perform intrinsic parameter calibration on the target camera again according to the calibration board after the replacement of the chessboard grid.

[0145] The error satisfaction confirmation module is configured to repeatedly replace the chessboard grid with the smallest specification among the chessboard grids until the error of the target intrinsic parameter group of the target camera satisfies the requirement.

[0146] Optionally, the camera intrinsic parameter calibration device further comprises:

[0147] The brightness detection module is configured to perform brightness detection on the chessboard grid regions in the calibration board.

[0148] The brightness adjustment module is configured to adjust the scene light source to make the brightness of the chessboard grid regions uniform, the brightness uniformity being that the gray value of the black grid is within a first range and the gray value of the white grid is within a second range, and the first range being different from the second range.

[0149] The gray processing module is configured to perform gray processing on the photo of the calibration board and perform intrinsic parameter calibration on the target camera after the adjustment of the brightness of the photo.

[0150] Based on the same design concept, another embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in the camera intrinsic parameter calibration method according to any one of the above embodiments of the present application.

[0151] Based on the same design concept, another embodiment of the present application provides a computer program product, which comprises computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the steps in the camera intrinsic parameter calibration method according to any one of the above embodiments of the present application.

[0152] Based on the same design concept, another embodiment of the present application provides an electronic device, as shown in Figure 4 Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present application. The electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the camera intrinsic parameter calibration method according to any one of the above embodiments of the present application when executed.​

[0153] For the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts are referred to the part of the method embodiments.

[0154] Each of the embodiments in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between the embodiments can be referred to each other.

[0155] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, apparatus or computer program product. Therefore, the embodiments of the present application can be in the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. In addition, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the method, terminal device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0157] These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0158] These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are performed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1steps of the functions specified in the one or more blocks.

[0159] While the preferred embodiments of the application have been described above, it should be understood that many modifications and variations to these embodiments will be apparent to those skilled in the art once they learn of the basic inventive concepts. Therefore, the attached claims are intended to cover all such modifications and variations.

[0160] Finally, it is to be understood that the phraseology or terminology employed herein, such as "first" and "second", etc., are for descriptive purposes only and should not be construed to be limiting unless otherwise indicated. It is to be understood that the terms "including", "comprising", or "having" contain for the purposes of disclosure an open term such that the methods or compositions described can include some other elements or steps not expressly named or inherent to such methods or compositions. Absence of such phrases or terms does not preclude the corresponding methods or compositions from having such elements or steps inherent thereto.

[0161] The above provides a camera intrinsic parameter calibration method, device, equipment, medium and product, the principle and implementation mode of the application are described in the text by applying specific examples, the above example is only used to help understand the method and core idea of the application; At the same time, for those skilled in the art, according to the idea of the application, the specific implementation mode and application range will be changed; In view of the above, the content of the specification should not be understood as the limitation of the application.

Claims

1. A camera intrinsic parameter calibration method, characterized in that, The method comprises: Taking a photo of a calibration board by a target camera to be calibrated, the calibration board containing a plurality of different specifications and mutually non-occluded checkerboards, and the field of view of the camera covering each of the checkerboards; According to each checkerboard, calculating a respective calculated intrinsic parameter group of the target camera; Fusing the respective calculated intrinsic parameter groups according to the target weight values of the respective checkerboards to determine a target intrinsic parameter group of the target camera.

2. The camera intrinsic parameter calibration method of claim 1, wherein, The method further comprises: Obtaining weight parameters of the respective checkerboards; According to the weight parameters of the respective checkerboards, calculating target weight values of the respective checkerboards.

3. The camera intrinsic parameter calibration method of claim 2, wherein, According to the weight parameters of the respective checkerboards, calculating target weight values of the respective checkerboards, comprises: Under a first accuracy requirement, calculating first weight values of the respective checkerboards according to a first parameter group, and calculating second weight values of the respective checkerboards according to a second parameter group, the first parameter group at least including a first weight parameter and a second weight parameter, and the second parameter group at least including a third weight parameter; According to the first weight values and the second weight values, determining the target weight values; Under a second accuracy requirement, calculating the first weight values of the respective checkerboards according to the first parameter group, and calculating third weight values of the respective checkerboards according to a third parameter group, the third parameter group at least including a third weight parameter and a fourth weight parameter; According to the first weight values and the third weight values, determining the target weight values; Wherein, the second accuracy requirement is higher than the first accuracy requirement.

4. The camera intrinsic parameter calibration method of claim 3, wherein, Under a first accuracy requirement, calculating second weight values of the respective checkerboards according to a second parameter group, comprises: When the sizes of the squares in the respective checkerboards are inconsistent, the second parameter group further includes a fifth weight parameter.

5. The camera intrinsic parameter calibration method of claim 1, wherein, After determining the target intrinsic parameter group of the target camera, when the error of the target intrinsic parameter group does not meet the requirement, the method further comprises: Replacing the checkerboard with the smallest specification in the respective checkerboards with a checkerboard with a larger specification and different from the specifications of the other checkerboards, the checkerboard with the smallest specification being the checkerboard with the least number of corner points in the respective checkerboards; Re-calibrating the intrinsic parameters of the target camera according to the calibration board after replacing the checkerboard; Repeating the replacement of the checkerboard with the smallest specification in the respective checkerboards until the error of the target intrinsic parameter group of the target camera meets the requirement.

6. The camera intrinsic parameter calibration method of claim 1, wherein, After determining the target intrinsic parameter group of the target camera, when the error of the target intrinsic parameter group does not meet the requirement, the method further comprises: Performing brightness detection on the respective checkerboard regions in the calibration board; Adjusting the scene light source to make the brightness of the respective checkerboard regions uniform, the brightness uniformity being that the gray values of the black squares in the respective checkerboard regions are within a first range, and the gray values of the white squares are within a second range, wherein the first range is different from the second range; Performing gray-scale processing on the photo of the calibration board, and re-calibrating the intrinsic parameters of the target camera after adjusting the brightness of the photo.

7. An apparatus for camera intrinsic parameter calibration, characterized in that, The device comprises: A photographing module is configured to take a photograph of a calibration board by a target camera to be calibrated, the calibration board comprising a plurality of different specifications of non-occluded checkerboards, and a field of view of the camera covering each of the checkerboards; A calculation intrinsic parameter group determination module is configured to calculate each calculation intrinsic parameter group of the target camera according to each checkerboard respectively; A target intrinsic parameter group determination module is configured to fuse the calculation intrinsic parameter groups according to target weight values of the corresponding checkerboards to determine a target intrinsic parameter group of the target camera.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program is executed by the processor to implement the camera intrinsic parameter calibration method according to any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the camera intrinsic parameter calibration method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the camera intrinsic parameter calibration method according to any one of claims 1 to 6.

Images