Camera calibration method, device, equipment and computer readable storage medium

Abstract

The application relates to a camera calibration method, device, equipment and computer readable storage medium. The method comprises the following steps: acquiring a first checkerboard image collected by a to-be-calibrated camera for a first checkerboard and a second checkerboard image collected by the to-be-calibrated camera for a second checkerboard, performing corner point detection on the first checkerboard image to obtain first corner points, and performing corner point detection on the second checkerboard image to obtain second corner points; determining an initial calibration model according to first image coordinates and first physical coordinates of the first corner points; determining candidate physical coordinates corresponding to the second corner points according to second image coordinates of the second corner points and the initial calibration model; aligning the second physical coordinates of the second corner points with the candidate physical coordinates to obtain target physical coordinates corresponding to the second physical coordinates; and determining a target calibration model according to the target physical coordinates and the second image coordinates. According to the application, high-precision calibration of a camera can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of device calibration, in particular to a camera calibration method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] In computer vision technology, in order to determine the mapping relationship between the three-dimensional position of a point in the world coordinate system and the two-dimensional position in the image coordinate system, a geometric model of camera imaging needs to be established. The parameters of the geometric model are the camera calibration parameters, and the process of solving the camera calibration parameters is the camera calibration.

[0003] In the traditional technology, the three-dimensional reconstruction method based on images, by shooting a number of unordered images in the room at a number of angles, the camera to be calibrated also shoots an image, combines these images to determine the pose corresponding to all images by using the three-dimensional reconstruction motion structure algorithm, and the accuracy of camera calibration is low. SUMMARY

[0004] Therefore, it is necessary to provide a camera calibration method, device, equipment, computer readable storage medium and computer program product, which can realize high-precision calibration of the camera.

[0005] In a first aspect, the present application provides a camera calibration method, comprising:

[0006] Obtaining a first checkerboard image collected by a camera to be calibrated for a first checkerboard, and a second checkerboard image collected by the camera to be calibrated for a second checkerboard, and performing corner detection on the first checkerboard image to obtain first corners, and performing corner detection on the second checkerboard image to obtain second corners; the accuracy of the second checkerboard is higher than that of the first checkerboard;

[0007] Determining an initial calibration model according to the first image coordinates and the first physical coordinates of the first corners;

[0008] Determining the candidate physical coordinates corresponding to the second corners according to the second image coordinates of the second corners and the initial calibration model;

[0009] Aligning the second physical coordinates of the second corners with the candidate physical coordinates to obtain target physical coordinates corresponding to the second physical coordinates;

[0010] Determining a target calibration model according to the target physical coordinates and the second image coordinates.

[0011] In a second aspect, the present application provides a camera calibration device, comprising:

[0012] an angle point detection module, configured to acquire a first checkerboard image collected by a camera to be calibrated for a first checkerboard and a second checkerboard image collected by the camera to be calibrated for a second checkerboard, and perform angle point detection on the first checkerboard image to obtain first angle points and perform angle point detection on the second checkerboard image to obtain second angle points, wherein the second checkerboard has a higher precision than the first checkerboard;

[0013] an initial model determination module, configured to determine an initial calibration model according to first image coordinates and first physical coordinates of the first angle points;

[0014] a physical coordinate calculation module, configured to determine candidate physical coordinates corresponding to the second angle points according to second image coordinates of the second angle points and the initial calibration model;

[0015] a physical coordinate alignment module, configured to align the second physical coordinates of the second angle points with the candidate physical coordinates to obtain target physical coordinates corresponding to the second physical coordinates;

[0016] a target model determination module, configured to determine a target calibration model according to the target physical coordinates and the second image coordinates.

[0017] In a third aspect, the present application provides a camera calibration device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps in the above method when executing the computer program.

[0018] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0019] In a fifth aspect, the present application provides a computer program product, which comprises a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0020] The above camera calibration method, device, equipment, computer readable storage medium and computer program product calibrate the camera to be calibrated by using checkerboard images collected by the camera to be calibrated for first and second checkerboards with different precisions, can realize high-precision calibration of the camera to be calibrated by collecting the first checkerboard image of the first checkerboard with lower precision to obtain an initial calibration model, and then optimizing the initial calibration model by collecting the second checkerboard image of the second checkerboard with higher precision to obtain a target calibration model with higher precision, thereby realizing high-precision positioning measurement of the target. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1An application environment diagram of a camera calibration method provided by an embodiment of the present application;

[0022] Figure 2 A flowchart of a camera calibration method provided by an embodiment of the present application;

[0023] Figure 3 A calibration chessboard diagram provided by an embodiment of the present application;

[0024] Figure 4 A chessboard corner point diagram provided by an embodiment of the present application;

[0025] Figure 5 A chessboard diagram including a marker code provided by an embodiment of the present application;

[0026] Figure 6 A flowchart of another camera calibration method provided by an embodiment of the present application;

[0027] Figure 7 A structural block diagram of a camera calibration device provided by an embodiment of the present application;

[0028] Figure 8 An internal structure diagram of a camera calibration device provided by an embodiment of the present application;

[0029] Figure 9 An internal structure diagram of another camera calibration device provided by an embodiment of the present application;

[0030] Figure 10 An internal structure diagram of a computer readable storage medium provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0032] The camera calibration method provided by the embodiments of the present application can be applied to, for example, Figure 1The application environment shown. Among them, the camera to be calibrated 102 can collect a first checkerboard image for the first checkerboard, and then place the second checkerboard at the position of the first checkerboard, or place the second checkerboard and the first checkerboard on the same plane, and the accuracy of the second checkerboard is higher than that of the first checkerboard; the camera to be calibrated 102 collects a second checkerboard image for the second checkerboard, and then the camera to be calibrated 102 can transmit the first checkerboard image and the second checkerboard image collected to the server 104, and the server 104 obtains the first checkerboard image and the second checkerboard image, and then detects the corner points of the first checkerboard image to obtain the first corner points, detects the corner points of the second checkerboard image to obtain the second corner points, determines the initial calibration model according to the first image coordinates and the first physical coordinates of the first corner points, determines the candidate physical coordinates corresponding to the second corner points according to the second image coordinates and the initial calibration model, aligns the second physical coordinates of the second corner points with the candidate physical coordinates to obtain the target physical coordinates corresponding to the second physical coordinates, and determines the target calibration model according to the target physical coordinates and the second image coordinates. It should be noted that the first checkerboard image and the second checkerboard image collected by the camera to be calibrated can also be transmitted to the terminal, and the first checkerboard image and the second checkerboard image are processed through the terminal. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. Alternatively, the processor of the camera to be calibrated can process the first checkerboard image and the second checkerboard image, and finally obtain the target calibration model of the camera to be calibrated, so that the camera to be calibrated can convert the image coordinates of any position in the photographed image into coordinates of the same physical coordinate system.

[0033] As shown in Figure 2 The embodiments of the present application provide a camera calibration method. The method is applied to the server 104 in Figure 1 It can be understood that the camera calibration device can include at least one of a terminal and a server. The method includes the following steps:

[0034] S202, obtaining a first checkerboard image collected by a camera to be calibrated for a first checkerboard, and a second checkerboard image collected by the camera to be calibrated for a second checkerboard, and detecting corner points of the first checkerboard image to obtain first corner points, and detecting corner points of the second checkerboard image to obtain second corner points; wherein the accuracy of the second checkerboard is higher than that of the first checkerboard.

[0035] The to-be-calibrated camera refers to a camera that needs to be calibrated, and the to-be-calibrated camera can include one or more. If the to-be-calibrated camera includes multiple cameras, the multiple to-be-calibrated cameras can respectively capture multiple first chessboard images corresponding to the first chessboard, and respectively capture multiple second chessboard images corresponding to the second chessboard. The first chessboard and the second chessboard are chessboards with different precisions, and the second chessboard has higher precision, for example, can be a professional customized chessboard, or a modified chessboard. For example, a calibration chessboard diagram is shown in Figure 3 The precision of the second chessboard can be higher than a precision threshold, and the precision threshold can include at least one of a scale threshold, a straightness threshold, or a flatness threshold. The scale threshold is a threshold representing the accuracy of the chessboard scale, the straightness threshold is a threshold representing the straightness of the straight line in the chessboard, and the flatness threshold is a threshold representing the flatness of the surface of the chessboard. The precision of the second chessboard is higher than the precision of the first chessboard, for example, at least one of the chessboard scale precision, the straightness, or the surface flatness of the second chessboard is higher than that of the first chessboard. For example, the chessboard scale precision, the straightness, and the surface flatness of the second chessboard are all higher than those of the first chessboard, which indicates that the precision of the second chessboard is higher than that of the first chessboard.

[0036] The first chessboard image is an image captured by the to-be-calibrated camera on the first chessboard, and the first chessboard image can be an image captured by the to-be-calibrated camera on the entire first chessboard, or can be an image captured on part of the first chessboard. Correspondingly, the second chessboard image is an image captured by the to-be-calibrated camera on the second chessboard, and the second chessboard image can be an image captured by the to-be-calibrated camera on the second chessboard, or can be an image captured on part of the second chessboard. In actual application scenarios, the to-be-calibrated camera can first capture the first chessboard image on the first chessboard, then place the second chessboard at the position of the first chessboard, or cover the first chessboard with the second chessboard, and then capture the second chessboard image on the second chessboard. Alternatively, the first chessboard and the second chessboard can be placed on the same plane, and the to-be-calibrated camera can capture the first chessboard image and the second chessboard image in sequence. For example, the first chessboard and the second chessboard both include marker codes, and in the same chessboard, the marker codes can include multiple marker codes, and each marker code represents the physical coordinate information corresponding to the position of the corresponding marker code. When the to-be-calibrated camera captures the image of the first chessboard or the second chessboard, the corresponding marker code needs to be captured, that is, the first chessboard image and the second chessboard image both include the corresponding marker code information.

[0037] In an exemplary embodiment, corner detection algorithms can be used to detect corners in a checkerboard image, and the detected corners are as follows: Figure 4 As shown (the top corner is circled in black in the image). A corner point can be the intersection of two lines or a point located on two adjacent objects with different main directions. Corner detection algorithms can include at least one of the following operators: Harris operator, Moravec operator, Forstner operator, or Susan operator. Alternatively, corner detection can be performed using OpenCV's `findChessboardCorners` function. A corner detection algorithm can be used to detect corners in the first chessboard image to obtain the first corner point, and then the same algorithm can be used to detect corners in the second chessboard image to obtain the second corner point. That is, the first corner point is a corner point in the first chessboard image, and the second corner point is a corner point in the second chessboard image. It is easy to understand that the same or different corner detection algorithms can be used to detect corners in both the first and second chessboard images to obtain the corresponding first and second corner points.

[0038] S204. Determine the initial calibration model based on the first image coordinates and the first physical coordinates of the first corner point.

[0039] Here, the first image coordinates refer to the image coordinates of the first corner point in the first checkerboard image, and the first physical coordinates refer to the physical coordinates of the first corner point. Physical coordinates are coordinates on the physical imaging plane, essentially describing the position of a pixel using physical units; they are two-dimensional coordinates. The physical coordinate system can be set according to the actual application scenario; that is, the origin and coordinate axis directions of the physical coordinate system can be set according to actual needs. The initial calibration model is a calibration model determined based on the first image coordinates and the first physical coordinates. Image coordinates refer to the coordinates of the image being captured. In simple terms, image coordinates are easily obtained from the acquired images; that is, the first image coordinates can be obtained from the first checkerboard image, the second image coordinates can be obtained from the second checkerboard image, and images acquired by the same camera have the same image coordinate system.

[0040] For example, an initial calibration model can be determined based on multiple first image coordinates and multiple first physical coordinates of multiple first corner points. Specifically, a transformation relationship between the first image coordinates and the first physical coordinates can be determined based on the multiple first image coordinates and the multiple first physical coordinates. The initial calibration model is then determined based on the transformation relationship and a calibration difference condition. The calibration difference condition is used to constrain the accuracy of the initial calibration model. For example, the calibration difference condition is that the difference between the calculated image coordinates corresponding to the physical coordinates and the actual image coordinates is less than a difference threshold.

[0041] S206, determining the candidate physical coordinate corresponding to the second corner point according to the second image coordinate of the second corner point and the initial calibration model.

[0042] The second image coordinate refers to the image coordinate of the second corner point in the second checkerboard image. Specifically, the second image coordinate of the second corner point can be input into the initial calibration model to obtain the candidate physical coordinate corresponding to the second corner point. It is easy to understand that the candidate physical coordinate calculated according to the initial calibration model can be the same as the second physical coordinate of the second corner point, or can be different.

[0043] S208, aligning the second physical coordinate of the second corner point with the candidate physical coordinate to obtain the target physical coordinate corresponding to the second physical coordinate.

[0044] The alignment of the second physical coordinate of the second corner point with the candidate physical coordinate is equivalent to the alignment of the physical coordinate system of the second corner point with the physical coordinate system of the first corner point. For example, the second physical coordinate and the candidate physical coordinate can be aligned through a certain rotation operation and translation operation to obtain the aligned target physical coordinate.

[0045] Exemplarily, the alignment vector can be determined according to the second physical coordinate and the candidate physical coordinate, and then the target physical coordinate corresponding to the second physical coordinate can be obtained according to the second physical coordinate and the alignment vector.

[0046] S210, determining the target calibration model according to the target physical coordinate and the second image coordinate.

[0047] The accuracy of the target calibration model is usually higher than that of the initial calibration model. After the camera to be calibrated is calibrated through the target calibration model, the image coordinates of different images can be converted into physical coordinates in the same physical coordinate system, and high-precision measurement can be realized.

[0048] It is easy to understand that the way of determining the target calibration model according to the target physical coordinate and the second image coordinate can be implemented by referring to the way of determining the initial calibration model according to the first image coordinate of the first corner point and the first physical coordinate, and replacing the first physical coordinate with the target physical coordinate and replacing the first image coordinate with the second image coordinate.

[0049] It can be seen that, in the embodiment of the application, the calibration of the to-be-calibrated camera is performed by using the checkerboard images obtained by sampling the first checkerboard and the second checkerboard with different precisions, so that the initial calibration model is obtained by using the first checkerboard image obtained by collecting the first checkerboard with lower precision, then the initial calibration model is optimized by using the second checkerboard image obtained by collecting the second checkerboard with higher precision, and the target calibration model with higher precision is obtained, thereby realizing high-precision calibration of the to-be-calibrated camera and high-precision positioning measurement of the measurement target.

[0050] In some embodiments, the aligning of the second physical coordinates of the second corner points with the candidate physical coordinates in S208 obtains target physical coordinates corresponding to the second physical coordinates, including:

[0051] The second physical coordinates of the second corner points are fitted with the candidate physical coordinates to obtain a coordinate fitting relationship; the target rotation amount and the target translation amount are determined according to the coordinate fitting relationship; and the second physical coordinates are aligned with the candidate physical coordinates according to the target rotation amount and the target translation amount to obtain the target physical coordinates corresponding to the second physical coordinates.

[0052] The coordinate fitting relationship is used to represent the conversion relationship between the first physical coordinates and the candidate physical coordinates. The target rotation amount and the target translation amount are alignment parameters for aligning the second physical coordinates with the candidate physical coordinates.

[0053] Exemplarily, the relationship between the second physical coordinates of the second corner points and the candidate physical coordinates can be fitted in a rigid transformation or a non-rigid transformation manner, so as to obtain the coordinate fitting relationship. The target rotation amount and the target translation amount can be determined from the coordinate fitting relationship. After the second physical coordinates are rotated by the target rotation amount and translated by the target translation amount, the target physical coordinates corresponding to the second physical coordinates are obtained, that is, the alignment of the second physical coordinates with the candidate physical coordinates is realized.

[0054] In an exemplary embodiment, the second physical coordinates and the candidate physical coordinates of the plurality of second corner points are fitted by a 2D (two-dimensional) rigid transformation, and the coordinate fitting relationship is as shown in the following formula (1).

[0055] Formula (1)

[0056] Wherein, R represents the target rotation amount, t represents the target translation amount, N represents the number of second corner points, i represents the i-th second corner point, represents the second physical coordinates of the i-th second corner point, represents the candidate physical coordinates of the i-th second corner point. It should be noted that the target rotation amount includes the angle and direction of rotation, that is, the target rotation amount can be represented in the form of a rotation vector.

[0057] According to the coordinate fitting relationship in formula (1), the target rotation amount R and the target translation amount t can be obtained, and thus according to R and t, the target physical coordinates corresponding to the specific second physical coordinates of the second corner points can be determined, for example, the target physical coordinates of the jth second corner point corresponding target physical coordinates The target physical coordinates can be determined by formula (2) as follows.

[0058] Formula (2)

[0059] It should be noted that the specific fitting process of the coordinate fitting relationship can be selected according to the actual application scenario, which is not limited here.

[0060] As can be seen, in the embodiment, by fitting the second physical coordinates of the second corner points with the candidate physical coordinates, the target rotation amount and the target translation amount for aligning the second physical coordinates of the second corner points with the candidate physical coordinates are obtained, and thus the target physical coordinates after alignment are obtained by aligning according to the target rotation amount and the target translation amount, which can accurately align the second physical coordinates with the candidate physical coordinates.

[0061] In some embodiments, the first corner points include a plurality of; according to the first image coordinates and the first physical coordinates of the first corner points, an initial calibration model is determined, including:

[0062] According to the first image coordinates and the first physical coordinates of each first corner point, a conversion relationship between the image coordinates and the physical coordinates is determined; according to the conversion relationship and the first physical coordinates, candidate image coordinates corresponding to the first physical coordinates are determined; and according to the difference between the candidate image coordinates and the first image coordinates, the conversion relationship is optimized to obtain the initial calibration model.

[0063] Exemplarily, the plurality of first image coordinates and the plurality of first physical coordinates of the plurality of first corner points can be calibrated to determine the conversion relationship between the image coordinates and the physical coordinates, the first physical coordinates are substituted into the conversion relationship to obtain the candidate image coordinates corresponding to the first physical coordinates, and the parameters of the conversion relationship are optimized according to the difference between the candidate image coordinates and the first image coordinates, so that the difference between the candidate image coordinates and the first image coordinates is less than the difference threshold, and the optimized conversion relationship is obtained. The optimized conversion relationship parameters are the model parameters of the initial calibration model.

[0064] In one example, the plurality of first image coordinates and the plurality of first physical coordinates of the plurality of first corner points can be represented as shown in formula (3) as follows:

[0065] Formula (3)

[0066] wherein M represents the number of the first corner points.

[0067] Suppose the conversion relationship between the image coordinates and the physical coordinates is θ, then the first physical coordinates of any first corner point can be input into θ to obtain the candidate image coordinates corresponding to the first physical coordinates, the candidate image coordinates are compared with the first image coordinates, if the difference between the candidate image coordinates and the first image coordinates is greater than the difference threshold, the parameters of θ are continuously adjusted until the difference between the candidate image coordinates and the first image coordinates is less than or equal to the difference threshold, and the optimized conversion relationship is obtained as an initial calibration model. Exemplarily, which can be represented as shown in the following formula (4).

[0068] Formula (4)

[0069] wherein, represents the first physical coordinates of the i-th first corner point, represents the first image coordinates of the i-th corner point, represents the candidate image coordinates corresponding to the first physical coordinates.

[0070] In other words, the initial calibration model can make the difference between the candidate image coordinates and the first image coordinates less than the difference threshold.

[0071] It can be seen that in the embodiment, the candidate image coordinates corresponding to the first physical coordinates are determined through the conversion relationship between the image coordinates and the physical coordinates corresponding to the first corner points, the parameters of the conversion relationship are optimized according to the difference between the candidate image coordinates and the first image coordinates, and the initial calibration model is obtained after the optimization condition is met, so that a more accurate initial calibration model can be obtained, thereby laying a good foundation for reducing the camera calibration error.

[0072] In some embodiments, the first chessboard image includes marker code information; before the initial calibration model is determined according to the first image coordinates and the first physical coordinates of the first corner points, the above method further includes: identifying the marker code information in the first chessboard image to determine the physical coordinates of the first marker code; and determining the first physical coordinates of the first corner point according to the positional relationship between the first marker code and the first corner point and the physical coordinates of the first marker code.

[0073] ​It is easily understood that the second chessboard image can include the marker code information or not include the marker code information. If the second chessboard image does not include the marker code information, after the second corner points are detected, the second physical coordinates of the second corner points can be determined in sequence according to the positions of the second corner points in the second chessboard image and the physical size of the chessboard squares in the second chessboard. For example, if the second physical coordinates of the first second corner point at the top left corner in the second chessboard image can be set as (0, 0), if the size of the chessboard square is 5 mm, the second physical coordinates of the second corner point adjacent to the right of the first second corner point can be set as (5, 0), the second physical coordinates of the second corner point adjacent to the bottom of the first second corner point can be set as (0, 5), and so on, so that the second physical coordinates of all the second corner points can be obtained.

[0074] Exemplarily, the second chessboard image includes the marker code information. Before the second physical coordinates of the second corner points are aligned with the candidate physical coordinates to obtain the target physical coordinates corresponding to the second physical coordinates, the marker code information in the second chessboard image can be recognized to determine the physical coordinates of the second marker code; and the second physical coordinates of the second corner points can be determined according to the positional relationship between the second marker code and the second corner points and the physical coordinates of the second marker code.

[0075] The marker code information refers to the position information corresponding to the marker code. The marker code information included in the first chessboard image refers to the position information corresponding to the first marker code included in the first chessboard, and the marker code information included in the second chessboard image refers to the position information corresponding to the second marker code included in the second chessboard. The marker code can be represented by, for example, a two-dimensional QR (2-dimensional bar code) code, an aruco (Hamming code square diagram) code or other forms of marker codes, and the physical coordinates of the marker code position can be determined according to the marker code. It is easily understood that the first marker code and the second marker code can be the same type of marker code or different types of marker code. Exemplarily, the chessboard including the marker code is as shown in FIG. 5. Figure 5 Figure 5 The black square 502 in FIG. 5 represents the marker code position. The marker code can be at any position in the chessboard, and one or more marker codes can be included in the same chessboard. That is, the first marker code can be one or more, and the second marker code can also be one or more.

[0076] ​For example, by identifying the marker information in a checkerboard image, the physical coordinates of the marker can be determined. Based on the positional relationship between the marker and the corner points of the checkerboard, and given the physical coordinates of the marker, the origin and coordinate axis directions of the corresponding physical coordinate system can be determined. Based on this physical coordinate system, the physical coordinates of the corresponding checkerboard corner points can then be determined. It should be noted that the position (physical coordinates) of the marker can represent any position within the corresponding physical coordinate system. For example... Figure 5 As shown, if the marker code information is identified, the origin and coordinate axis directions of the physical coordinate system can be determined. Specifically, the top-left corner of the chessboard image is the origin, the horizontal arrow represents the X-axis, and the vertical arrow represents the Y-axis. Assuming the physical length and width of each chessboard square are both 15, then the physical coordinates of the first corner point (top-left) are (15, 15), the second horizontal corner point is (30, 15), the second vertical corner point is (15, 30), and so on. This allows us to obtain the physical coordinates of any corner point in the corresponding chessboard image. Alternatively, the marker code information includes the X-axis and Y-axis offsets of the marker code's center position relative to the origin of the physical coordinate system, thus allowing us to determine the coordinates of the corner points surrounding the marker code within that physical coordinate system. For example, scanning the marker code and identifying its information yields (19.5, 6.5, P10, M2), where 19.5 represents the marker code's center position being offset by 19.5 chessboard squares relative to the origin of the physical coordinate system along the X-axis; 6.5 represents the marker code's center position being offset by 6.5 chessboard squares relative to the origin of the physical coordinate system along the Y-axis; M2 represents the marker code's width as 2 chessboard squares, with a transition area of ​​0.5 chessboard squares at the top, bottom, left, and right edges; and P10 represents... The physical size of a chessboard square is 10mm. Therefore, the center of the marker occupies one chessboard square horizontally and 0.5 chessboard squares vertically. Excluding the 1.5 chessboard squares, the top-left corner of the marker is offset by 19.5 - 1.5 = 18 chessboard squares along the X-axis and 6.5 - 1.5 chessboard squares along the Y-axis. Thus, the coordinates of the top-left corner of the marker are (18 * 10mm, 5 * 10mm). Here, a chessboard square is the smallest unit of the chessboard grid, i.e., the smallest square in the chessboard.

[0077] The aforementioned marker information can be marker information from either the first or second chessboard image. If the marker information is from the first chessboard image, the physical coordinates of the first marker in the first chessboard can be determined, thereby determining the first physical coordinates of the first corner point. If the marker information is from the second chessboard image, the physical coordinates of the second marker in the second chessboard can be determined, thereby determining the second physical coordinates of the second corner point.

[0078] It can be seen that, in this embodiment, the physical coordinates of the first marker code can be determined by recognizing the marker code information in the first checkerboard image, and the first physical coordinates of the first corner point can be determined according to the positional relationship between the first marker code and the first corner point, so that the first physical coordinates of the first corner point can be accurately and quickly determined, thereby laying a foundation for accurately determining the target calibration model subsequently.

[0079] In some embodiments, the plurality of cameras to be calibrated includes a plurality of cameras; the first checkerboard image collected by the cameras to be calibrated for the first checkerboard and the second checkerboard image collected by the cameras to be calibrated for the second checkerboard include:

[0080] The plurality of cameras to be calibrated simultaneously captures the first checkerboard to obtain a plurality of first checkerboard images; the plurality of cameras to be calibrated simultaneously captures the second checkerboard to obtain a plurality of second checkerboard images; wherein each first checkerboard image includes marker code information.

[0081] In actual application scenarios, the plurality of cameras to be calibrated can be calibrated simultaneously, and specifically, the plurality of cameras to be calibrated can simultaneously capture the same first checkerboard to obtain a first checkerboard image corresponding to each camera to be calibrated, thereby obtaining a plurality of first checkerboard images. Correspondingly, the plurality of cameras to be calibrated can simultaneously capture the same second checkerboard to obtain a second checkerboard image corresponding to each camera to be calibrated, thereby obtaining a plurality of second checkerboard images.

[0082] Exemplarily, the plurality of cameras to be calibrated can simultaneously capture the first checkerboard, and in the process of capturing, each camera to be calibrated needs to collect the first marker code in the first checkerboard to obtain a plurality of first checkerboard images, each first checkerboard image including marker code information corresponding to the first marker code. Then, the second checkerboard is placed at the position of the first checkerboard, or the second checkerboard is placed on the plane where the first checkerboard is located, that is, the first checkerboard and the second checkerboard are in the same plane, and the plurality of cameras to be calibrated simultaneously capture the second checkerboard. The second checkerboard image can include marker code information or can not include marker code information. In the process of capturing, if the second checkerboard image includes marker code information, each camera to be calibrated needs to collect the second marker code in the second checkerboard to obtain a plurality of second checkerboard images, and correspondingly, each second checkerboard image includes marker code information corresponding to the second marker code.

[0083] It can be seen that, in this embodiment, the plurality of cameras to be calibrated can be calibrated simultaneously by capturing the checkerboard to obtain the checkerboard image, so that the plurality of cameras to be calibrated can be calibrated simultaneously, and the image coordinates can be converted to the same physical coordinate system. When a camera is used to measure a large target, high-precision measurement of a large target can be realized by networking two or more cameras.

[0084] In some embodiments, the first chessboard is a printed chessboard, and the second chessboard is a standard chessboard.

[0085] The printed chessboard refers to a chessboard obtained by a printer. The printer can be an office or home printer, and can be an inkjet printer or a laser printer. In other words, the printed chessboard can be obtained in a relatively simple manner. For example, the chessboard pattern can be pre-set by office software, and then a paper printed chessboard can be obtained by printing.

[0086] The standard chessboard refers to a chessboard with a higher precision than a precision threshold. For example, the standard chessboard can be a customized chessboard specially used for calibration, which can be engraved on glass or acrylic material by high-precision laser. It is easy to understand that the precision of the chessboard can be reflected in the precision of the chessboard scale, the straightness, and the surface flatness. In other words, if at least one of the precision of the chessboard scale, the straightness, or the surface flatness of the chessboard is higher than the corresponding threshold, it can be said that the precision of the chessboard is higher than the precision threshold. For example, if the precision of the chessboard scale is higher than the scale threshold, it can be said that the precision of the chessboard is higher than the precision threshold. Or, if the precision of the chessboard scale is higher than the scale threshold and the straightness is higher than the straightness threshold, it can be said that the precision of the chessboard is higher than the precision threshold. Or, if the precision of the chessboard scale is higher than the scale threshold, the straightness is higher than the straightness threshold, and the surface flatness is higher than the flatness threshold, it can be said that the precision of the chessboard is higher than the precision threshold.

[0087] As can be seen, in this embodiment, the coordinate systems of the plurality of cameras to be calibrated can be calibrated to the same physical coordinate system by using the printed chessboard, and then the calibration accuracy of the initial calibration model can be improved by using the standard chessboard, so that high-precision calibration of the cameras can be achieved. That is, high-precision calibration of the cameras can be achieved by using the printed chessboard which is relatively low in cost and easy to obtain, and the standard chessboard which is fixedly required, so that the overall calibration cost can be reduced.

[0088] In an exemplary embodiment, a camera calibration flowchart is shown as Figure 6 In an exemplary embodiment, a camera calibration flowchart is shown as

[0089] S602, a printed chessboard with a first mark code is simultaneously photographed by two cameras to be calibrated, and a first chessboard image is obtained.

[0090] S604, corner points of the first chessboard image are detected to obtain first corner points, and an initial calibration model of each camera is determined according to the first image coordinates and the first physical coordinates of the first corner points.

[0091] S606, obtain a second checkerboard image by simultaneously shooting the standard checkerboard through the two cameras to be calibrated.

[0092] S608, perform corner point detection on the second checkerboard image to obtain second corner points, and determine candidate physical coordinates of the second corner points according to the second image coordinates of the second corner points and the initial calibration model.

[0093] S610, align the second physical coordinates of the second corner points corresponding to each camera with the candidate physical coordinates to obtain target physical coordinates corresponding to the second physical coordinates.

[0094] S612, update the initial calibration model according to the target physical coordinates to obtain a target calibration model.

[0095] According to the target physical coordinates and the second image coordinates of the second corner points, a conversion relationship between the target physical coordinates and the second image coordinates is determined to obtain the target calibration model.

[0096] As can be seen, in the above embodiments, the initial camera geometric model is determined by printing the checkerboard, and then the calibration model is optimized by the standard checkerboard, so that high-precision calibration of the camera can be achieved at a low calibration cost. In addition, the calibration models of all cameras correspond to the same physical coordinate system, and after completing the respective detection and positioning tasks, the image coordinates of the positioning can be converted into physical coordinates in the same physical coordinate system, so as to realize high-precision positioning and measurement of large-size targets.

[0097] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0098] Based on the same inventive concept, the embodiments of the present application also provide a camera calibration device. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more camera calibration device embodiments provided below can refer to the limitations of the camera calibration method in the above text, which will not be repeated here.

[0099] As Figure 7As shown, the embodiment of the present application provides a camera calibration device, comprising:

[0100] The corner point detection module 702 is configured to acquire a first checkerboard image collected by the camera to be calibrated for a first checkerboard and a second checkerboard image collected by the camera to be calibrated for a second checkerboard, and perform corner point detection on the first checkerboard image to obtain first corner points and perform corner point detection on the second checkerboard image to obtain second corner points; the accuracy of the second checkerboard is higher than that of the first checkerboard.

[0101] The initial model determination module 704 is configured to determine an initial calibration model according to the first image coordinates and the first physical coordinates of the first corner points.

[0102] The physical coordinate calculation module 706 is configured to determine candidate physical coordinates corresponding to the second corner points according to the second image coordinates of the second corner points and the initial calibration model.

[0103] The physical coordinate alignment module 708 is configured to align the second physical coordinates of the second corner points with the candidate physical coordinates to obtain target physical coordinates corresponding to the second physical coordinates.

[0104] The target model determination module 710 is configured to determine a target calibration model according to the target physical coordinates and the second image coordinates.

[0105] In some embodiments, in aligning the second physical coordinates of the second corner points with the candidate physical coordinates to obtain target physical coordinates corresponding to the second physical coordinates, the physical coordinate alignment module 708 is specifically configured to fit the second physical coordinates of the second corner points with the candidate physical coordinates to obtain a coordinate fitting relationship; determine a target rotation amount and a target translation amount according to the coordinate fitting relationship; and align the second physical coordinates with the candidate physical coordinates according to the target rotation amount and the target translation amount to obtain the target physical coordinates corresponding to the second physical coordinates.

[0106] In some embodiments, the first corner points include a plurality of; in determining the initial calibration model according to the first image coordinates and the first physical coordinates of the first corner points, the initial model determination module 704 is specifically configured to determine a conversion relationship between image coordinates and physical coordinates according to the first image coordinates and the first physical coordinates of each first corner point; determine candidate image coordinates corresponding to the first physical coordinates according to the conversion relationship and the first physical coordinates; and optimize the conversion relationship according to the difference between the candidate image coordinates and the first image coordinates to obtain the initial calibration model.

[0107] In some embodiments, the first chessboard image includes marker code information; the device further includes a physical coordinate determination module, specifically configured to identify the marker code information in the first chessboard image to determine the physical coordinates of the first marker code, before determining the initial calibration model according to the first image coordinates and the first physical coordinates of the first corner point.

[0108] In some embodiments, the cameras to be calibrated include multiple cameras; in terms of acquiring the first chessboard image collected by the cameras to be calibrated for the first chessboard and the second chessboard image collected for the second chessboard, the corner point detection module 702 is specifically configured to simultaneously capture the first chessboard by the multiple cameras to be calibrated to obtain multiple first chessboard images; and simultaneously capture the second chessboard by the multiple cameras to be calibrated to obtain multiple second chessboard images; wherein each first chessboard image includes marker code information.

[0109] The modules in the camera calibration device can be all or partially implemented by software, hardware, and combinations thereof. The modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so as to be called and executed by the processor to perform the operations corresponding to the modules.

[0110] In some embodiments, a camera calibration device is provided, which can be a server, and its internal structure diagram can be as shown in Figure 8 The camera calibration device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the camera calibration device is configured to provide computing and control capabilities. The memory of the camera calibration device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the camera calibration device is configured to store camera calibration data. The input / output interface of the camera calibration device is configured to exchange information between the processor and external devices. The communication interface of the camera calibration device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement the steps in the camera calibration method described above.

[0111] In some embodiments, a camera calibration device is provided, which can be a terminal, and its internal structure diagram can be as shown in Figure 9As shown in the figure. The camera calibration device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the camera calibration device is used to provide computing and control capability. The memory of the camera calibration device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the camera calibration device is used to exchange information between the processor and external devices. The communication interface of the camera calibration device is used for wired or wireless communication with external terminals, and wireless communication can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to realize the steps in the above-mentioned camera calibration method. The display unit of the camera calibration device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; The input device of the camera calibration device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the camera calibration device, or an external keyboard, touchpad or mouse, etc.

[0112] Those skilled in the art can understand that, Figure 8 or Figure 9 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the camera calibration device to which the scheme of the present application is applied. The specific camera calibration device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0113] In some embodiments, a camera calibration device is provided, which includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps in the above-mentioned method embodiments.

[0114] In some embodiments, as Figure 10 A block diagram of the internal structure of a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to realize the steps in the above-mentioned method embodiments.

[0115] In some embodiments, a computer program product is provided, which includes a computer program, and the computer program is executed by a processor to realize the steps in the above-mentioned method embodiments.

[0116] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of the country and region.

[0117] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. In the embodiments provided in the present application, any reference to memory, database or other medium can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0118] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0119] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A camera calibration method, characterized by, The method comprises the following steps: acquiring a first checkerboard image collected by a camera to be calibrated for a first checkerboard and a second checkerboard image collected by the camera to be calibrated for a second checkerboard, and performing corner point detection on the first checkerboard image to obtain first corner points and performing corner point detection on the second checkerboard image to obtain second corner points; the accuracy of the second checkerboard is higher than that of the first checkerboard; the first checkerboard and the second checkerboard are placed on the same plane; determining an initial calibration model according to first image coordinates and first physical coordinates of the first corner points; determining candidate physical coordinates corresponding to the second corner points according to second image coordinates of the second corner points and the initial calibration model; fitting the second physical coordinates of the second corner points with the candidate physical coordinates to obtain a coordinate fitting relationship; determining a target rotation amount and a target translation amount according to the coordinate fitting relationship; aligning the second physical coordinates with the candidate physical coordinates according to the target rotation amount and the target translation amount to obtain target physical coordinates corresponding to the second physical coordinates; determining a target calibration model according to the target physical coordinates and the second image coordinates.

2. The method of claim 1, wherein, The method comprises the following steps: inputting the second image coordinates of the second corner points into the initial calibration model to obtain the candidate physical coordinates corresponding to the second corner points.

3. The method of claim 1, wherein, The first corner points comprise a plurality of first corner points; the method comprises the following steps: determining a conversion relationship between image coordinates and physical coordinates according to the first image coordinates and the first physical coordinates of each first corner point; determining candidate image coordinates corresponding to the first physical coordinates according to the conversion relationship and the first physical coordinates; optimizing the conversion relationship according to the difference between the candidate image coordinates and the first image coordinates to obtain the initial calibration model.

4. The method of claim 1, wherein, The first checkerboard image comprises marker code information; Before the step of determining the initial calibration model according to the first image coordinates and the first physical coordinates of the first corner points, the method further comprises the following steps: identifying the marker code information in the first checkerboard image to determine the physical coordinates of a first marker code; determining the first physical coordinates of the first corner points according to the positional relationship between the first marker code and the first corner points and the physical coordinates of the first marker code.

5. The method of claim 1, wherein, The camera to be calibrated comprises a plurality of cameras; the method comprises the following steps: simultaneously shooting the first checkerboard by the plurality of cameras to be calibrated to obtain a plurality of first checkerboard images; simultaneously shooting the second checkerboard by the plurality of cameras to be calibrated to obtain a plurality of second checkerboard images; each first checkerboard image comprises marker code information.

6. The method according to any one of claims 1 to 5, characterized in that, The first checkerboard is a printed checkerboard, and the second checkerboard is a standard checkerboard.

7. A camera calibration apparatus characterized by comprising: The method comprises the following steps: The corner point detection module is configured to acquire a first checkerboard image collected by the camera to be calibrated for a first checkerboard and a second checkerboard image collected by the camera to be calibrated for a second checkerboard, and perform corner point detection on the first checkerboard image to obtain first corner points and perform corner point detection on the second checkerboard image to obtain second corner points; the second checkerboard has a higher precision than the first checkerboard; The first checkerboard and the second checkerboard are placed on the same plane; The initial model determination module is configured to determine an initial calibration model according to first image coordinates and first physical coordinates of the first corner points; The physical coordinate calculation module is configured to determine candidate physical coordinates corresponding to the second corner points according to second image coordinates of the second corner points and the initial calibration model; The physical coordinate alignment module is configured to fit the second physical coordinates of the second corner points with the candidate physical coordinates to obtain a coordinate fitting relationship, determine a target rotation amount and a target translation amount according to the coordinate fitting relationship, and align the second physical coordinates with the candidate physical coordinates according to the target rotation amount and the target translation amount to obtain target physical coordinates corresponding to the second physical coordinates. The target model determination module is configured to determine a target calibration model according to the target physical coordinates and the second image coordinates.

8. The apparatus of claim 7, wherein, The first corner points include a plurality of first corner points; in the step of determining the initial calibration model according to the first image coordinates and the first physical coordinates of the first corner points, the initial model determination module is specifically configured to determine a conversion relationship between image coordinates and physical coordinates according to the first image coordinates and the first physical coordinates of each of the first corner points, determine candidate image coordinates corresponding to the first physical coordinates according to the conversion relationship and the first physical coordinates, and optimize the conversion relationship according to a difference between the candidate image coordinates and the first image coordinates to obtain the initial calibration model.

9. A camera calibration device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

11. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images