A novel cube-based three-dimensional calibration method for depth cameras

Abstract

The present invention relates to the technical field of camera calibration, and relates to a new three-dimensional calibration method for a depth camera based on a cube. The method includes the following steps: establishing a three-dimensional model of a square block; establishing a world three-dimensional coordinate system; selecting world three-dimensional point coordinates (Xi Yi Zi); converting a color image I0(R0(i, j), G0(i, j), B0(i, j)) into a grayscale image Ig(i, j); using a sobel operator to extract the image edges of the grayscale image Ig(i, j); using a hough transform circle detection to identify the center coordinates (μ c ν c ) of the three-dimensional model of the square block; using a pnp mapping algorithm to calculate the external parameter rotation matrix R and the translation vector t of the camera. Through the above steps, the present invention can obtain a relatively high-precision external parameter rotation matrix R and translation vector t of the camera, thereby solving the problem of relatively low calibration accuracy of the depth camera.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and more specifically, to a new cube-based three-dimensional calibration method for depth cameras. Background Art

[0002] In recent years, with the increasing applications of depth cameras in fields such as three-dimensional ranging, consumer electronics, security monitoring, and robotics, more and more research has been conducted on the process of three-dimensional calibration of depth cameras to obtain the external parameter rotation matrix and translation vector of the camera, and finally complete the coordinate transformation. Currently, the accuracy of the calibration method using a stereoscopic square model is relatively low. Therefore, many special calibration devices for three-dimensional calibration of TOF depth cameras have emerged, but these devices are difficult to popularize, resulting in a relatively high threshold for camera calibration. Therefore, the calibration of general depth cameras is difficult and the accuracy is relatively low. Summary of the Invention

[0003] Aiming at the defects of relatively large difficulty and low accuracy in the calibration of depth cameras in the prior art, the present invention provides a new cube-based three-dimensional calibration method for depth cameras. It can achieve obtaining a relatively high-precision external parameter rotation matrix R and translation vector t of the camera, thereby solving the problem of relatively low accuracy in the calibration of depth cameras.

[0004] To solve the above technical problems, the present invention is solved by the following technical solutions.

[0005] A new cube-based three-dimensional calibration method for depth cameras, which includes the following steps;

[0006] Step S1: Establish a stereoscopic square block model;

[0007] Step S2: Establish a world three-dimensional coordinate system based on the stereoscopic square block model;

[0008] Step S3: Select i world three-dimensional point coordinates (Xi, Yi, Zi) in the world three-dimensional coordinate system of Step S2, and establish a corresponding world three-dimensional point coordinate matrix [X W Y W Z W ] T ;

[0009] Step S4: Use the color image I0 (R0(i, j), G0(i, j), B0(i, j)) collected by the depth camera, and set its grayscale image as Ig(i, j), where R0(i, j) represents the proportion of red in the color image, G0(i, j) represents the proportion of green in the color image, and B0(i, j) represents the proportion of blue in the color image;

[0010] Step S5: Use the sobel operator to extract the image edge of the grayscale image Ig(i, j) in Step S4;

[0011] Step S6: Use the Hough transform circle detection to identify the center coordinates (μ c ν c ) of c points of the three-dimensional square block model. The center coordinate matrix is represented as [μ c ν c T , and the homogeneous coordinate matrix of the center coordinate matrix is represented as [μ c ν c 1] T ;

[0012] Step S7: Use the PnP mapping algorithm to calculate the external camera rotation matrix R and translation vector t.

[0013] In the present invention, by establishing a three-dimensional square block model and establishing a world three-dimensional coordinate system based on the three-dimensional square block model, i world coordinate points (Xi Yi Zi) in the world coordinate system within the world three-dimensional coordinate system are selected, and a corresponding world three-dimensional point coordinate matrix [X W Y W Z W T is established. Secondly, the color image I0 (R0(i, j), G0(i, j), B0(i, j)) obtained by the depth camera is grayscaled into a grayscale image Ig(i, j), thereby providing a grayscale environment for the Sobel operator, that is, calculating the image edges of the grayscale image Ig(i, j). Further, the Hough transform circle detection is used to detect the center coordinates (μ c ν c ) in the pixel coordinate system corresponding to the world three-dimensional point coordinates (Xi Yi Zi) in the world three-dimensional coordinate system, thereby providing corresponding data for the PnP mapping algorithm, and further preferably solving the external camera rotation matrix R and translation vector t; through the above steps, a higher-precision external camera rotation matrix R and translation vector t can be preferably solved, so as to provide high-precision external camera parameters for the conversion from the camera three-dimensional coordinate system to the world three-dimensional coordinate system, that is, preferably realizing the calibration of the camera.

[0014] Preferably, before step S1, the internal camera matrix K is obtained.

[0015] In the present invention, the internal camera parameters are preferably obtained by the checkerboard calibration method, that is, the internal camera matrix K is obtained, and then the internal camera matrix K is provided for the PnP mapping algorithm, that is, a higher-precision external camera rotation matrix R and translation vector t are solved.

[0016] Preferably, in step S1, ten cubes with a side length of l and a black center dot are stacked from bottom to top into 6*3*1 blocks. ​​

[0017] In the present invention, the lower right corner of the three-dimensional square block model is selected as the origin (0, 0, 0) of the world three-dimensional coordinate system, which is more convenient for selecting the world three-dimensional point coordinates.

[0018] Preferably, in step S3, six world three-dimensional coordinate points are selected, denoted as P1, P2, P3, P4, P5, and P6 respectively, and the corresponding world three-dimensional point coordinates are denoted as P1(0.5*l, l, 2.5*l), P2(0.5*l, 2*l, 1.5*l), P3(0.5*l, 3*l, 0.5*l), P4(1.5*l, l, 1.5*l), P5(1.5*l, 2*l, 0.5*l), P6(2.5*l, l, 0.5*l), and the corresponding world three-dimensional point coordinate matrices are denoted as [0.5*l, l, 2.5*l]T, [0.5*l, 2*l, 1.5*l]T, [0.5*l, 3*l, 0.5*l]T, [1.5*l, l, 1.5*l]T, [1.5*l, 2*l, 0.5*l]T, [2.5*l, l, 0.5*l]T.

[0019] In the present invention, six world point coordinates (Xi Yi Zi) in the world coordinate system are selected, where i = 1, 2,..., 6, and the corresponding world three-dimensional point coordinate matrix [X W Y W Z W T and the corresponding homogeneous world three-dimensional point coordinate matrix [X W Y W Z W 1] T are established. Therefore, a homogeneous world coordinate point matrix [X W Y W Z W 1] T data is provided for the pnp mapping algorithm, and thus a camera external parameter rotation matrix R and a translation vector t with higher precision are calculated.

[0020] Preferably, in step S4, the color image I0(R0(i, j), G0(i, j), B0(i, j)) and the grayscale image Ig(i, j) satisfy the following relationship;

[0021]

[0022] In the present invention, according to the relationship between the above-mentioned color image I0(R0(i, j), G0(i, j), B0(i, j)) and the grayscale image Ig(i, j), the color image I0(R0(i, j), G0(i, j), B0(i, j)) acquired by the depth camera is grayscaled. Therefore, preferably, the grayscale image Ig(i, j) of the color image I0(R0(i, j), G0(i, j), B0(i, j)) is obtained, thereby providing a grayscaled environment for the sobel operator used in step S5.

[0023] Preferably, in step S5, it specifically includes the following steps;

[0024] Step S51, use the horizontal gradient template to weight each row in the grayscale image Ig(i, j) with A(i, j) = Ig(i, j) * d x ;

[0025] where d x represents the horizontal gradient template, and A(i, j) represents the weight of the grayscale image Ig(i, j) in the horizontal gradient;

[0026] Step S52, use the vertical gradient template to weight each column in the grayscale image Ig(i, j) with B(i, j) = Ig(i, j) * d y ;

[0027] where d y represents the vertical gradient template, and B(i, j) represents the weight of the grayscale image Ig(i, j) in the vertical gradient;

[0028] Step S53, add A(i, j) and B(i, j), that is, the weighted sum S(i, j) = A(i, j) + B(i, j);

[0029] where S(i, j) represents the weighted sum of the grayscale image Ig(i, j) in the horizontal gradient and the vertical gradient;

[0030] Step S54, set a threshold M; the threshold M adopts the shannon entropy threshold method, and the formula is P i is the probability that the pixel with pixel value i accounts for the entire image;

[0031] Step S55, compare the weighted sum S(i, j) obtained in step S53 with the threshold M set in step S54. If the pixel value in the weighted sum S(i, j) is greater than the set threshold M, that is, the center edge pixels IB of the 6 points of the obtained three-dimensional square block model are obtained, that is, IB = S(i, j) > threshold M.

[0032] In the present invention, in step S51, the horizontal gradient template d is used x , to obtain the weighting of A(i, j) on the horizontal gradient of the grayscale image Ig(i, j), and thus preferably obtain the image edge in the horizontal direction. In step S52, the vertical gradient template d is used y , to obtain B(i, j) which is expressed as the weighting of the grayscale image Ig(i, j) on the vertical gradient, and thus preferably obtain the image edge in the vertical direction. Therefore, A(i, j) and B(i, j) are added to obtain the weighted sum S(i, j), and the weighted sum S(i, j) is compared with a set threshold. If the pixel value in the weighted sum S(i, j) is greater than the pixel value of the set threshold M, that is, the image edge of the grayscale image Ig(i, j), the center-edge pixels IB of the 6 points of the three-dimensional square block model can be preferably obtained.

[0033] Preferably, in step S6, it specifically includes the following steps;

[0034] Step S61, set the center detection coordinates (a, b), the search radius r, and the gradient angle θ;

[0035] Step S62, convert each edge point as shown in the formula to the Hough domain;

[0036] Step S63, the straight lines in the Hough domain corresponding to each edge point intersect at a point, and the intersection point is the center coordinate (μ c ν c ).

[0037] In the present invention, a point in the parameter space is set as the center detection coordinates (a, b), and the search radius r and the gradient angle θ are set, and then the coordinates of the edge points in the actual image space are obtained. The edge point coordinates satisfy the formula where s(i) and s(j) both represent the coordinates of an edge point in the actual image space of the edge point. Therefore, the straight lines in the Hough domain corresponding to each edge point intersect at a point, that is, the intersection point is the center coordinate (μ c ν c ), that is, preferably obtain the center matrix [μ c ν c T , and then obtain the pixel coordinate points corresponding to the world coordinate points in the corresponding world coordinate system in the pixel coordinate system, that is, it can provide the pixel coordinate point matrix [μ W ν W Z W 1] T in the pixel coordinate system corresponding to the homogeneous world coordinate point matrix [X W Y W Z W 1] T of the pnp mapping algorithm c ν c 1] T, thus obtaining a relatively high-precision external camera rotation matrix \(R\) and translation vector \(t\).

[0038] Preferably, in step S7, the solution process is as follows:

[0039] According to the perspective projection model of the external camera rotation matrix \(R\) and translation vector \(t\), \(z\) c is the depth value of the camera,

[0040]

[0041] Expand:

[0042]

[0043] Write as:

[0044]

[0045] Eliminate \(z\) c , and after arrangement, we get:

[0046]

[0047] Since the coordinates of 6 points are collected, solve the 6 - element linear equations:

[0048]

[0049] That is, obtain the parameter values of \([f\) 11 \(f\) 12 \(f\) 13 \(f\) 14 \(f\) 21 \(f\) 22 \(f\) 23 \(f\) 24 \(f\) 31 \(f\) 32 \(f\) 33 \(f\) 34 ,

[0050] Therefore, the external camera rotation matrix \(R\) and translation vector \(t\) are obtained as:

[0051]

[0052]

[0053] Among them, \(f\) 11 , \(f\) 12 , \(f\) 13 , \(f\) 21 , \(f\) 22 , \(f\) 23 , \(f\) 31 , \(f\) 32 , \(f\) 33Internal parameter values represented as the external rotation matrix R of the camera, f 14 , f 24 , f 34 Internal parameter values represented as the camera translation vector t.

[0054] In the present invention, through the above process of solving the external rotation matrix R and translation vector t of the camera, it is thus possible to preferably obtain the external rotation matrix R and translation vector t of the camera with higher precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flowchart of a novel cube-based three-dimensional calibration method for a depth camera in Embodiment 1.

[0056] Figure 2 It is a schematic diagram of a three-dimensional square block model in Embodiment 1. DETAILED DESCRIPTION OF THE INVENTION

[0057] To further understand the content of the present invention, the present invention will be described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments are only for explaining the present invention and not for limiting it.

[0058] Embodiment 1

[0059] As Figure 1 shown, this embodiment provides a novel cube-based three-dimensional calibration method for a depth camera, which includes the following steps;

[0060] Step S1: Establish a three-dimensional square block model;

[0061] Step S2: Establish a world three-dimensional coordinate system based on the three-dimensional square block model;

[0062] Step S3: Select i world three-dimensional point coordinates (Xi Yi Zi) in the world three-dimensional coordinate system of Step S2, and establish a corresponding world three-dimensional point coordinate matrix [X W Y W Z W T ;

[0063] Step S4: Use the color image I0(R0(i, j), G0(i, j), B0(i, j)) collected by the depth camera, and set its grayscale image as Ig(i, j), where R0(i, j) represents the proportion of red in the color image, G0(i, j) represents the proportion of green in the color image, and B0(i, j) represents the proportion of blue in the color image;

[0064] Step S5: Use the sobel operator to extract the image edge of the grayscale image Ig(i, j) in Step S4;

[0065] Step S6: Use the Hough transform circle detection to identify the center coordinates (μ c ν c ) of c points of the three-dimensional square block model. The center coordinate matrix is represented as [μ c ν c T . The homogeneous coordinate matrix of the center coordinate matrix is represented as [μ c ν c 1] T ;

[0066] Step S7: Use the PnP mapping algorithm to calculate the camera extrinsic rotation matrix R and the translation vector t.

[0067] In this embodiment, by establishing a three-dimensional square block model and establishing a world three-dimensional coordinate system based on the three-dimensional square block model, i world coordinate points (Xi Yi Zi) in the world coordinate system are selected within the world three-dimensional coordinate system, and a corresponding world three-dimensional point coordinate matrix [X W Y W Z W T is established. Secondly, the color image I0 (R0(i, j), G0(i, j), B0(i, j)) obtained by the depth camera is grayscaled to a grayscale image Ig(i, j), thereby providing a grayscale environment for the Sobel operator, that is, calculating the image edge of the grayscale image Ig(i, j). Further, the Hough transform circle detection is used to detect the center coordinates (μ c ν c ) in the pixel coordinate system corresponding to the world three-dimensional point coordinates (Xi Yi Zi) in the world three-dimensional coordinate system, thereby providing corresponding data for the PnP mapping algorithm, and further preferably solving the camera extrinsic rotation matrix R and the translation vector t. Through the above steps, a relatively high-precision camera extrinsic rotation matrix R and translation vector t can be preferably solved, so as to provide high-precision camera extrinsic parameters for the conversion from the camera three-dimensional coordinate system to the world three-dimensional coordinate system, that is, preferably realizing the calibration of the camera.

[0068] In this embodiment, before step S1, the intrinsic matrix K of the camera is obtained.

[0069] In this embodiment, the intrinsic parameters of the camera are preferably obtained through the checkerboard calibration method, that is, the intrinsic matrix K is obtained, and then the intrinsic matrix K of the camera is provided for the PnP mapping algorithm in step S7, that is, a relatively high-precision camera extrinsic rotation matrix R and translation vector t are solved.

[0070] In this embodiment, in step S1, 10 cubes with a side length of l and a black center dot are stacked from bottom to top into 6*3*1 blocks.

[0071] ​​In this embodiment, the lower right corner of the three-dimensional square block model is selected as the origin (0, 0, 0) of the world three-dimensional coordinate system, which is more convenient for selecting the world three-dimensional point coordinates.

[0072] In this embodiment, in step S3, six world three-dimensional coordinate points are selected, denoted as P1, P2, P3, P4, P5, and P6 respectively. The corresponding world three-dimensional point coordinates are denoted as P1(0.5*l, l, 2.5*l), P2(0.5*l, 2*l, 1.5*l), P3(0.5*l, 3*l, 0.5*l), P4(1.5*l, l, 1.5*l), P5(1.5*l, 2*l, 0.5*l), and P6(2.5*l, l, 0.5*l) respectively. The corresponding world three-dimensional point coordinate matrices are denoted as [0.5*l, l, 2.5*l]T, [0.5*l, 2*l, 1.5*l]T, [0.5*l, 3*l, 0.5*l]T, [1.5*l, l, 1.5*l]T, [1.5*l, 2*l, 0.5*l]T, and [2.5*l, l, 0.5*l]T respectively.

[0073] In this embodiment, six world point coordinates (Xi Yi Zi) in the world coordinate system are selected, where i = 1, 2,..., 6, and the corresponding world three-dimensional point coordinate matrix [X W Y W Z W T and the corresponding homogeneous world three-dimensional point coordinate matrix [X W Y W Z W 1] T are established. Therefore, a homogeneous world coordinate point matrix [X W Y W Z W 1] T data is provided for the pnp mapping algorithm in step S7, and thus a relatively high-precision camera extrinsic rotation matrix R and translation vector t are calculated.

[0074] In this embodiment, in step S4, the color image I0(R0(i, j), G0(i, j), B0(i, j)) and the grayscale image Ig(i, j) satisfy the following relationship;

[0075]

[0076] In this embodiment, according to the relationship formula between the above color image I0(R0(i, j), G0(i, j), B0(i, j)) and the grayscale image Ig(i, j), the color image I0(R0(i, j), G0(i, j), B0(i, j)) obtained by the depth camera is grayscaled. Therefore, preferably, the grayscale image Ig(i, j) of the color image I0(R0(i, j), G0(i, j), B0(i, j)) is obtained, thereby providing a grayscaled environment for the sobel operator used in step S5.

[0077] In this embodiment, in step S5, it specifically includes the following steps;

[0078] Step S51, use the horizontal gradient template Perform weighting on each row in the grayscale image Ig(i, j), A(i, j) = Ig(i, j) * d x ;

[0079] where d x represents the horizontal gradient template, and A(i, j) represents the weighting of the grayscale image Ig(i, j) on the horizontal gradient;

[0080] Step S52, use the vertical gradient template Perform weighting on each column in the grayscale image Ig(i, j), B(i, j) = Ig(i, j) * d y ;

[0081] where d y represents the vertical gradient template, and B(i, j) represents the weighting of the grayscale image Ig(i, j) on the vertical gradient;

[0082] Step S53, add A(i, j) and B(i, j), that is, the weighted sum S(i, j) = A(i, j) + B(i, j);

[0083] where S(i, j) represents the weighted sum of the grayscale image Ig(i, j) on the horizontal gradient and the vertical gradient;

[0084] Step S54, set a threshold M; the threshold M adopts the shannon entropy threshold method, and the formula is P i is the probability that the pixel with pixel value i accounts for the entire image;

[0085] Step S55, compare the weighted sum S(i, j) obtained in step S53 with the threshold M set in step S54. If the pixel value in the weighted sum S(i, j) is greater than the pixel value of the set threshold M, that is, the center edge pixels IB of the 6 points of the obtained three-dimensional square block model are obtained, that is, IB = S(i, j) > threshold M.

[0086] In this embodiment, in step S51, the horizontal gradient template d is used x , to obtain the weighting of A(i, j) on the horizontal gradient of the grayscale image Ig(i, j), and thus preferably obtain the image edge in the horizontal direction. In step S52, the vertical gradient template d is used y , to obtain B(i, j) expressed as the weighting of the grayscale image Ig(i, j) on the vertical gradient, and thus preferably obtain the image edge in the vertical direction. Therefore, A(i, j) and B(i, j) are added to obtain the weighted sum S(i, j). The weighted sum S(i, j) is compared with a set threshold. If the pixel value in the weighted sum S(i, j) is greater than the pixel value of the set threshold M, that is, the image edge of the grayscale image Ig(i, j), the edge pixels IB of the centers of the 6 points of the three-dimensional square block model can be preferably obtained.

[0087] In this embodiment, step S6 specifically includes the following steps;

[0088] Step S61, set the center detection coordinates (a, b), the search radius r, and the gradient angle θ;

[0089] Step S62, convert each edge point as shown in the formula into the Hough domain;

[0090] Step S63, the straight lines in the Hough domain corresponding to each edge point intersect at a point, and the intersection point is the center coordinate (μ c ν c ).

[0091] In this embodiment, a point in the parameter space is set as the center detection coordinates (a, b), and the search radius r and the gradient angle θ are set, and then the coordinates of the edge points in the actual image space are obtained. The edge point coordinates satisfy the formula where s(i) and s(j) both represent the coordinates of an edge point in the actual image space of the edge point. Therefore, the straight lines in the Hough domain corresponding to each edge point intersect at a point, that is, the intersection point is the center coordinate (μ c ν c ), that is, preferably obtain the center matrix [μ c ν c T , and then obtain the pixel coordinate points corresponding to the world coordinate points in the corresponding world coordinate system in the pixel coordinate system, that is, it can provide the pixel coordinate point matrix [μ W Y W Z W 1] corresponding to the homogeneous world coordinate point matrix [X T in the pnp mapping algorithm in step S7 c ν c 1] T ​, so as to solve the external camera parameter rotation matrix R and translation vector t with high precision.

[0092] In this embodiment, in step S7, the solution process is as follows:

[0093] According to the perspective projection model of the external camera parameter rotation matrix R and translation vector t, z c is the depth value of the camera,

[0094]

[0095] Expand:

[0096]

[0097] Write as:

[0098]

[0099] Eliminate z c , and after arrangement, we get:

[0100]

[0101] Since the coordinates of 6 points are collected, a 6 - element linear equation system is solved:

[0102]

[0103] That is, we obtain the parameter values of [f 11 f 12 f 13 f 14 f 21 f 22 f 23 f 24 f 31 f 32 f 33 f 34 ,

[0104] Therefore, the external camera parameter rotation matrix R and translation vector t are obtained as:

[0105]

[0106]

[0107] Among them, f 11 、f 12 、f 13 、f 21 、f 22 、f 23 、f 31 、f 32 、f 33The internal parameter values represented as the external rotation matrix R of the camera, f 14 、f 24 、f 34 The internal parameter values represented as the camera translation vector t.

[0108] In this embodiment, through the above process of solving the external rotation matrix R and translation vector t of the camera, it is better to obtain the external rotation matrix R and translation vector t of the camera with higher precision.

[0109] In this embodiment, a new three-dimensional calibration method for a depth camera based on a cube is proposed. First, the internal parameter matrix K of the camera is obtained using the checkerboard calibration method. Then, a three-dimensional square block model is established, which is composed of 10 cubes with side length l and a black center dot stacked from bottom to top to form a 6*3*1 block. A world three-dimensional coordinate system is established with the lower right corner of the three-dimensional square block model as the origin (0,0,0). Six world three-dimensional coordinate points are selected as P1(0.5*l, l, 2.5*l), P2(0.5*l, 2*l, 1.5*l), P3(0.5*l, 3*l, 0.5*l), P4(1.5*l, l, 1.5*l), P5(1.5*l, 2*l, 0.5*l), P6(2.5*l, l, 0.5*l). Therefore, the world three-dimensional coordinate point matrices are respectively [0.5*l, l, 2.5*l] T 、[0.5*l, 2*l, 1.5*l] T 、[0.5*l, 3*l, 0.5*l] T 、[1.5*l, l, 1.5*l] T 、[1.5*l, 2*l, 0.5*l] T 、[2.5*l, l, 0.5*l] T And the homogeneous world three-dimensional coordinate point matrices corresponding to the world three-dimensional coordinate point matrices are respectively represented as [0.5*l, l, 2.5*l, l] T 、[0.5*l, 2*l, 1.5*l, l] T 、[0.5*l, 3*l, 0.5*l, l]T, [1.5*l, l, 1.5*l, l]T, [1.5*l, 2*l, 0.5*l, l]T, [2.5*l, l, 0.5*l, l]T; Then, the color image I0(R0(i, j), G0(i, j), B0(i, j)) collected by the camera is grayscaled to obtain the grayscale image Ig(i, j), which provides a grayscale environment for the sobel operator, that is, the image edge of the grayscale image Ig(i, j) is calculated. Further, the hough transform circle detection is used to detect the center coordinates (μ c νc ), through the homogeneous world coordinate point matrix [X W Y W Z W 1] T , w = 1, 2,..., 6, and the homogeneous center coordinate matrix [μ c ν c 1] T , according to the PnP mapping algorithm, the conversion between the world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system is completed, that is,

[0110]

[0111] Furthermore, the external camera rotation matrix R and the translation vector t are preferably solved; therefore, high-precision external camera parameters are provided for the conversion of the camera three-dimensional coordinate system to the world three-dimensional coordinate system, that is, the calibration of the camera is preferably realized.

[0112] In summary, the above are only the preferred embodiments of the present invention, and all equal changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope covered by the patent of the present invention.

Claims

1. A novel cube - based three - dimensional calibration method for depth cameras, Characterized in that: It includes the following steps; Step S1: Establish a three - dimensional square block model; Step S2: Establish a world three - dimensional coordinate system based on the three - dimensional square block model; Step S3: Select i world three-dimensional point coordinates (XiYiZi) in the world three-dimensional coordinate system of Step S2, and establish a corresponding world three-dimensional point coordinate matrix [X W Y W Z W T ;​ Step S4: Use the color image I0(R0(i, j), G0(i, j), B0(i, j)) collected by the depth camera, and set its grayscale image as Ig(i, j), where R0(i, j) represents the proportion of red in the color image, G0(i, j) represents the proportion of green in the color image, and B0(i, j) represents the proportion of blue in the color image; Step S5: Use the sobel operator to extract the image edges of the grayscale image Ig(i, j) in Step S4; Step S6, using Hough transform circle detection, the center coordinates (μ c ν c ) of c points of the three-dimensional square block model are recognized, and the center coordinate matrix is expressed as [μ c ν c T . The homogeneous coordinate matrix of the center coordinate matrix is expressed as [μ c ν c 1] T ;​ Step S7, using the world three-dimensional point coordinate matrix [X W Y W Z W T , and the homogeneous coordinate matrix of the center coordinate matrix [μ c ν c 1] T , calculate the external camera rotation matrix R and translation vector t through the pnp mapping algorithm.​ 2. The novel cube - based three - dimensional calibration method for depth cameras according to Claim 1, Characterized in that: Before Step S1, obtain the internal parameter matrix K of the camera.

3. The novel cube - based three - dimensional calibration method for depth cameras according to Claim 1, Characterized in that: In Step S1, stack 10 cubes with side length l and a black center dot from bottom to top into a 6 * 3 * 1 block.

4. The novel cube - based three - dimensional calibration method for depth cameras according to Claim 3, Characterized in that: In step S3, six world three-dimensional coordinate points are selected, denoted as P1, P2, P3, P4, P5, and P6 respectively. The corresponding world three-dimensional point coordinates are represented as P1(0.5*l, l, 2.5*l), P2(0.5*l, 2*l, 1.5*l), P3(0.5*l, 3*l, 0.5*l), P4(1.5*l, l, 1.5*l), P5(1.5*l, 2*l, 0.5*l), P6(2.5*l, l, 0.5*l), and the corresponding world three-dimensional point coordinate matrices are represented as [0.5*l, l, 2.5*l] T , [0.5*l, 2*l, 1.5*l] T , [0.5*l, 3*l, 0.5*l] T , [1.5*l, l, 1.5*l] T , [1.5*l, 2*l, 0.5*l] T , [2.5*l, l, 0.5*l] T .

5. The novel cube - based three - dimensional calibration method for depth cameras according to Claim 1, Characterized in that: In Step S4, the color image I0(R0(i, j), G0(i, j), B0(i, j)) and the grayscale image Ig(i, j) satisfy the following relationship; 6. The novel cube - based three - dimensional calibration method for depth cameras according to Claim 4, Characterized in that: In Step S5, it specifically includes the following steps; Step S51, use a horizontal gradient template Perform weighting on each row in the grayscale image Ig(i, j) as A(i, j) = Ig(i, j) * d x ; where d x represents the horizontal gradient template, and A(i, j) represents the weighting of the grayscale image Ig(i, j) on the horizontal gradient; Step S52, use a vertical gradient template Perform weighting on each column in the grayscale image Ig(i, j) as B(i, j) = Ig(i, j) * d y ; where d y represents the vertical gradient template, and B(i, j) represents the weighting of the grayscale image Ig(i, j) on the vertical gradient; Step S53: Add A(i, j) and B(i, j), that is, the weighted sum S(i, j)=A(i, j)+B(i, j); Where S(i, j) represents the weighted sum of the grayscale image Ig(i, j) in the horizontal gradient and vertical gradient; Step S54, set a threshold M; the threshold M adopts the Shannon entropy threshold method, and the formula is P i is the probability that the pixels with pixel value i account for the entire image; Step S55: Compare the weighted sum S(i, j) obtained in Step S53 with the set threshold M. If the pixel value in the weighted sum S(i, j) is greater than the pixel value of the set threshold M, that is, obtain the 6 - point center edge pixels IB of the three - dimensional square block model, that is, IB = S(i, j)>threshold M.

7. The novel cube - based three - dimensional calibration method for depth cameras according to Claim 6, Characterized in that: In Step S6, it specifically includes the following steps; Step S61: Set the center detection coordinates (a, b), search radius r, and gradient angle θ; Step S62, convert each edge point as shown in the formula into the Hough domain; Step S63, the straight lines in the Hough domain corresponding to each edge point intersect at one point, and the intersection point is the center coordinate (μ c ν c ).

8. The novel cube - based three - dimensional calibration method for depth cameras according to Claim 6, Characterized in that: In Step S7, the solution process is as follows: According to the perspective projection model of the external reference rotation matrix R and translation vector t, z c is the depth value of the camera, Expand: Write as: Eliminate z c , and after rearrangement, we get: Since the coordinates of 6 points are collected, solve the 6 - element linear equations: That is, f is obtained 11 f 12 f 13 f 14 f 21 f 22 f 23 f 24 f 31 f 32 f 33 f 34 parameter values of Therefore, the external parameter rotation matrix R and translation vector t of the camera are obtained as: Among them, K is the internal parameter matrix of the camera, f 11 、f 12 、f 13 、f 21 、f 22 、f 23 、f 31 、f 32 、f 33 represent the internal parameter values of the external rotation matrix R of the camera, f 14 、f 24 、f 34 represent the internal parameter values of the translation vector t of the camera.

Images