Checkerboard calibration method and system for 3D TOF module and readable storage medium

Abstract

The invention discloses a checkerboard calibration method and system for a 3D TOF module and a readable storage medium, and the method comprises the steps: collecting checkerboard images under different poses, and obtaining an image data pair which comprises a gray-scale map and a distance map; carrying out angular point extraction and sorting based on the grey-scale map to obtain a sub-pixel angular point coordinate set; generating world coordinates corresponding to each angular point based on the angular point coordinate set in combination with a preset checkerboard physical size, and calculating homography matrixes under different poses; performing internal reference calibration based on the homography matrix to obtain an internal reference and a distortion parameter; performing external parameter calibration to obtain external parameters; and performing precision verification based on the internal parameter, the distortion parameter and the external parameter, and outputting a corresponding calibration result when the verification is passed. According to the method, the multi-attitude image is introduced to be combined with the sub-pixel-level angular point optimization technology, so that the coordinate detection precision is remarkably improved, the reprojection error is optimized by adopting rotation matrix orthogonalization processing and maximum likelihood estimation, and the calibration precision of the internal parameters and the external parameters of the 3D TOF module is improved.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging technology, and more specifically, to a checkerboard calibration method, system, and readable storage medium for a 3D TOF module. Background Technology

[0002] 3D TOF (Time-of-Flight) modules calculate the distance to a target by measuring the flight time of light signals and generate a three-dimensional point cloud by combining the two-dimensional coordinates of a pixel array. They are widely used in fields such as facial recognition, industrial inspection, and autonomous driving.

[0003] The accuracy of 3D coordinate calculation directly depends on the calibration accuracy of the module's intrinsic and extrinsic parameters. Among existing calibration methods, the checkerboard calibration method is widely used due to its ease of operation and low cost, but it has the following shortcomings: First, the image acquisition pose is singular in the calibration process, resulting in insufficient homography matrix constraints and large errors in solving the intrinsic parameters. Secondly, corner point extraction only stays at the pixel level and does not perform sub-pixel optimization, which reduces the accuracy of coordinate correspondence. Third, the external parameters were not orthogonalized or optimized for reprojection error after decomposition, which makes it difficult to meet the needs of high-precision scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a checkerboard calibration method, system, and readable storage medium for 3D TOF modules. This invention solves the problems in existing checkerboard calibration methods, such as insufficient intrinsic parameter constraints due to the single image acquisition pose, corner point extraction accuracy remaining at the pixel level, and low calibration accuracy and difficulty in meeting the requirements of high-precision 3D imaging caused by the lack of orthogonalization and reprojection optimization after extrinsic parameter decomposition.

[0005] The first aspect of this invention provides a checkerboard calibration method for a 3D TOF module, comprising the following steps: Image data pairs are obtained by acquiring chessboard images in different poses, wherein the image data pairs include grayscale images and distance images; Based on the grayscale image, corner points are extracted and sorted to obtain a set of sub-pixel corner point coordinates; Based on the set of corner coordinates and the preset physical dimensions of the chessboard, the world coordinates corresponding to each corner are generated, and the homography matrix under different poses is calculated. Intrinsic parameters and distortion parameters are obtained by calibrating the intrinsic parameters based on the homography matrix; and extrinsic parameters are obtained by calibrating the extrinsic parameters. Accuracy verification is performed based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and the corresponding calibration result is output when the verification is successful.

[0006] In this solution, the acquisition of chessboard images at different poses to obtain image data pairs specifically includes: The 3D TOF module is fixed on a preset optical platform, and the upper left corner of the world coordinate system corresponding to the chessboard grid is the origin. Different poses are obtained, including frontal view, left deviation 30°, right deviation 30°, up deviation 20°, down deviation 20°, and different translation positions; The image data pairs are obtained by acquiring grayscale images and distance maps of the chessboard grid based on different poses.

[0007] In this solution, the step of extracting and sorting corner points based on the grayscale image to obtain a set of sub-pixel corner point coordinates specifically includes: The internal corner points corresponding to each grayscale image are extracted using a preset corner detection algorithm to obtain the initial pixel coordinates; Subpixel optimization is achieved by performing Gaussian fitting and interpolation on the grayscale gradient information of the grayscale image within a preset neighborhood. The initial pixel coordinates of the detected corner points are logically sorted based on the number of rows and columns of the checkerboard grid to obtain the set of corner point coordinates, wherein the precision of the corner point coordinates is 0.1 pixels.

[0008] In this scheme, the step of generating world coordinates for each corner point based on the set of corner coordinates and a preset physical size of the chessboard grid, and calculating the homography matrix under different poses, specifically includes: Obtain the physical dimensions corresponding to the chessboard grid, wherein the physical dimensions include 10mm; Calculate the world coordinates corresponding to each corner point based on the origin coordinates and the physical dimensions; The homography matrix under different poses is calculated based on the observed pixel coordinates and world coordinates corresponding to the corner points, wherein the homography matrix represents the projection matrix from the checkerboard plane to the image plane.

[0009] In this scheme, the step of obtaining intrinsic parameters and distortion parameters by calibrating intrinsic parameters based on the homography matrix, and obtaining extrinsic parameters by calibrating extrinsic parameters, specifically includes: During intrinsic parameter calibration, Zhang Zhengyou calibration constraint equations are constructed using a preset number of homography matrices, and the intrinsic parameters are solved using the least squares method. The distortion parameters are obtained by performing distortion correction on the observed pixel coordinates. When calibrating the extrinsic parameters, the homography matrix is ​​decomposed to obtain the rotation vector and translation vector, and the rotation vector is cross-multiplied to orthogonalize it to obtain the rotation matrix. The rotation matrix and translation vector under different poses are optimized based on the maximum likelihood estimation method to obtain the extrinsic parameters.

[0010] In this solution, the accuracy verification based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and the output of the corresponding calibration result upon successful verification, specifically includes: The intrinsic parameters, the distortion parameters, and the extrinsic parameters are extracted to obtain the calibration parameter package; Based on the calibration parameter package, the world coordinates of each corner point are reprojected onto the pixel coordinate system to obtain the theoretical pixel coordinates; The pixel error is calculated based on the observed pixel coordinates and theoretical pixel coordinates of the corner points, and the average reprojection error is obtained by averaging the pixel errors of all corner points. Compare the relationship between the average reprojection error and the preset threshold range, wherein, The verification is successful when the average reprojection error is within the preset threshold range, and the corresponding calibration parameter package is output. If the average reprojection error is not within the preset threshold range, then recalibrate.

[0011] A second aspect of the present invention also provides a checkerboard calibration system for a 3D TOF module, comprising a memory and a processor. The memory includes a checkerboard calibration method program for the 3D TOF module. When the checkerboard calibration method program for the 3D TOF module is executed by the processor, it performs the following steps: Image data pairs are obtained by acquiring chessboard images in different poses, wherein the image data pairs include grayscale images and distance images; Based on the grayscale image, corner points are extracted and sorted to obtain a set of sub-pixel corner point coordinates; Based on the set of corner coordinates and the preset physical dimensions of the chessboard, the world coordinates corresponding to each corner are generated, and the homography matrix under different poses is calculated. Intrinsic parameters and distortion parameters are obtained by calibrating the intrinsic parameters based on the homography matrix; and extrinsic parameters are obtained by calibrating the extrinsic parameters. Accuracy verification is performed based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and the corresponding calibration result is output when the verification is successful.

[0012] In this solution, the acquisition of chessboard images at different poses to obtain image data pairs specifically includes: The 3D TOF module is fixed on a preset optical platform, and the upper left corner of the world coordinate system corresponding to the chessboard grid is the origin. Different poses are obtained, including frontal view, left deviation 30°, right deviation 30°, up deviation 20°, down deviation 20°, and different translation positions; The image data pairs are obtained by acquiring grayscale images and distance maps of the chessboard grid based on different poses.

[0013] In this solution, the step of extracting and sorting corner points based on the grayscale image to obtain a set of sub-pixel corner point coordinates specifically includes: The internal corner points corresponding to each grayscale image are extracted using a preset corner detection algorithm to obtain the initial pixel coordinates; Subpixel optimization is achieved by performing Gaussian fitting and interpolation on the grayscale gradient information of the grayscale image within a preset neighborhood. The initial pixel coordinates of the detected corner points are logically sorted based on the number of rows and columns of the checkerboard grid to obtain the set of corner point coordinates, wherein the precision of the corner point coordinates is 0.1 pixels.

[0014] In this scheme, the step of generating world coordinates for each corner point based on the set of corner coordinates and a preset physical size of the chessboard grid, and calculating the homography matrix under different poses, specifically includes: Obtain the physical dimensions corresponding to the chessboard grid, wherein the physical dimensions include 10mm; Calculate the world coordinates corresponding to each corner point based on the origin coordinates and the physical dimensions; The homography matrix under different poses is calculated based on the observed pixel coordinates and world coordinates corresponding to the corner points, wherein the homography matrix represents the projection matrix from the checkerboard plane to the image plane.

[0015] In this scheme, the step of obtaining intrinsic parameters and distortion parameters by calibrating intrinsic parameters based on the homography matrix, and obtaining extrinsic parameters by calibrating extrinsic parameters, specifically includes: During intrinsic parameter calibration, Zhang Zhengyou calibration constraint equations are constructed using a preset number of homography matrices, and the intrinsic parameters are solved using the least squares method. The distortion parameters are obtained by performing distortion correction on the observed pixel coordinates. When calibrating the extrinsic parameters, the homography matrix is ​​decomposed to obtain the rotation vector and translation vector, and the rotation vector is cross-multiplied to orthogonalize it to obtain the rotation matrix. The rotation matrix and translation vector under different poses are optimized based on the maximum likelihood estimation method to obtain the extrinsic parameters.

[0016] In this solution, the accuracy verification based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and the output of the corresponding calibration result upon successful verification, specifically includes: The intrinsic parameters, the distortion parameters, and the extrinsic parameters are extracted to obtain the calibration parameter package; Based on the calibration parameter package, the world coordinates of each corner point are reprojected onto the pixel coordinate system to obtain the theoretical pixel coordinates; The pixel error is calculated based on the observed pixel coordinates and theoretical pixel coordinates of the corner points, and the average reprojection error is obtained by averaging the pixel errors of all corner points. Compare the relationship between the average reprojection error and the preset threshold range, wherein, The verification is successful when the average reprojection error is within the preset threshold range, and the corresponding calibration parameter package is output. If the average reprojection error is not within the preset threshold range, then recalibrate.

[0017] A third aspect of the present invention provides a computer-readable storage medium including a checkerboard calibration method program for a 3D TOF module of a machine, wherein when the checkerboard calibration method program for the 3D TOF module is executed by a processor, the checkerboard calibration method program for the 3D TOF module implements the steps of the checkerboard calibration method for the 3D TOF module as described in any of the preceding claims.

[0018] The present invention discloses a checkerboard calibration method, system, and readable storage medium for a 3D Time-of-Flight (TOF) module, which has the following advantages: 1. By using 15 to 20 sets of images with diverse poses, the constraints of the homography matrix are increased, the uncertainty of intrinsic parameter solution is reduced, and the accuracy of intrinsic parameter calibration is improved. 2. By optimizing corner coordinates through sub-pixel interpolation, the corner detection accuracy is improved from the pixel level to the sub-pixel level, reducing the error in coordinate correspondence; 3. The rotation matrix is ​​orthogonalized, and the reprojection error is optimized by maximum likelihood estimation, ensuring that the extrinsic parameters meet the orthogonality constraint of the rotation matrix, thereby further improving the calibration accuracy of the extrinsic parameters; 4. High-precision ceramic checkerboard targets are selected to reduce errors caused by target deformation. At the same time, a reprojection error verification step is set up to ensure the reliability of the calibration results. Attached Figure Description

[0019] Figure 1 A flowchart of a checkerboard calibration method for a 3D TOF module according to the present invention is shown; Figure 2 A schematic diagram of a checkerboard grid for a 3D TOF module calibration method according to the present invention is shown. Figure 3 A block diagram of a checkerboard calibration system for a 3D TOF module according to the present invention is shown. Detailed Implementation

[0020] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0022] This invention relates to the field of three-dimensional imaging technology, specifically to a checkerboard calibration method for 3D TOF (Time-of-Flight) modules, used to accurately solve the intrinsic and extrinsic parameters of 3D TOF modules. The aim is to overcome the problem of insufficient accuracy in the checkerboard calibration of 3D TOF modules in existing technologies, and to provide a simple and highly accurate checkerboard calibration method for 3D TOF modules. By optimizing the image acquisition, corner point extraction, and parameter solving processes, the accuracy of the intrinsic and extrinsic parameter calibration is improved.

[0023] Figure 1 A flowchart of a checkerboard calibration method for a 3D TOF module according to this application is shown.

[0024] like Figure 1 As shown, this application discloses a checkerboard calibration method for a 3D TOF module, including the following steps: S102, acquire chessboard images under different poses to obtain image data pairs, wherein the image data pairs include grayscale images and distance images; S104, Based on the grayscale image, corner points are extracted and sorted to obtain a set of sub-pixel corner point coordinates; S106, Based on the set of corner coordinates and the preset physical dimensions of the chessboard, generate the world coordinates corresponding to each corner point, and calculate the homography matrix under different poses; S108, based on the homography matrix, perform intrinsic parameter calibration to obtain intrinsic parameters and distortion parameters; and perform extrinsic parameter calibration to obtain extrinsic parameters; S110, perform accuracy verification based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and output the corresponding calibration result when the verification is successful.

[0025] It should be noted that, in this embodiment, the calibration of the checkerboard pattern specifically involves calibrating the camera's intrinsic, distortion, and extrinsic parameters. Therefore, it is necessary to first acquire checkerboard images in different poses to obtain image data pairs. These image data pairs include grayscale images and distance maps, where the distance map plays a crucial but not primary role. Its main function is to verify the calibration results and integrate the system, rather than directly solving for the camera's intrinsic and extrinsic parameters. When calculating the calibration parameters, the calculations are specifically based on the grayscale images. Specifically, corner points are extracted and sorted based on the grayscale images to obtain a set of sub-pixel corner coordinates. The world coordinates corresponding to each corner point are generated based on the set of corner coordinates and the preset physical dimensions of the checkerboard pattern. The homography matrix under different poses is then calculated. The homography matrix is ​​used for parameter calibration to obtain the intrinsic, distortion, and extrinsic parameters. Finally, the accuracy is verified based on the intrinsic, distortion, and extrinsic parameters, and the corresponding calibration result is output when the verification is successful.

[0026] According to an embodiment of the present invention, the acquisition of image data pairs from chessboard images in different poses specifically includes: The 3D TOF module is fixed on a preset optical platform, and the upper left corner of the world coordinate system corresponding to the chessboard grid is the origin. Different poses are obtained, including frontal view, left deviation 30°, right deviation 30°, up deviation 20°, down deviation 20°, and different translation positions; The image data pairs are obtained by acquiring grayscale images and distance maps of the chessboard grid based on different poses.

[0027] It should be noted that, in this embodiment, a high-precision checkerboard target needs to be prepared before calibration and fixed on the calibration platform of the 3D TOF module. Simultaneously, a corresponding world coordinate system is defined, with the first interior corner point of the upper left corner of the checkerboard as the origin, and the checkerboard plane as... flat, Synchronously record the physical dimensions of the chessboard squares Among them, such as Figure 2 As shown, a chessboard pattern is displayed. Specifically, a ceramic chessboard with "9×6" interior corner points and a physical size of "10mm" is selected. Furthermore, during image acquisition, the pose of the 3D TOF module remains unchanged, and images of the chessboard target in "15~20" different poses are acquired (covering the center of the field of view and rotation angles within ±30°). This allows for the acquisition of grayscale images and distance maps for each pose. The poses include frontal view, 30° left deviation, 30° right deviation, 20° upward deviation, 20° downward deviation, and different translational positions. Grayscale images and distance maps are acquired for each pose to ensure that the chessboard is unobstructed and free of reflections.

[0028] According to an embodiment of the present invention, the step of extracting and sorting corner points based on the grayscale image to obtain a set of sub-pixel corner point coordinates specifically includes: The internal corner points corresponding to each grayscale image are extracted using a preset corner detection algorithm to obtain the initial pixel coordinates; Subpixel optimization is achieved by performing Gaussian fitting and interpolation on the grayscale gradient information of the grayscale image within a preset neighborhood. The initial pixel coordinates of the detected corner points are logically sorted based on the number of rows and columns of the checkerboard grid to obtain the set of corner point coordinates, wherein the precision of the corner point coordinates is 0.1 pixels.

[0029] It should be noted that, in this embodiment, the observed pixel coordinates of the inner corner points are extracted using Shi-Tomasi (corner detection algorithm) to obtain the initial pixel coordinates for each grayscale image. Then, sub-pixel optimization is performed using Gaussian fitting interpolation method in a "3×3" neighborhood. The detected initial pixel coordinates of the corner points are logically sorted based on the number of rows and columns of the checkerboard to obtain the set of corner coordinates. The precision of the corner coordinates is 0.1 pixels.

[0030] According to an embodiment of the present invention, the step of generating world coordinates corresponding to each corner point based on the set of corner coordinates combined with a preset physical size of the chessboard grid, and calculating the homography matrix under different poses, specifically includes: Obtain the physical dimensions corresponding to the chessboard grid, wherein the physical dimensions include 10mm; Calculate the world coordinates corresponding to each corner point based on the origin coordinates and the physical dimensions; The homography matrix under different poses is calculated based on the observed pixel coordinates and world coordinates corresponding to the corner points, wherein the homography matrix represents the projection matrix from the checkerboard plane to the image plane.

[0031] It should be noted that, in this embodiment, the configuration is based on the origin (top left corner of the chessboard) and the physical dimensions of the squares. It automatically calculates the world coordinates corresponding to each corner point, for example, the first corner point... Line number The coordinates of the corner points of the column are For each pose, the observed pixel coordinates of all corner points in that pose are used. and world coordinates Based on the pinhole camera model, a homography matrix is ​​obtained by solving linear methods such as the least squares method. , is used to describe the projection transformation matrix from the checkerboard plane to the image plane under a specific pose.

[0032] According to an embodiment of the present invention, the step of obtaining intrinsic parameters and distortion parameters by calibrating intrinsic parameters based on the homography matrix, and obtaining extrinsic parameters by calibrating extrinsic parameters, specifically includes: During intrinsic parameter calibration, Zhang Zhengyou calibration constraint equations are constructed using a preset number of homography matrices, and the intrinsic parameters are solved using the least squares method. The distortion parameters are obtained by performing distortion correction on the observed pixel coordinates. When calibrating the extrinsic parameters, the homography matrix is ​​decomposed to obtain the rotation vector and translation vector, and the rotation vector is cross-multiplied to orthogonalize it to obtain the rotation matrix. The rotation matrix and translation vector under different poses are optimized based on the maximum likelihood estimation method to obtain the extrinsic parameters.

[0033] It should be noted that, in this embodiment, during intrinsic parameter calibration, Zhang Zhengyou's calibration constraint equations are constructed using a preset number of homography matrices, and the intrinsic parameters are solved using the least squares method. Furthermore, distortion parameters are obtained by performing distortion correction on the observed pixel coordinates. Specifically, this is achieved using 20 sets of homography matrices. Construct Zhang Zhengyou's calibration constraint equations and solve the corresponding intrinsic parameter matrix using the least squares method. ,in, The distortion parameters are obtained by simultaneously performing distortion correction on the observed pixel coordinates of the corner points, wherein... , This is the distortion parameter.

[0034] Furthermore, in this embodiment, during extrinsic parameter calibration, the homography matrix is ​​decomposed to obtain rotation and translation vectors, and the rotation vectors are cross-multiplied to orthogonalize them to obtain a rotation matrix. The rotation matrix and translation vectors under different poses are optimized based on the maximum likelihood estimation method to obtain the extrinsic parameters, i.e., combined with the intrinsic parameter matrix. Decomposition of homography matrix Obtain the rotation matrix Translation vector ,right Orthogonalization is performed, and then the reprojection error is minimized through maximum likelihood estimation to optimize the extrinsic parameters. Specifically, the homography matrix is ​​decomposed. get and calculate And orthogonalize to obtain the rotation matrix. Translation vector The reprojection error is optimized by maximum likelihood estimation to reduce the average reprojection error. In one embodiment, the average reprojection error can be reduced to "0.3" pixels.

[0035] Furthermore, it should be noted that the formula for homography matrix decomposition is: ,in The formula for calculating reprojection error is: ,in, To observe pixel coordinates, Calculate pixel coordinates theoretically.

[0036] According to an embodiment of the present invention, the step of performing accuracy verification based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and outputting the corresponding calibration result when the verification passes, specifically includes: The intrinsic parameters, the distortion parameters, and the extrinsic parameters are extracted to obtain the calibration parameter package; Based on the calibration parameter package, the world coordinates of each corner point are reprojected onto the pixel coordinate system to obtain the theoretical pixel coordinates; The pixel error is calculated based on the observed pixel coordinates and theoretical pixel coordinates of the corner points, and the average reprojection error is obtained by averaging the pixel errors of all corner points. Compare the relationship between the average reprojection error and the preset threshold range, wherein, The verification is successful when the average reprojection error is within the preset threshold range, and the corresponding calibration parameter package is output. If the average reprojection error is not within the preset threshold range, then recalibrate.

[0037] It should be noted that, in this embodiment, the intrinsic parameters, the distortion parameters, and the extrinsic parameters are extracted to obtain a calibration parameter package, and then the world coordinates of each corner point are reprojected to the pixel coordinate system based on the calibration parameter package to obtain the theoretical pixel coordinates and complete the reprojection.

[0038] Furthermore, in this embodiment, the pixel error is calculated based on the observed pixel coordinates and theoretical pixel coordinates of the corner points, and the average reprojection error is obtained by averaging the pixel errors of all corner points. Then, the relationship between the average reprojection error and a preset threshold range is compared, wherein the preset threshold range is [0.3 pixels, 0.5 pixels]. When the average reprojection error is within the preset threshold range, the verification is successful, and the corresponding calibration parameter package is output; when the average reprojection error is not within the preset threshold range, recalibration is performed.

[0039] Figure 3 A block diagram of a checkerboard calibration system for a 3D TOF module according to the present invention is shown.

[0040] like Figure 3 As shown, this invention discloses a checkerboard calibration system for a 3D TOF module, including a memory and a processor. The memory includes a checkerboard calibration method program for the 3D TOF module. When the processor executes the checkerboard calibration method program for the 3D TOF module, it performs the following steps: Image data pairs are obtained by acquiring chessboard images in different poses, wherein the image data pairs include grayscale images and distance images; Based on the grayscale image, corner points are extracted and sorted to obtain a set of sub-pixel corner point coordinates; Based on the set of corner coordinates and the preset physical dimensions of the chessboard, the world coordinates corresponding to each corner are generated, and the homography matrix under different poses is calculated. Intrinsic parameters and distortion parameters are obtained by calibrating the intrinsic parameters based on the homography matrix; and extrinsic parameters are obtained by calibrating the extrinsic parameters. Accuracy verification is performed based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and the corresponding calibration result is output when the verification is successful.

[0041] It should be noted that, in this embodiment, the calibration of the chessboard specifically involves calibrating the camera's intrinsic parameters, distortion parameters, and extrinsic parameters. Therefore, it is necessary to first acquire chessboard images in different poses to obtain image data pairs, which include grayscale images and distance images. Then, based on the grayscale images, corner points are extracted and sorted to obtain a set of sub-pixel corner coordinates. Based on the set of corner coordinates and the preset physical size of the chessboard, world coordinates corresponding to each corner point are generated, and the homography matrix in different poses is calculated. The homography matrix is ​​used to calibrate the parameters to obtain the intrinsic parameters, distortion parameters, and extrinsic parameters. Finally, the accuracy is verified based on the intrinsic parameters, distortion parameters, and extrinsic parameters, and the corresponding calibration result is output when the verification is successful.

[0042] According to an embodiment of the present invention, the acquisition of image data pairs from chessboard images in different poses specifically includes: The 3D TOF module is fixed on a preset optical platform, and the upper left corner of the world coordinate system corresponding to the chessboard grid is the origin. Different poses are obtained, including frontal view, left deviation 30°, right deviation 30°, up deviation 20°, down deviation 20°, and different translation positions; The image data pairs are obtained by acquiring grayscale images and distance maps of the chessboard grid based on different poses.

[0043] It should be noted that, in this embodiment, a high-precision checkerboard target needs to be prepared before calibration and fixed on the calibration platform of the 3D TOF module. Simultaneously, a corresponding world coordinate system is defined, with the first interior corner point of the upper left corner of the checkerboard as the origin, and the checkerboard plane as... flat, Synchronously record the physical dimensions of the chessboard squares Among them, such as Figure 2As shown, a chessboard pattern is displayed. Specifically, a ceramic chessboard with "9×6" interior corner points and a physical size of "10mm" is selected. Furthermore, during image acquisition, the pose of the 3D TOF module remains unchanged, and images of the chessboard target in "15~20" different poses are acquired (covering the center of the field of view and rotation angles within ±30°). This allows for the acquisition of grayscale images and distance maps for each pose. The poses include frontal view, 30° left deviation, 30° right deviation, 20° upward deviation, 20° downward deviation, and different translational positions. Grayscale images and distance maps are acquired for each pose to ensure that the chessboard is unobstructed and free of reflections.

[0044] According to an embodiment of the present invention, the step of extracting and sorting corner points based on the grayscale image to obtain a set of sub-pixel corner point coordinates specifically includes: The internal corner points corresponding to each grayscale image are extracted using a preset corner detection algorithm to obtain the initial pixel coordinates; Subpixel optimization is achieved by performing Gaussian fitting and interpolation on the grayscale gradient information of the grayscale image within a preset neighborhood. The initial pixel coordinates of the detected corner points are logically sorted based on the number of rows and columns of the checkerboard grid to obtain the set of corner point coordinates, wherein the precision of the corner point coordinates is 0.1 pixels.

[0045] It should be noted that, in this embodiment, the observed pixel coordinates of the inner corner points are extracted using Shi-Tomasi (corner detection algorithm) to obtain the initial pixel coordinates for each grayscale image. Then, sub-pixel optimization is performed using Gaussian fitting interpolation method in a "3×3" neighborhood. The detected initial pixel coordinates of the corner points are logically sorted based on the number of rows and columns of the checkerboard to obtain the set of corner coordinates. The precision of the corner coordinates is 0.1 pixels.

[0046] According to an embodiment of the present invention, the step of generating world coordinates corresponding to each corner point based on the set of corner coordinates combined with a preset physical size of the chessboard grid, and calculating the homography matrix under different poses, specifically includes: Obtain the physical dimensions corresponding to the chessboard grid, wherein the physical dimensions include 10mm; Calculate the world coordinates corresponding to each corner point based on the origin coordinates and the physical dimensions; The homography matrix under different poses is calculated based on the observed pixel coordinates and world coordinates corresponding to the corner points, wherein the homography matrix represents the projection matrix from the checkerboard plane to the image plane.

[0047] It should be noted that, in this embodiment, the configuration is based on the origin (top left corner of the chessboard) and the physical dimensions of the squares. It automatically calculates the world coordinates corresponding to each corner point, for example, the first corner point... Line number The coordinates of the corner points of the column are For each pose, the observed pixel coordinates of all corner points in that pose are used. and world coordinates Based on the pinhole camera model, a homography matrix is ​​obtained by solving linear methods such as the least squares method. , is used to describe the projection transformation matrix from the checkerboard plane to the image plane under a specific pose.

[0048] According to an embodiment of the present invention, the step of obtaining intrinsic parameters and distortion parameters by calibrating intrinsic parameters based on the homography matrix, and obtaining extrinsic parameters by calibrating extrinsic parameters, specifically includes: During intrinsic parameter calibration, Zhang Zhengyou calibration constraint equations are constructed using a preset number of homography matrices, and the intrinsic parameters are solved using the least squares method. The distortion parameters are obtained by performing distortion correction on the observed pixel coordinates. When calibrating the extrinsic parameters, the homography matrix is ​​decomposed to obtain the rotation vector and translation vector, and the rotation vector is cross-multiplied to orthogonalize it to obtain the rotation matrix. The rotation matrix and translation vector under different poses are optimized based on the maximum likelihood estimation method to obtain the extrinsic parameters.

[0049] It should be noted that, in this embodiment, during intrinsic parameter calibration, Zhang Zhengyou's calibration constraint equations are constructed using a preset number of homography matrices, and the intrinsic parameters are solved using the least squares method. Furthermore, distortion parameters are obtained by performing distortion correction on the observed pixel coordinates. Specifically, this is achieved using 20 sets of homography matrices. Construct Zhang Zhengyou's calibration constraint equations and solve the corresponding intrinsic parameter matrix using the least squares method. ,in, The distortion parameters are obtained by simultaneously performing distortion correction on the observed pixel coordinates of the corner points, wherein... , This is the distortion parameter.

[0050] Furthermore, in this embodiment, during extrinsic parameter calibration, the homography matrix is ​​decomposed to obtain rotation and translation vectors, and the rotation vectors are cross-multiplied to orthogonalize them to obtain a rotation matrix. The rotation matrix and translation vectors under different poses are optimized based on the maximum likelihood estimation method to obtain the extrinsic parameters, i.e., combined with the intrinsic parameter matrix. Decomposition of homography matrix Obtain the rotation matrix Translation vector ,right Orthogonalization is performed, and then the reprojection error is minimized through maximum likelihood estimation to optimize the extrinsic parameters. Specifically, the homography matrix is ​​decomposed. get and calculate And orthogonalize to obtain the rotation matrix. Translation vector The reprojection error is optimized by maximum likelihood estimation to reduce the average reprojection error. In one embodiment, the average reprojection error can be reduced to "0.3" pixels.

[0051] Furthermore, it should be noted that the formula for homography matrix decomposition is: ,in The formula for calculating reprojection error is: ,in, To observe pixel coordinates, Calculate pixel coordinates theoretically.

[0052] According to an embodiment of the present invention, the step of performing accuracy verification based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and outputting the corresponding calibration result when the verification passes, specifically includes: The intrinsic parameters, the distortion parameters, and the extrinsic parameters are extracted to obtain the calibration parameter package; Based on the calibration parameter package, the world coordinates of each corner point are reprojected onto the pixel coordinate system to obtain the theoretical pixel coordinates; The pixel error is calculated based on the observed pixel coordinates and theoretical pixel coordinates of the corner points, and the average reprojection error is obtained by averaging the pixel errors of all corner points. Compare the relationship between the average reprojection error and the preset threshold range, wherein, The verification is successful when the average reprojection error is within the preset threshold range, and the corresponding calibration parameter package is output. If the average reprojection error is not within the preset threshold range, then recalibrate.

[0053] It should be noted that, in this embodiment, the intrinsic parameters, the distortion parameters, and the extrinsic parameters are extracted to obtain a calibration parameter package, and then the world coordinates of each corner point are reprojected to the pixel coordinate system based on the calibration parameter package to obtain the theoretical pixel coordinates and complete the reprojection.

[0054] Furthermore, in this embodiment, the pixel error is calculated based on the observed pixel coordinates and theoretical pixel coordinates of the corner points, and the average reprojection error is obtained by averaging the pixel errors of all corner points. Then, the relationship between the average reprojection error and a preset threshold range is compared, wherein the preset threshold range is [0.3 pixels, 0.5 pixels]. When the average reprojection error is within the preset threshold range, the verification is successful, and the corresponding calibration parameter package is output; when the average reprojection error is not within the preset threshold range, recalibration is performed.

[0055] A third aspect of the present invention provides a computer-readable storage medium including a checkerboard calibration method program for a 3D TOF module. When the checkerboard calibration method program for the 3D TOF module is executed by a processor, it implements the steps of a checkerboard calibration method for a 3D TOF module as described in any of the preceding claims.

[0056] This invention discloses a checkerboard calibration method, system, and readable storage medium for a 3D Time-of-Flight (TOF) module. By introducing multi-pose image acquisition to enhance homography matrix constraints and combining it with sub-pixel-level corner optimization technology, the accuracy of coordinate detection is significantly improved. Furthermore, rotation matrix orthogonalization and maximum likelihood estimation are used to optimize reprojection errors, thereby systematically improving the calibration accuracy of the intrinsic and extrinsic parameters of the 3D TOF module. Simultaneously, the use of a high-precision ceramic checkerboard target and the establishment of a reprojection error verification mechanism ensure the reliability and stability of the calibration results, ultimately achieving a 3D coordinate calculation error of less than 0.5%, which can meet the needs of high-precision applications such as industrial inspection and autonomous driving.

[0057] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0058] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0059] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0060] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0061] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A checkerboard calibration method for a 3D TOF module, characterized in that, Includes the following steps: Image data pairs are obtained by acquiring chessboard images in different poses, wherein the image data pairs include grayscale images and distance images; Based on the grayscale image, corner points are extracted and sorted to obtain a set of sub-pixel corner point coordinates; Based on the set of corner coordinates and the preset physical dimensions of the chessboard, the world coordinates corresponding to each corner are generated, and the homography matrix under different poses is calculated. Intrinsic parameters and distortion parameters are obtained by calibrating the intrinsic parameters based on the homography matrix; and extrinsic parameters are obtained by calibrating the extrinsic parameters. Accuracy verification is performed based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and the corresponding calibration result is output when the verification is successful.

2. The checkerboard calibration method for a 3D TOF module according to claim 1, characterized in that, The acquisition of chessboard images in different poses to obtain image data pairs specifically includes: The 3D TOF module is fixed on a preset optical platform, and the upper left corner of the world coordinate system corresponding to the chessboard grid is the origin. Different poses are obtained, including frontal view, left deviation 30°, right deviation 30°, up deviation 20°, down deviation 20°, and different translation positions; The image data pairs are obtained by acquiring grayscale images and distance maps of the chessboard grid based on different poses.

3. The checkerboard calibration method for a 3D TOF module according to claim 2, characterized in that, The step of extracting and sorting corner points based on the grayscale image to obtain a set of sub-pixel corner point coordinates specifically includes: The internal corner points corresponding to each grayscale image are extracted using a preset corner detection algorithm to obtain the initial pixel coordinates; Subpixel optimization is achieved by performing Gaussian fitting and interpolation on the grayscale gradient information of the grayscale image within a preset neighborhood. The initial pixel coordinates of the detected corner points are logically sorted based on the number of rows and columns of the checkerboard grid to obtain the set of corner point coordinates, wherein the precision of the corner point coordinates is 0.1 pixels.

4. The checkerboard calibration method for a 3D TOF module according to claim 3, characterized in that, The process of generating world coordinates for each corner point based on the set of corner coordinates and a preset physical size of the chessboard grid, and calculating the homography matrix under different poses, specifically includes: Obtain the physical dimensions corresponding to the chessboard grid, wherein the physical dimensions include 10mm; Calculate the world coordinates corresponding to each corner point based on the origin coordinates and the physical dimensions; The homography matrix under different poses is calculated based on the observed pixel coordinates and world coordinates corresponding to the corner points, wherein the homography matrix represents the projection matrix from the checkerboard plane to the image plane.

5. The checkerboard calibration method for a 3D TOF module according to claim 4, characterized in that, The process of obtaining intrinsic parameters and distortion parameters by calibrating intrinsic parameters based on the homography matrix, and obtaining extrinsic parameters by calibrating extrinsic parameters, specifically includes: During intrinsic parameter calibration, Zhang Zhengyou calibration constraint equations are constructed using a preset number of homography matrices, and the intrinsic parameters are solved using the least squares method. The distortion parameters are obtained by performing distortion correction on the observed pixel coordinates. When calibrating the extrinsic parameters, the homography matrix is ​​decomposed to obtain the rotation vector and translation vector, and the rotation vector is cross-multiplied to orthogonalize it to obtain the rotation matrix. The rotation matrix and translation vector under different poses are optimized based on the maximum likelihood estimation method to obtain the extrinsic parameters.

6. The checkerboard calibration method for a 3D TOF module according to claim 5, characterized in that, The process of performing accuracy verification based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and outputting the corresponding calibration result upon successful verification, specifically includes: The intrinsic parameters, the distortion parameters, and the extrinsic parameters are extracted to obtain the calibration parameter package; Based on the calibration parameter package, the world coordinates of each corner point are reprojected onto the pixel coordinate system to obtain the theoretical pixel coordinates; The pixel error is calculated based on the observed pixel coordinates and theoretical pixel coordinates of the corner points, and the average reprojection error is obtained by averaging the pixel errors of all corner points. Compare the relationship between the average reprojection error and the preset threshold range, wherein, The verification is successful when the average reprojection error is within the preset threshold range, and the corresponding calibration parameter package is output. If the average reprojection error is not within the preset threshold range, then recalibrate.

7. A checkerboard calibration system for a 3D TOF module, characterized in that, The system includes a memory and a processor. The memory contains a checkerboard calibration method program for a 3D Time-of-Flight (TOF) module. When the processor executes the checkerboard calibration method program for the 3D TOF module, it performs the following steps: Image data pairs are obtained by acquiring chessboard images in different poses, wherein the image data pairs include grayscale images and distance images; Based on the grayscale image, corner points are extracted and sorted to obtain a set of sub-pixel corner point coordinates; Based on the set of corner coordinates and the preset physical dimensions of the chessboard, the world coordinates corresponding to each corner are generated, and the homography matrix under different poses is calculated. Intrinsic parameters and distortion parameters are obtained by calibrating the intrinsic parameters based on the homography matrix; and extrinsic parameters are obtained by calibrating the extrinsic parameters. Accuracy verification is performed based on the intrinsic parameters, the distortion parameters, and the extrinsic parameters, and the corresponding calibration result is output when the verification is successful.

8. A checkerboard calibration system for a 3D TOF module according to claim 7, characterized in that, The acquisition of chessboard images in different poses to obtain image data pairs specifically includes: The 3D TOF module is fixed on a preset optical platform, and the upper left corner of the world coordinate system corresponding to the chessboard grid is the origin. Different poses are obtained, including frontal view, left deviation 30°, right deviation 30°, up deviation 20°, down deviation 20°, and different translation positions; The image data pairs are obtained by acquiring grayscale images and distance maps of the chessboard grid based on different poses.

9. A checkerboard calibration system for a 3D TOF module according to claim 8, characterized in that, The step of extracting and sorting corner points based on the grayscale image to obtain a set of sub-pixel corner point coordinates specifically includes: The internal corner points corresponding to each grayscale image are extracted using a preset corner detection algorithm to obtain the initial pixel coordinates; Subpixel optimization is achieved by performing Gaussian fitting and interpolation on the grayscale gradient information of the grayscale image within a preset neighborhood. The initial pixel coordinates of the detected corner points are logically sorted based on the number of rows and columns of the checkerboard grid to obtain the set of corner point coordinates, wherein the precision of the corner point coordinates is 0.1 pixels.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a checkerboard calibration method program for a 3D TOF module. When the checkerboard calibration method program for the 3D TOF module is executed by a processor, it implements the steps of a checkerboard calibration method for a 3D TOF module as described in any one of claims 1 to 6.

Images