Calibration system based on TOF camera

Abstract

The invention relates to the technical field of TOF camera calibration, in particular to a TOF camera-based calibration system, which comprises the following steps of: acquiring an image and point cloud data of a calibration board, and setting a checkerboard pattern on the calibration board for providing a calibration reference for a TOF camera. According to the system, the three-dimensional coordinates of the checkerboard angular points can be more accurately recovered, so that the precision of pose calibration between the TOF camera and the vehicle body is improved, the calibration plate plane equation is used as a constraint condition, the 3D coordinates of the checkerboard angular points are accurately recovered in combination with the homography matrix, the noise influence is effectively removed, and the accuracy of a calibration result is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of TOF camera calibration, and in particular to a calibration system based on a TOF camera. Background Art

[0002] With the vigorous development of the logistics industry, automated warehousing and logistics systems have gradually become the mainstream of the industry. Among them, automatic guided vehicles (AGVs), as key equipment, play a vital role in the handling, storage and sorting of goods. The precise operation ability of AGVs, especially its accuracy in picking up and placing goods, is directly related to the efficiency and accuracy of the entire logistics system.

[0003] When AGV performs the task of picking up and placing goods, the precise positioning of the goods is the basis for ensuring the accuracy of the operation. However, in actual applications, due to the limitations of warehouse design, goods placement and the AGV's own positioning system, it is often difficult to avoid certain positioning errors. Especially when there are no limit blocks to restrict the goods, this error may gradually accumulate, causing the AGV to deviate from the predetermined position when picking up goods, affecting the efficiency and accuracy of the operation.

[0004] To solve this problem, AGVs are usually equipped with various sensors, such as Time-of-Flight (TOF) cameras, to obtain the actual posture information of the goods in real time and correct the picking deviation based on this information. As an advanced sensor, TOF cameras can provide two-dimensional image data and three-dimensional point cloud data at the same time, providing AGV with rich environmental perception capabilities.

[0005] However, despite the many advantages of TOF cameras, they also face some challenges in practical applications. Among them, the accurate position calibration between the TOF camera and the AGV body is a key issue. Due to various factors in the installation process, such as the unevenness of the installation surface and the accuracy limitations of the installation tools, it is often difficult for the TOF camera to achieve completely accurate alignment with the AGV body. This installation error, even if it is only 1 to 2 degrees, may cause significant deviations when the AGV picks up goods, thereby affecting the operation effect.

[0006] Therefore, how to quickly and accurately calibrate the precise position of the TOF camera and the AGV body has become the key to improving the accuracy and efficiency of AGV operations. Traditional calibration methods are often complicated and require professional equipment and skills, which not only increases the calibration cost but also reduces the efficiency of AGV. Therefore, developing a simple, efficient and accurate TOF camera calibration system is of great significance to promoting the widespread application of AGV in the warehousing and logistics industry. Summary of the invention

[0007] (I) Purpose of the invention

[0008] In view of this, the purpose of the present invention is to propose a calibration system based on a TOF camera, which can complete the external parameter calibration by combining the 2D image data and 3D point cloud data of the checkerboard calibration plate, thereby solving the problems raised in the above background.

[0009] (II) Technical solution

[0010] A TOF camera-based calibration system comprises the following steps:

[0011] 1): Obtain the image and point cloud data of the calibration plate, where a checkerboard pattern is set on the calibration plate to provide a calibration reference for the TOF camera;

[0012] 2): Construct a pixel coordinate matrix based on the pixel coordinates of the checkerboard corner points in the calibration plate image;

[0013] 3): Using the sampling consistency algorithm, the pixel coordinate matrix and the corresponding noisy 3D coordinates are combined to calculate the homography matrix, and the plane equation in the camera coordinate system is fitted through the point cloud data of the calibration plate. Based on the plane equation constraint and the homography matrix, the denoised 3D coordinates are restored from the pixel coordinates of the checkerboard corner points;

[0014] 4): Using the SVD (singular value decomposition) algorithm, the position and posture of the TOF camera relative to the calibration board coordinate system are calculated based on the denoised 3D coordinates of the checkerboard corner points in the calibration board coordinate system and the camera coordinate system;

[0015] 5): Set the calibration plate coordinate system to be consistent with the vehicle coordinate system, so that the rotation matrix of the calibration plate coordinate system relative to the vehicle coordinate system is the unit matrix, and obtain the translation vector of the calibration plate coordinate system relative to the vehicle coordinate system by measurement, so as to combine the position matrix of the calibration plate coordinate system relative to the vehicle coordinate system;

[0016] 6): According to the posture information obtained in step 4) and step 3), start calculating and outputting the posture of the TOF camera relative to the vehicle body coordinate system, and then end.

[0017] Preferably, the calibration plate is a plane calibration plate with a checkerboard pattern, which is used to provide an accurate calibration reference for the TOF camera.

[0018] Preferably, in step 3), the plane equation of the calibration plate is used as a constraint condition, combined with the homography matrix, to accurately restore the 3D coordinates of the corner points of the chessboard to remove the influence of noise.

[0019] Preferably, in step 5), the actual distance between the calibration plate and the vehicle body is measured by a measuring tool such as a tape measure to determine the accurate value of the translation vector.

[0020] Preferably, the TOF camera is a time-domain flight camera capable of simultaneously acquiring images and point cloud data.

[0021] Preferably, the TOF camera, calibration plate, image processing unit, point cloud processing unit and posture calculation unit, wherein: the TOF camera is used to capture an image of the calibration plate and obtain point cloud data;

[0022] The calibration plate is a flat plate with a checkerboard pattern, used to provide a calibration reference for the TOF camera;

[0023] The image processing unit is used to detect the pixel coordinates of the corner points of the chessboard in the image;

[0024] The point cloud processing unit is used to process the point cloud data of the calibration plate, fit the plane equation, and restore the 3D coordinates after denoising;

[0025] The pose calculation unit is used to calculate and output the pose of the TOF camera relative to the vehicle body coordinate system based on the restored 3D coordinates and the relative position relationship between the calibration plate and the vehicle body.

[0026] It can be seen from the above technical solutions that the present application has the following beneficial effects:

[0027] 1. By combining the two-dimensional image data and three-dimensional point cloud data of the chessboard calibration plate, the system can more accurately restore the three-dimensional coordinates of the corner points of the chessboard, thereby improving the accuracy of the pose calibration between the TOF camera and the vehicle body. The plane equation of the calibration plate is used as a constraint condition combined with the homography matrix to accurately restore the 3D coordinates of the corner points of the chessboard, effectively eliminating the influence of noise and improving the accuracy of the calibration results.

[0028] 2. This invention avoids the cumbersome equipment and operations in traditional calibration methods. It can realize the rapid calibration of TOF camera by using only a flat calibration plate with a checkerboard pattern. Through the automated image processing, point cloud processing and posture calculation unit, it greatly simplifies the calibration process and reduces the operation difficulty and labor cost.

[0029] 3. The precise camera calibration of the invention can ensure that the AGV can accurately identify the location of goods when picking up and placing goods, avoiding the problem of deviation caused by camera installation errors, and can significantly improve the operating efficiency and accuracy of the AGV, reducing cargo damage or operation delays caused by positioning errors.

[0030] 4. The invention does not require expensive calibration equipment and complicated calibration processes, thereby reducing calibration costs. At the same time, due to the simplification of the calibration process, the time and human resources required in the calibration process are also reduced. By improving the operating accuracy and efficiency of AGV, the system helps to improve the operating efficiency and accuracy of the entire logistics system, injecting new impetus into the sustainable development of the logistics industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the practical framework flow. DETAILED DESCRIPTION

[0032] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, and use. It should be understood that in all of these figures, the same or similar reference numerals indicate the same or similar parts and features. The various drawings only schematically represent the concepts and principles of the embodiments of the present disclosure, and do not necessarily show the specific dimensions and proportions of the various embodiments of the present disclosure. Specific parts in specific drawings may be exaggerated to illustrate the relevant details or structures of the embodiments of the present disclosure.

[0033] See also Figure 1 , an embodiment provided by the present invention:

[0034] A TOF camera-based calibration system comprises the following steps:

[0035] 1): Obtain the image and point cloud data of the calibration plate, where a checkerboard pattern is set on the calibration plate to provide a calibration reference for the TOF camera;

[0036] 2): Construct a pixel coordinate matrix based on the pixel coordinates of the checkerboard corner points in the calibration plate image;

[0037] 3): Using the sampling consistency algorithm, the pixel coordinate matrix and the corresponding noisy 3D coordinates are combined to calculate the homography matrix, and the plane equation in the camera coordinate system is fitted through the point cloud data of the calibration plate. Based on the plane equation constraint and the homography matrix, the denoised 3D coordinates are restored from the pixel coordinates of the checkerboard corner points;

[0038] 4): Using the SVD (singular value decomposition) algorithm, the position and posture of the TOF camera relative to the calibration board coordinate system are calculated based on the denoised 3D coordinates of the checkerboard corner points in the calibration board coordinate system and the camera coordinate system;

[0039] 5): Set the calibration plate coordinate system to be consistent with the vehicle coordinate system, so that the rotation matrix of the calibration plate coordinate system relative to the vehicle coordinate system is the unit matrix, and obtain the translation vector of the calibration plate coordinate system relative to the vehicle coordinate system by measurement, so as to combine the position matrix of the calibration plate coordinate system relative to the vehicle coordinate system;

[0040] 6): According to the posture information obtained in step 4) and step 3), start calculating and outputting the posture of the TOF camera relative to the vehicle body coordinate system, and then end.

[0041] Furthermore, the calibration plate is a plane calibration plate with a checkerboard pattern, which is used to provide an accurate calibration reference for the TOF camera. This design can provide a clear and accurate calibration reference for the TOF camera. The checkerboard pattern is easy to identify and match, which helps to accurately extract the pixel coordinates of the corner points, thereby improving the accuracy of the calibration.

[0042] Furthermore, in step 3), the 3D coordinates of the corner points of the chessboard are accurately restored by using the calibration plate plane equation as a constraint condition and combining the homography matrix to remove the influence of noise. This method can remove the influence of noise and improve the accuracy of 3D coordinates, thereby ensuring the accuracy of subsequent pose calculations.

[0043] Further, in step 5), the actual distance between the calibration plate and the vehicle body is measured by a measuring tool such as a tape measure to determine the accurate value of the translation vector. This method is simple and easy to implement, does not require complex measuring equipment or technology, reduces calibration costs, and ensures the accuracy of the translation vector.

[0044] Furthermore, the TOF camera is a time-domain flight camera that can simultaneously acquire images and point cloud data. This camera can provide two-dimensional images and three-dimensional point cloud data at the same time, providing a rich source of information for the calibration system and helping to more accurately calculate the camera's position and posture.

[0045] Further, a TOF camera, a calibration plate, an image processing unit, a point cloud processing unit and a pose calculation unit, wherein: the TOF camera is used to capture an image of the calibration plate and obtain point cloud data;

[0046] The calibration plate is a flat plate with a checkerboard pattern, used to provide a calibration reference for the TOF camera;

[0047] The image processing unit is used to detect the pixel coordinates of the corner points of the chessboard in the image;

[0048] The point cloud processing unit is used to process the point cloud data of the calibration plate, fit the plane equation, and restore the 3D coordinates after denoising;

[0049] The pose calculation unit is used to calculate and output the pose of the TOF camera relative to the vehicle body coordinate system based on the restored 3D coordinates and the relative position relationship between the calibration plate and the vehicle body.

[0050] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A calibration system based on a TOF camera, characterized in that: The following steps are involved: 1): Obtain the image and point cloud data of the calibration plate, where a checkerboard pattern is set on the calibration plate to provide a calibration reference for the TOF camera; 2): Construct a pixel coordinate matrix based on the pixel coordinates of the checkerboard corner points in the calibration plate image; 3): Using the sampling consistency algorithm, the pixel coordinate matrix and the corresponding noisy 3D coordinates are combined to calculate the homography matrix, and the plane equation in the camera coordinate system is fitted through the point cloud data of the calibration plate. Based on the plane equation constraint and the homography matrix, the denoised 3D coordinates are restored from the pixel coordinates of the checkerboard corner points; 4): Using the SVD (singular value decomposition) algorithm, the position and posture of the TOF camera relative to the calibration board coordinate system are calculated based on the denoised 3D coordinates of the checkerboard corner points in the calibration board coordinate system and the camera coordinate system; 5): Set the calibration plate coordinate system to be consistent with the vehicle coordinate system, so that the rotation matrix of the calibration plate coordinate system relative to the vehicle coordinate system is the unit matrix, and obtain the translation vector of the calibration plate coordinate system relative to the vehicle coordinate system by measurement, so as to combine the position matrix of the calibration plate coordinate system relative to the vehicle coordinate system; 6): According to the posture information obtained in step 4) and step 3), start calculating and outputting the posture of the TOF camera relative to the vehicle body coordinate system, and then end.

2. A TOF camera-based calibration system according to claim 1, characterized in that: The calibration plate is a plane calibration plate with a checkerboard pattern, which is used to provide an accurate calibration reference for the TOF camera.

3. A TOF camera-based calibration system according to claim 1, characterized in that: In step 3), the plane equation of the calibration plate is used as a constraint condition, combined with the homography matrix, to accurately restore the 3D coordinates of the corner points of the chessboard to remove the influence of noise.

4. A TOF camera-based calibration system according to claim 1, characterized in that: In step 5), the actual distance between the calibration plate and the vehicle body is measured by a measuring tool such as a tape measure to determine the accurate value of the translation vector.

5. The TOF camera-based calibration system according to claim 1, characterized in that: The TOF camera is a time-domain flight camera that can simultaneously acquire images and point cloud data.

6. The TOF camera-based calibration system according to claim 1, characterized in that: The TOF camera, calibration plate, image processing unit, point cloud processing unit and posture calculation unit, wherein: the TOF camera is used to capture the image of the calibration plate and obtain point cloud data; The calibration plate is a flat plate with a checkerboard pattern, used to provide a calibration reference for the TOF camera; The image processing unit is used to detect the pixel coordinates of the corner points of the chessboard in the image; The point cloud processing unit is used to process the point cloud data of the calibration plate, fit the plane equation, and restore the 3D coordinates after denoising; The pose calculation unit is used to calculate and output the pose of the TOF camera relative to the vehicle body coordinate system based on the restored 3D coordinates and the relative position relationship between the calibration plate and the vehicle body.

Images