A 3D camera rapid calibration device and calibration method

Abstract

This invention discloses a rapid calibration device and method for 3D cameras. This invention utilizes a laser ranging module to achieve rapid and high-precision calibration of 3D structured light cameras, which can be widely applied in fields such as robot navigation, virtual reality, augmented reality, and industrial 3D inspection, and is particularly suitable for industrial environments with high requirements for calibration efficiency and accuracy. By introducing a high-precision laser ranging module and designing an integrated calibration scheme, this invention adds "hard constraints" from the physical world to the vision system, significantly reducing calibration complexity and improving calibration efficiency and accuracy in three aspects: reducing operational complexity and decreasing dependence on the pose of the tilted checkerboard calibration board; the laser ranging module directly provides the absolute depth values ​​of the feature points on the tilted checkerboard calibration board, and by combining these depth values ​​with the image coordinates of the feature points, it is not necessary to move the tilted checkerboard calibration board significantly in the forward and backward Z-axis direction as in traditional methods.

Description

Technical Field

[0001] This invention relates to the field of computer vision and camera calibration technology, specifically to a rapid calibration device and method for 3D cameras. Background Technology

[0002] 3D camera calibration is a fundamental and core task in the field of computer vision. Its core objective is to determine the camera's internal parameters (such as focal length, principal point position, distortion coefficients, etc.) and external parameters (the camera's position and orientation in space). The accuracy of calibration directly determines the reliability of subsequent image processing and 3D analysis.

[0003] In existing 3D camera calibration technologies, the mainstream method for intrinsic parameter calibration is Zhang's calibration method, which involves acquiring projected images of a planar checkerboard grid from different viewpoints and using corner point information to solve for the camera's intrinsic parameters. Extrinsic parameter calibration is mostly based on the Perspective-n-Point (PnP) algorithm, which uses the known correspondence between 3D spatial points and their projection points in the image to solve for the camera's pose transformation matrix. However, traditional calibration methods suffer from the following significant technical drawbacks:

[0004] First, the algorithm is highly dependent on the diversity of the pose of the tilted checkerboard calibration board. To solve for the depth information (Z-axis) in the camera's extrinsic parameters, the tilted checkerboard calibration board must be moved significantly and at multiple angles within the camera's field of view, especially requiring multiple transformations along the forward and backward Z-axis directions. If the tilted checkerboard calibration board moves only in one plane, the algorithm will fail to converge due to a lack of parallax information. This method requires highly manual operation, is cumbersome, and has low calibration efficiency.

[0005] Secondly, the parameter solution process is highly coupled. The camera's intrinsic and extrinsic parameters typically require coupled optimization or step-by-step iterative solutions, a complex nonlinear optimization process often figuratively called "black box" optimization. This process is computationally intensive and prone to getting trapped in local optima, posing a serious challenge to the accuracy and stability of the algorithm implementation.

[0006] Finally, a large number of image samples are needed to ensure calibration accuracy. To compensate for noise and measurement errors, traditional methods typically require collecting dozens or even hundreds of images of the tilted checkerboard calibration board at different angles and then statistically averaging them. This not only makes calibration time-consuming but also generates a huge amount of data processing, making it difficult to meet the needs of rapid deployment in industrial settings.

[0007] While existing technologies have attempted to combine laser ranging with camera calibration—for example, using laser ranging modules to assist in extrinsic parameter calibration of the camera, or using lasers to assist in solving the light plane equation—none of these solutions propose a technical approach that rigidly connects a single-point laser ranging module to a structured light generator and combines it with a tilted checkerboard calibration plate to achieve integrated and rapid calibration of the relationship between camera intrinsic and extrinsic parameters and laser plane transformation. More importantly, existing technologies fail to utilize the absolute depth value provided by laser ranging to achieve decoupled solution of calibration parameters, nor do they fundamentally simplify the stringent requirements for the pose of the tilted checkerboard calibration plate. Therefore, they cannot completely solve the core problems of traditional 3D camera calibration, such as cumbersome operation, complex algorithms, and long processing times. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention aims to provide a rapid 3D camera calibration device and method, specifically addressing the technical problems of cumbersome operation, complex algorithms, and time-consuming calibration in current 3D camera calibration methods. More specifically, this invention utilizes a laser ranging module to achieve rapid and high-precision calibration of a 3D structured light camera. This method can be widely applied in fields such as robot navigation, virtual reality, augmented reality, and industrial 3D inspection, and is particularly suitable for industrial environments where high calibration efficiency and accuracy are required.

[0009] This invention introduces a high-precision laser ranging module as a "physical world hard constraint" for the vision system, aiming to simplify the operation requirements of the tilted checkerboard calibration board, decouple the parameter solving process, and improve the accuracy of single calibration, thereby realizing fast and high-precision calibration of 3D structured light cameras.

[0010] To achieve the above objectives, the technical method employed in this invention is as follows:

[0011] This invention discloses a rapid calibration device for a 3D structured light camera, including a rigid base, an industrial camera fixed on the rigid base, a single-point laser ranging module and a structured light generator, which are rigidly connected by a rigid bracket on the rigid base. It also includes an inclined checkerboard calibration plate for providing corner features with known physical coordinates.

[0012] As a further improvement, the tilt angle of the tilted checkerboard calibration plate described in this invention can be adjusted between 30 and 50 degrees, and the corner point extraction accuracy reaches the sub-pixel level.

[0013] This invention discloses a calibration method for a rapid calibration device for a 3D structured light camera, comprising the following steps:

[0014] 1) The relative positions and attitudes of the industrial camera, single-point laser ranging module, and structured light generator are kept fixed to form an integrated calibration device;

[0015] 2) The structured light camera was calibrated using Zhang's calibration method to obtain and fix the internal parameters of the industrial camera;

[0016] 3) Place the tilted checkerboard calibration plate within the field of view of the industrial camera, extract the image coordinates and actual physical coordinates of the corner points of the tilted checkerboard calibration plate, and calculate the rotation matrix and translation matrix of the tilted checkerboard calibration plate relative to the coordinate system of the structured light camera.

[0017] 4) Keep the calibration device stable, use the laser ranging module to measure the absolute distance D of a feature point on the inclined checkerboard calibration board, and combine the actual physical coordinates of the feature point with the rotation matrix and translation matrix obtained in step 3) to calculate the spatial three-dimensional coordinates of all planar points within the range of the inclined checkerboard calibration board in the coordinate system of the structured light camera through coordinate transformation.

[0018] 5) Keep the calibration device stable, control the structured light generator to project the structured light stripes onto the surface of the inclined checkerboard calibration plate, extract the image coordinates of the light stripes, combine them with the three-dimensional coordinates of the plane points of the inclined checkerboard calibration plate obtained in step 4), calculate the laser plane equation through coordinate fitting, and then establish the transformation relationship between the laser plane and the industrial camera coordinate system to complete the calibration of the 3D structured light camera.

[0019] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0020] This invention introduces a high-precision laser ranging module and designs an integrated calibration scheme, adding "hard constraints" from the physical world to the vision system. This significantly reduces the complexity of calibration and improves calibration efficiency and accuracy in three aspects:

[0021] Reduced operational complexity and less reliance on the pose of the tilted checkerboard calibration plate: The laser ranging module directly provides the absolute depth values ​​(Z-axis information) of the feature points on the tilted checkerboard calibration plate. Combining this depth value with the image coordinates of the feature points, their three-dimensional spatial position can be directly locked, eliminating the need for significant forward and backward Z-axis movement of the calibration plate as required by traditional methods. Operators only need to position the tilted checkerboard calibration plate anywhere within the camera's field of view to quickly calculate parameters, significantly simplifying the manual operation process and improving calibration efficiency.

[0022] Reducing algorithm complexity and decoupling parameter solving: The high-precision depth values ​​provided by laser ranging can serve as "truth value" constraints for parameter optimization. This means that the optimization of parameters such as camera distortion coefficients no longer relies entirely on the geometric calculation of image corner points, transforming the traditionally coupled intrinsic and extrinsic parameter optimization process into a constrained linear problem or a step-by-step solution problem. This not only avoids the risk of the algorithm getting trapped in local optima but also significantly reduces the computational load and the difficulty of algorithm implementation.

[0023] Improving single-shot calibration accuracy and reducing image acquisition and repetition counts: Laser ranging accuracy can typically reach millimeter or even sub-millimeter levels. By fusing high-precision depth information from laser ranging, the system can perform real-time correction on visual measurement results, significantly improving the confidence level of a single measurement. Therefore, there is no need to collect a large number of image samples for statistical averaging; high-precision calibration can be achieved with only a small number of images, effectively reducing calibration time and data processing volume.

[0024] Achieving integrated calibration: By rigidly connecting the camera and the laser ranging module, and combining it with an inclined checkerboard calibration plate, this invention can directly solve the laser plane equation and establish its transformation relationship with the camera coordinate system while completing the calibration of the camera's intrinsic and extrinsic parameters, without the need for additional calibration steps, perfectly meeting the actual needs of rapid and integrated calibration in industrial settings.

[0025] To verify the superiority of this invention, it was compared with a traditional 3D camera calibration method, and the results are as follows:

[0026] Operational complexity: Traditional methods require acquiring more than 50 images of the inclined checkerboard calibration plate at different angles, including the forward and backward movement along the Z-axis, which is cumbersome. In contrast, this method only requires acquiring 10 intrinsic parameter calibration images and 1 extrinsic parameter calibration image, and does not require moving the inclined checkerboard calibration plate along the Z-axis, making the operation extremely simple.

[0027] Algorithm complexity: Traditional methods require coupled optimization of intrinsic and extrinsic parameters, which is complex and takes about 30 minutes to calculate; this method adopts a step-by-step solution strategy, and the total calculation time is only about 1 minute, which greatly improves the computational efficiency.

[0028] Calibration accuracy: Under the same test environment, the three-dimensional measurement error after calibration by the traditional method is ±0.5mm; while after this method integrates laser ranging information, the three-dimensional measurement error is reduced to ±0.15mm, and the accuracy is improved by more than 70%. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the hardware structure of the 3D camera rapid calibration system of the present invention;

[0030] Figure 2 This is a diagram illustrating the process of solving the pose matrix of the inclined chessboard calibration board.

[0031] Figure 3 This is a diagram showing the solution of the three-dimensional coordinates of the plane points on the inclined chessboard calibration plate;

[0032] Figure 4 This is a diagram establishing the relationship between the laser plane and the camera transformation.

[0033] Figure 5 This is a flowchart of the calibration method for a rapid calibration device for a 3D structured light camera.

[0034] In the diagram, 1 is the laser ranging module, 2 is the industrial camera, 3 is the structured light generator, 4 is the tilted checkerboard calibration plate, and 5 is the rigid base. Detailed Implementation

[0035] The technical solution of the present invention will be described in detail below with reference to specific embodiments. It should be understood that the following embodiments are only used to explain the present invention and are not intended to limit the scope of protection of the present invention.

[0036] The core of this invention lies in rigidly fixing the structured light camera and the single-point laser ranging module 1, and using the absolute depth value provided by the laser ranging module 1, combined with the tilted checkerboard calibration plate 4, to complete the step-by-step calibration of the intrinsic and extrinsic parameters of the structured light camera and the laser plane transformation relationship. The specific technical solution is as follows:

[0037] Figure 1 This is a schematic diagram of the hardware structure of the 3D structured light camera rapid calibration system of the present invention. The hardware includes a rigid base 5, an industrial camera 2 fixed on the rigid base 5, a single-point laser ranging module 1, and a structured light generator 3, all rigidly connected by a rigid bracket on the rigid base 5. It also includes an inclined checkerboard calibration plate 4 for providing corner features with known physical coordinates. The hardware specifications in this embodiment are as follows:

[0038] Industrial Camera 2: Select an industrial camera 2 with a resolution of 1920×1080 and a frame rate of 30 frames per second or higher.

[0039] Structured light generator 3: Uses DLP4500 optical engine, programmable patterns, high contrast, high frame rate of 30 frames / second or higher.

[0040] Single-point laser ranging module 1: SICK DS35 is selected. The high-precision laser ranging module 1 has a measurement accuracy of ±0.1mm. The ranging range covers 0.1 meters to 2 meters to meet the calibration requirements of different scenarios.

[0041] The rigid base 5 is made of high-strength aluminum alloy, and a rigid bracket is vertically installed on the base. The bracket is made of the same material as the base. The industrial camera 2, laser ranging module 1, and result light generator are securely fixed to ensure that the relative position deviation between them during the calibration process is less than 0.05mm and the relative attitude deviation is less than 0.1°, thereby ensuring the consistency of the measurement reference.

[0042] Tilted checkerboard calibration plate 4: A high-precision tilted checkerboard calibration plate 4 with a physical size of 20mm×20mm is selected. The tilted checkerboard calibration plate 4 is designed with an adjustable tilt angle, and its corner point sub-pixel extraction accuracy is better than 0.01 pixels to ensure the accuracy of the calibration reference.

[0043] The industrial camera 2 and the structured light generator 3 together form a structured light camera.

[0044] Figure 5 This is a flowchart of the calibration method for a rapid calibration device for a 3D structured light camera. The entire calibration process consists of the following steps.

[0045] 1) Intrinsic parameter calibration of industrial camera 2: The classic Zhang's calibration method is used to calibrate the intrinsic parameters of industrial camera 2, obtaining intrinsic parameters such as focal length, principal point position, and distortion coefficients, and fixing these parameters. Specifically, following the standard procedure of Zhang's calibration method, the tilted checkerboard calibration board 4 is placed at different positions and angles within the field of view of industrial camera 2, and 10 images of the tilted checkerboard calibration board 4 are acquired. Corner points in the images are accurately extracted using a corner extraction algorithm. Through homography matrix solving and nonlinear optimization, the intrinsic parameter matrix K of industrial camera 2 (including focal lengths fx and fy, principal point coordinates u0 and v0, and distortion coefficients k1, k2, p1, and p2) is obtained. The reprojection error of the optimized intrinsic parameters should be controlled within 0.5 pixels.

[0046] 2) Solving the 4-position pose matrix of the tilted checkerboard calibration board: Figure 2 This diagram illustrates the process of solving the pose matrix of the tilted checkerboard calibration plate 4. A tilted checkerboard calibration plate 4 is placed within the field of view of industrial camera 2. The corner coordinates of the tilted checkerboard calibration plate 4 are extracted, and combined with its known physical coordinates, the rotation and translation matrices of the tilted checkerboard calibration plate 4 relative to the structured light camera coordinate system are calculated using the PnP algorithm. Solving the pose matrix of the tilted checkerboard calibration plate 4: The tilted checkerboard calibration plate 4 is placed at the center of the field of view of industrial camera 2, and its tilt angle is adjusted to 30°. One image of the tilted checkerboard calibration plate 4 is acquired. The corner coordinates (X', Y') of the image are extracted, and combined with its known physical coordinates (Xw, Yw), the rotation matrix S and translation matrix T of the tilted checkerboard calibration plate 4 relative to the coordinate system of industrial camera 2 are solved.

[0047] 3) Solving for the three-dimensional coordinates of the four plane points on the inclined checkerboard calibration plate: Figure 3This is a diagram showing the 3D coordinates of the planar points on the inclined checkerboard calibration plate 4. Keeping the integrated calibration system stable, the absolute distance D from any feature point (such as the center point or corner point) on the inclined checkerboard calibration plate 4 to the laser ranging module 1 is measured using the single-point laser ranging module 1. Combining the physical coordinates of this feature point on the inclined checkerboard calibration plate 4 with the rotation and translation matrices obtained in step 2, the spatial 3D coordinates of all planar points within the range of the inclined checkerboard calibration plate 4 are calculated through coordinate transformation. Keeping the positions of the rigid support and the inclined checkerboard calibration plate 4 unchanged, the laser ranging module 1 emits a laser beam, precisely aiming at the center corner point of the inclined checkerboard calibration plate 4, and the absolute distance D from this point to the laser ranging module 1 is measured, for example, D = 1.5m. Image recognition is used to calculate the center of the light spot in the image coordinates (Xo', Yo'). Using the rotation matrix S and translation matrix T from step 2, the physical coordinates (Xo, Yo) of the light spot center on the tilted checkerboard calibration plate 4 are obtained. Combined with D, the 3D coordinates (Xo, Yo, D) of the light spot are obtained. The 3D coordinates of all planar points within the area of ​​the tilted checkerboard calibration plate 4 in the coordinate system of the industrial camera 2 can then be further derived.

[0048] 4) Establishment of the laser plane equation and transformation relationship: Figure 4 This is a diagram establishing the transformation relationship between the laser plane and industrial camera 2. Maintaining system stability, the structured light generator 3 projects structured light stripes onto the surface of the tilted checkerboard calibration plate 4, and extracts the image coordinates of the light stripes. Combining the three-dimensional spatial coordinates of the points on the tilted checkerboard calibration plate 4 obtained in step 3), the laser plane equation is calculated using a least-squares iso-coordinate fitting algorithm, thus establishing a precise transformation relationship between the laser plane and the coordinate system of industrial camera 2, completing the entire calibration process of the 3D structured light camera. Keeping the system position unchanged, the structured light generator 3 projects structured light stripes onto the surface of the tilted checkerboard calibration plate 4, acquires the light stripe image, and extracts the sub-pixel-level image coordinates (u, v) of the light stripe. Combining the three-dimensional coordinates (Xc, Yc, Zc) of the points on the tilted checkerboard calibration plate 4 obtained in step 3, plane fitting is performed using the least-squares method to obtain the laser plane equation ax + by + cz + d = 0, where a, b, c, and d are the coefficients of the fitted laser plane equation. Based on this laser plane equation, a precise transformation relationship between the laser plane and the industrial camera's coordinate system can be established, thus completing the entire calibration process for the 3D structured light camera. The entire calibration process takes less than 5 minutes.

[0049] The calibration method of this invention is not only applicable to structured light 3D cameras, but also adaptable to the calibration of various 3D structured light cameras, such as binocular 3D structured light cameras and monocular depth structured light cameras, by appropriately adjusting the relevant steps of structured light stripe projection. These equivalent substitutions or modifications should all fall within the protection scope of this invention. Furthermore, the rigid connection method between the laser ranging module 1 and the structured light camera can be flexibly adjusted according to the actual application scenario, such as using integrated fixing or detachable fixing, as long as the relative positions of the two remain unchanged during the calibration process, it falls within the protection scope of this invention.

[0050] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the core technical features of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A rapid calibration device for a 3D structured light camera, characterized in that, It includes a rigid base, an industrial camera fixed on the rigid base, a single-point laser ranging module and a structured light generator, which are rigidly connected by a rigid bracket on the rigid base. It also includes an inclined checkerboard calibration plate for providing corner features with known physical coordinates.

2. The rapid calibration device for a 3D structured light camera according to claim 1, characterized in that, The tilt angle of the tilted checkerboard calibration plate can be adjusted between 30 and 50 degrees, and the corner point extraction accuracy reaches the sub-pixel level.

3. A calibration method for a rapid calibration device for a 3D structured light camera, characterized in that, Includes the following steps: 1) The relative positions and attitudes of the industrial camera, single-point laser ranging module, and structured light generator are kept fixed to form an integrated calibration device; 2) The structured light camera was calibrated using Zhang's calibration method to obtain and fix the internal parameters of the industrial camera; 3) Place the tilted checkerboard calibration plate within the field of view of the industrial camera, extract the image coordinates and actual physical coordinates of the corner points of the tilted checkerboard calibration plate, and calculate the rotation matrix and translation matrix of the tilted checkerboard calibration plate relative to the coordinate system of the structured light camera. 4) Keep the calibration device stable, use the laser ranging module to measure the absolute distance D of a feature point on the inclined checkerboard calibration board, and combine the actual physical coordinates of the feature point with the rotation matrix and translation matrix obtained in step 3) to calculate the three-dimensional spatial coordinates of all planar points within the range of the inclined checkerboard calibration board in the structured light camera coordinate system through coordinate transformation. 5) Keep the calibration device stable, control the structured light generator to project the structured light stripes onto the surface of the inclined checkerboard calibration plate, extract the image coordinates of the light stripes, combine them with the three-dimensional coordinates of the plane points of the inclined checkerboard calibration plate obtained in step 4), calculate the laser plane equation through coordinate fitting, and then establish the transformation relationship between the laser plane and the industrial camera coordinate system to complete the calibration of the 3D structured light camera.

4. The calibration method of the 3D structured light camera rapid calibration device according to claim 3, characterized in that, The internal parameters of the structured light camera in step 2) include focal length, principal point position, and distortion coefficient.

5. The calibration method for the rapid calibration device for a 3D structured light camera according to claim 3 or 4, characterized in that, In step 4), the feature points are the corner points or center points of the inclined checkerboard calibration board, and the ranging accuracy of the laser ranging module is at the millimeter level or higher.

Images