Structured light camera external parameter calibration method based on laser tracker

Abstract

The invention discloses a structured light camera external parameter calibration method based on a laser tracker, and relates to the technical field of robot calibration. The first robot grabs the structured light camera, the second robot grabs the calibration plate, multiple sets of double-robot cooperation poses are planned, the two robots are controlled to move cooperatively in the working space, and multiple different calibration poses are obtained; synchronously acquiring a joint angle of the first robot, calibration plate point cloud data acquired by the structured light camera, and a three-dimensional coordinate of a sphere center of a spherical reflector on the calibration plate measured by the laser tracker under each calibration pose; for each calibration pose, the pose of a structured light camera coordinate system relative to a first robot tail end flange coordinate system can be solved through a coordinate system conversion relation, and a plurality of structured light camera poses obtained by solving multiple groups of poses are averagely fused to obtain a final pose transformation matrix; the method is used for solving the problems that a traditional calibration result is low in absolute precision and poor in credibility.

Description

Technical Field

[0001] This invention relates to the field of dual-arm collaborative calibration technology, specifically to a method for calibrating the extrinsic parameters of a structured light camera based on a laser tracker. Background Technology

[0002] In industrial robot 3D inspection systems, structured light cameras, as efficient and non-contact vision sensors, are widely used in tasks such as quality inspection, workpiece positioning, 3D reconstruction, and robot guidance. Their core function is to project coded light spots and acquire images of object surfaces, thereby reconstructing the 3D point cloud information of the object's surface. However, structured light cameras can only provide 3D data in their own coordinate system. To tightly integrate this data with the robot's motion system and achieve precise vision-based operation or measurement, it is essential to accurately obtain the pose transformation relationship between the structured light camera coordinate system and the robot's end effector flange coordinate system, i.e., the camera's extrinsic parameters. The calibration accuracy of this extrinsic parameter directly determines the accuracy of the visual measurement results in the robot's global coordinate system, which in turn affects the final performance of the entire detection or operating system.

[0003] Currently, the most widely used calibration method in industry is the hand-eye calibration method. Traditional hand-eye calibration typically relies on a high-precision calibration board (such as a checkerboard or dot array board) fixed in the workspace. A robot then moves the camera to multiple different poses, acquiring images of the calibration board and extracting feature points. Using the camera imaging model and the robot's kinematic model, coordinate system constraint equations are established under multiple poses, and finally, the transformation matrix of the camera relative to the robot's base or end effector is solved. Although this method is theoretically mature, it has several significant limitations in practical applications: First, its calibration process heavily relies on the absolute positioning accuracy of the robot. The calibration process directly uses the flange pose data provided by the robot controller. The absolute accuracy error of these data (mainly caused by factors such as robot geometric parameter calibration residuals, link deformation, and gear backlash) will be directly introduced into the external parameter calculation, becoming the main bottleneck restricting the calibration accuracy. Secondly, traditional methods typically require the calibration plate to remain stationary, which restricts the robot's reachable observation poses and results in a limited coverage of the spatial point set used for calibration. This may not be sufficient to constrain the various degrees of freedom of the extrinsic parameters, especially in the depth direction.

[0004] To improve the spatial distribution of calibration points, a dual-arm collaborative calibration method has been proposed. This method utilizes the coordinated movement of two robots, with a camera mounted on the end effector of one robot and a calibration board mounted on the end effector of the other. By planning multiple sets of relative poses, the calibration board can present a variety of postures and positions within the camera's field of view, thereby obtaining more comprehensive constraint information. Theoretically, this method can improve the robustness of the calibration results and the coverage of the workspace.

[0005] However, a key drawback in its practice lies in the lack of independent, high-precision truth verification. The entire calibration process still relies entirely on the robot's own joint encoder readings (calculating pose through positive kinematics) and the camera's internal measurements, thus forming a "closed" calibration loop. If there are uncompensated errors in the robot's kinematic parameters or deviations in the camera's intrinsic parameters, these errors will couple and propagate during the calibration process and eventually be absorbed into the extrinsic parameters to be determined, making it difficult to detect and quantify the systematic errors in the calibration results. Therefore, although dual-arm collaboration provides more data points, the absolute accuracy and reliability of its calibration results cannot be effectively guaranteed.

[0006] Therefore, we propose a method to improve the accuracy and reliability of calibration results. Summary of the Invention

[0007] The purpose of this invention is to provide a method for calibrating the extrinsic parameters of a structured light camera based on a laser tracker, which solves the problems of low absolute accuracy and poor reliability of traditional calibration results.

[0008] This invention is achieved through the following technical solution: A method for calibrating the extrinsic parameters of a structured light camera based on a laser tracker includes: Using the known positions of control points in the world coordinate system, the laser tracker is positioned in the world coordinate system; The first robot grasps the structured light camera, the second robot grasps the calibration board, and multiple sets of dual-robot cooperative poses are planned. The two robots are controlled to move cooperatively in the workspace to obtain multiple different calibration poses. Under each calibration pose, the joint angles of the first robot, the point cloud data of the calibration board acquired by the structured light camera, and the three-dimensional coordinates of the center of the spherical reflector on the calibration board measured by the laser tracker are simultaneously acquired. Among them, there are three spherical reflectors on the calibration board, and the three spherical reflectors are distributed at right angles. For each calibration pose, based on the measurement data of the laser tracker and the pose of the laser tracker in the world coordinate system, the pose transformation matrix of the calibration plate coordinate system relative to the world coordinate system is calculated. Based on the joint angles and kinematic model of the first robot, the pose transformation matrix of the robot end flange coordinate system relative to the world coordinate system is calculated. Based on the point cloud data acquired by the structured light camera, the pose transformation matrix of the calibration plate coordinate system relative to the camera coordinate system is calculated. Based on the three pose transformation matrices calculated under each calibration pose, the extrinsic pose transformation matrix of the structured light camera coordinate system relative to the first robot end flange coordinate system is solved through the coordinate system chain relationship. The extrinsic pose transformation matrices obtained from multiple poses are averaged and fused to obtain the final pose transformation matrix.

[0009] Furthermore, the process of using the known positions of control points in the world coordinate system to locate the laser tracker in the world coordinate system is as follows: Fix the laser tracker in the work area, and the laser tracker covers the workspace of the two robots respectively; Multiple control points are set up within the working area, and the coordinates of each control point in the coordinate system of the laser tracker are measured using a laser tracker. By registering control points, the transformation relationship between the laser tracker coordinate system and the global world coordinate system is established.

[0010] Furthermore, for each calibration pose, based on the measurement data of the laser tracker and the pose of the laser tracker in the world coordinate system, the pose transformation matrix of the calibration board coordinate system relative to the world coordinate system is calculated. The calculation process is as follows: Transform the three-dimensional coordinates of the center of the spherical mirror on the calibration plate, measured by the laser tracker, to the world coordinate system; Define the origin as the spherical mirror located at a right angle point; The x-axis direction is defined as the direction from the origin to the other spherical mirror. The normal vector perpendicular to the plane containing the three target points is taken as the z-axis, and the direction of the z-axis follows the right-hand rule; The vector obtained by cross product of the z-axis vector and the x-axis vector is the y-axis direction. Based on the origin, x-axis direction, z-axis direction, and y-axis direction, construct the pose transformation matrix of the calibration board coordinate system relative to the world coordinate system.

[0011] Furthermore, based on the joint angles and kinematic model of the first robot, the pose transformation matrix of the robot's end flange coordinate system relative to the world coordinate system is calculated using the following mathematical formula:

[0012] In the formula, Let be the pose transformation matrix of the first robot relative to the base coordinate system and the world coordinate system. Joint angle vector The pose of the flange coordinate system relative to the robot base coordinate system, obtained through forward kinematics calculations.

[0013] Furthermore, based on the point cloud data acquired by the structured light camera, the pose transformation matrix of the calibration board coordinate system relative to the camera coordinate system is calculated. The calculation process is as follows: Point cloud clusters of three spherical mirrors are extracted from the point cloud data of the calibration board acquired by the structured light camera. The RANSAC algorithm was used to fit the sphere to obtain the coordinates of the centers of the three spherical mirrors; Define the origin as the coordinate of the center of the spherical mirror located at a right angle point; The x-axis is defined as the direction pointing from the origin to the center of the other spherical mirror. The normal vector perpendicular to the plane containing the coordinates of the centers of the three target points is taken as the z-axis, and the direction of the z-axis follows the right-hand rule; The vector obtained by cross product of the z-axis vector and the x-axis vector is the y-axis direction. Based on the origin, x-axis direction, z-axis direction, and y-axis direction, construct the pose transformation matrix of the calibration board coordinate system relative to the camera coordinate system.

[0014] Furthermore, the three pose transformation matrices calculated based on each calibration pose are used to solve for the extrinsic pose transformation matrix of the structured light camera coordinate system relative to the first robot end flange coordinate system through coordinate system chain relationships. The calculation process is as follows: Construct the coordinate system chain relationship and calculate the camera extrinsic parameter estimation formula; For each pose, multiple extrinsic pose transformation matrices are obtained according to the camera extrinsic estimation formula.

[0015] Furthermore, the mathematical formula for constructing the coordinate system chain relationship is as follows:

[0016] In the formula, Let be the pose transformation matrix of the calibration plate coordinate system relative to the world coordinate system. Let be the pose transformation matrix of the first robot's end flange coordinate system relative to the world coordinate system; This is the pose transformation matrix of the calibration board coordinate system relative to the camera coordinate system; Calculate camera extrinsic parameter estimation The formula is: .

[0017] Furthermore, the process of averaging and fusing the extrinsic pose transformation matrices obtained from multiple poses to obtain the final pose transformation matrix is ​​as follows: Take the arithmetic mean of the translation vectors in multiple extrinsic pose transformation matrices; The rotation matrix in multiple extrinsic pose transformation matrices is averaged using a spherical linear interpolation method. The averaged translation vector is combined with the averaged rotation matrix to obtain the final pose transformation matrix.

[0018] Furthermore, the spherical linear interpolation method is used to average the rotation matrix in multiple extrinsic pose transformation matrices. The specific process is as follows: Convert multiple rotation matrices into unit quaternion form; Sort the quaternions in descending order, and then calculate the average quaternion using the sequential recursive averaging method. Transform the average quaternion back into a rotation matrix.

[0019] Furthermore, the average quaternion is calculated using the sequential recursive averaging method, and the formula is as follows:

[0020]

[0021]

[0022] In the formula, Let be the unit quaternion of the multiple rotation matrices in descending order. It represents the angular distance between two unit quaternions.

[0023] The technical solution of the present invention has at least the following advantages and beneficial effects: This invention discloses a method for extrinsic parameter calibration of a structured light camera based on a laser tracker. By establishing a global world coordinate system through a laser tracker, a high-precision absolute position reference independent of the robot body and the camera is provided for the entire calibration process. This allows the pose of the calibration board at every moment to be directly and accurately measured by external equipment, breaking the dilemma of the inability to quantify and trace errors in traditional "closed-loop" calibration, and significantly improving the absolute accuracy and reliability of the extrinsic parameter calibration results.

[0024] By adopting a dual-arm collaborative approach, multiple calibration poses covering the robot's workspace and various perspectives can be planned. The rich pose data effectively suppresses random errors caused by local singularities and measurement noise, thereby improving the stability and robustness of the calibration algorithm.

[0025] It is worth noting that by planning multiple sets of dual-arm collaborative observation poses, observations can be performed from multiple angles and times, thereby effectively smoothing out random errors introduced by robot motion errors, flange coordinate system calibration errors, and camera measurement noise, and thus significantly improving the robustness and accuracy of the calibration results.

[0026] In addition, the entire process, from laser tracker self-positioning, multi-posture coordinated motion, and synchronous data acquisition to final external parameter calculation and optimization, can be automatically completed by the host computer control system, which greatly reduces human intervention, avoids the uncertainty introduced by manual operation, and ensures the consistency and repeatability of the calibration process.

[0027] Furthermore, a calibration plate coordinate system is constructed using three spherical mirrors arranged at right angles, which provides clear geometric constraints and strong noise resistance. Moreover, the calibration plate coordinate system is constructed using the same logic in both the world coordinate system and the camera coordinate system, based on the coordinates of the spherical mirrors measured by the laser tracker and the center coordinates of the sphere fitted by the point cloud of the structured light camera, ensuring the consistency of the two coordinate system definitions. Attached Figure Description Figure 1 This is a schematic diagram of the process for calibrating the extrinsic parameters of a structured light camera based on a laser tracker according to the present invention; Figure 2 This is a schematic diagram of the structure of a dual-arm calibration method according to the present invention; Figure 3 This is a schematic diagram of the structure of a calibration plate according to the present invention.

[0028] Reference numerals in the attached figures: 1. Laser tracker; 2. First robot; 3. Second robot; 4. Structured light camera; 5. Calibration plate; 6. Spherical reflector. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0030] Example 1 like Figures 1-3 The method for calibrating the extrinsic parameters of a structured light camera based on a laser tracker, as shown, includes: Using the known positions of control points in the world coordinate system, the laser tracker 1 is positioned in the world coordinate system. Since the world coordinate system is an independent, unified, and high-precision spatial reference frame, the specific process of positioning the laser tracker 1 is as follows: The laser tracker 1 is a device capable of measuring the three-dimensional coordinates of spatial targets with high precision. Fixing it and covering the dual-robot workspace means that it becomes a static, high-precision observation station that can continuously monitor any target point it "sees" within the work area; Multiple control points are set up within the working area, and the coordinates of each control point in the coordinate system of the laser tracker are measured using a laser tracker 1. The control points are uniform markers fixed in the work area. Their positions are stable in the physical world and have been accurately calibrated, meaning their positions in the custom world coordinate system are known. The laser tracker 1 measures each of these control points to obtain their coordinates in the laser tracker 1's own coordinate system. At this point, a connection is established between the "stable physical world position" and the "coordinates in the tracker's coordinate system" of the control points. By registering control points, the transformation relationship from the laser tracker 1 coordinate system to the global world coordinate system is established; The purpose of control point registration is to align the "local temporary coordinate system" of laser tracker 1 with the "global world coordinate system" we want. Specifically, through mathematical algorithms (such as the least squares method), an optimal transformation matrix is ​​found. This matrix can accurately transform any coordinate measured by laser tracker 1 into the global world coordinate system. Establish a global world coordinate system based on this transformation relationship; The global world coordinate system established by the laser tracker 1 provides a more accurate external reference independent of the robot system and camera system. The actual pose of the calibration board 5 at each moment is directly measured by this external reference, rather than indirectly obtained through calculation by the robot model with errors or through camera measurement. This fundamentally cuts off the transmission path from robot kinematic errors and camera intrinsic errors to extrinsic parameters. In addition, since all key data (the pose of the calibration board 5) are ultimately unified under a high-precision global coordinate system, the intermediate results and final extrinsic parameters of the entire calibration process can be verified using the same "ruler".

[0031] The first robot 2 grasps the structured light camera 4, and the second robot 3 grasps the calibration board 5. Multiple sets of dual-robot cooperative poses are planned, and the two robots are controlled to move cooperatively in the workspace to obtain multiple different calibration poses. The plan includes approximately 90 collaborative bi-arm poses, providing a statistically sufficient sample size while achieving a balance between actual operation time and efficiency. Each pose satisfies the following conditions: 1. The optimal working distance between the optical center of the structured light camera 4 and the calibration plate 5 ensures the highest accuracy and best integrity of the acquired point cloud data, providing high-quality input for calculating the pose transformation matrix of the structured light camera 4 coordinate system relative to the first robot 2 flange coordinate system, thus ensuring imaging quality. 2. The calibration board 5 has good pose diversity in the camera's field of view, and the change of the calibration board relative to the camera is very small in each cooperative pose. In this way, the three spheres on the calibration board are basically in the center of the camera, resulting in good point cloud effect and high accuracy in extracting the sphere center. The cooperation of the two arms is mainly to balance the error between the robot and the camera. 3. The laser tracker 1 can clearly measure the center of the three spherical mirrors 6 on the calibration plate 5, which ensures that high-precision data can be reliably acquired in every pose.

[0032] Furthermore, the large amount of random pose data effectively suppresses calibration and motion errors in the robot's flange coordinate system. It's important to note that these errors primarily originate from two sources: first, the error in the pose calibration relationship between the robot's base coordinate system and the world coordinate system; and second, the deviation between the actual flange pose and the commanded pose caused by factors such as the robot's absolute positioning accuracy, link flexibility, and joint backlash during motion execution. Through 90 independent observations in different spatial configurations, these systematic calibration errors and robot motion errors can be statistically effectively smoothed, thereby reducing their systematic impact on the final external parameter calculation. This system effectively suppresses systemic and random errors in camera and calibration plate observations. These errors primarily stem from the inherent measurement errors of the structured light camera (such as optical distortion and phase calculation errors) and random noise extracted from feature points on the calibration plate during each observation. By conducting numerous repeated observations at different angles, the impact of random noise on the results can be effectively reduced, and the system's ability to constrain the parameters of the camera's inherent error model can be enhanced.

[0033] Under each calibration pose, the joint angles of the first robot 2, the point cloud data of the calibration plate 5 acquired by the structured light camera 4, and the three-dimensional coordinates of the center of the spherical reflector 6 on the calibration plate 5 measured by the laser tracker 1 are simultaneously collected; there are three spherical reflectors 6 on the calibration plate 5, and the three spherical reflectors 6 are distributed at right angles. In addition, the spherical reflector 6 includes a spherical shell with an observation window on the top. The reflector is fixedly positioned at the center of the sphere inside the shell. This unique design of the spherical reflector 6 creates a physical entity that is "friendly" to both the laser tracker 1 and the structured light camera 4 and can output the same geometric features. This fundamentally ensures that the pose transformation matrix of the calibration plate 5 coordinate system relative to the world coordinate system and the pose transformation matrix of the calibration plate 5 coordinate system relative to the camera coordinate system describe the same calibration plate 5 coordinate system, eliminating systematic alignment errors introduced by different sensor measurement objects.

[0034] For each calibration pose, based on the measurement data of laser tracker 1 and the pose of laser tracker 1 in the world coordinate system, the pose transformation matrix of the calibration plate 5 coordinate system relative to the world coordinate system is calculated. This step yields the "true" pose of calibration plate 5 in absolute space at the current moment. It is the most accurate and reliable reference value in the entire data chain because it comes directly from high-precision external measurement equipment. Based on the joint angles and kinematic model of the first robot 2, the pose transformation matrix of the robot end flange coordinate system relative to the world coordinate system is calculated. This step yields the "theoretical" pose of the robot end in absolute space at the current moment. Its accuracy is limited by the accuracy of the robot kinematic model (DH parameter error, link deflection, etc.) and the calibration accuracy of the pose transformation matrix of the first robot 2 base coordinate system relative to the world coordinate system. It is a key bridge connecting the robot body and the external world. Based on the point cloud data acquired by the structured light camera 4, the pose transformation matrix of the calibration plate 5 coordinate system relative to the camera coordinate system is calculated. This step yields the observed pose of the calibration plate 5 in the "eyes" of the camera at the current moment. Its accuracy is limited by the camera's intrinsic parameter calibration accuracy, optical distortion, point cloud noise, and the accuracy of the spherical fitting algorithm. Subsequently, the three are linked together through coordinate system chain equations, making the source of error clearly identifiable, while fusion provides the possibility of using high-precision true values ​​to calibrate and constrain the other two links.

[0035] Based on the three pose transformation matrices calculated under each calibration pose, the extrinsic pose transformation matrix of the structured light camera 4 coordinate system relative to the end flange coordinate system of the first robot 2 is solved through the coordinate system chain relationship. This constitutes a complete calculation chain from "data" to "parameters". The calculation process is as follows: The mathematical formula for constructing a coordinate system chain is as follows:

[0036] In the formula, This is the pose transformation matrix of the calibration board 5 coordinate system relative to the world coordinate system. Let be the pose transformation matrix of the end flange coordinate system of the first robot 2 relative to the world coordinate system; This is the pose transformation matrix of the calibration board 5 coordinate system relative to the camera coordinate system; This coordinate system chain describes a closed-loop coordinate transformation chain: Starting from the world coordinate system, there are two paths to reach the calibration plate 5 coordinate system; one of these paths (on the left side of the equals sign) is obtained directly through measurement by laser tracker 1. This represents the absolute truth value.

[0037] Path 2 (right side of the equals sign): First from the calibration plate to the camera, then from the camera to the robot flange, and finally from the robot flange to the world coordinate system to obtain the pose transformation matrix of the calibration plate in the world coordinate system. Furthermore, this formula "white-boxes" the process of solving external parameters. Each intermediate variable has a clear physical meaning and acquisition method, making the entire calibration process logically very clear, and the calculation results are completely traceable and reproducible. Formula for estimating camera extrinsic parameters: ; For each pose, multiple extrinsic pose transformation matrices are obtained according to the camera extrinsic estimation formula; The extrinsic pose transformation matrices obtained from multiple poses are averaged and fused to obtain the final pose transformation matrix, which is the final pose transformation matrix of the coordinate system of the structured light camera 4 relative to the coordinate system of the end flange of the first robot 2. The strategy of "independent preliminary estimation + averaging method fusion" is adopted to combine the absolute accuracy advantage brought by the laser tracker 1 with the statistical redundancy advantage brought by the dual-arm multi-pose, so as to produce the final high-precision and high-stability extrinsic pose transformation matrix with the highest efficiency and reliability.

[0038] Example 2 As one embodiment, based on the measurement data of the laser tracker 1 and the global world coordinate system, the pose transformation matrix of the calibration plate 5 coordinate system relative to the world coordinate system is calculated. The calculation process is as follows: Transform the three-dimensional coordinates of the center of the spherical mirror 6 on the calibration plate 5, measured by the laser tracker 1, to the world coordinate system; Define the spherical mirror 6 located at the right angle point as the origin; The x-axis direction is defined as the direction from the origin to the other spherical mirror 6. The normal vector perpendicular to the plane containing the three target points is taken as the z-axis, and the direction of the z-axis follows the right-hand rule; The vector obtained by cross product of the z-axis vector and the x-axis vector is the y-axis direction. Based on the origin, x-axis direction, z-axis direction, and y-axis direction, construct the pose transformation matrix of the calibration board 5 coordinate system relative to the world coordinate system; By directly measuring the absolute coordinates of the centers of the three spherical mirrors 6 using the laser tracker 1, and then obtaining the attitude through pure geometric calculations, the accuracy of this method is directly inherited from the measurement accuracy of the laser tracker 1. Moreover, the process is deterministic and the calculation is simple, with almost no steps that introduce additional errors, making it one of the key guarantees for achieving overall high precision. Furthermore, this method transforms a physical calibration plate 5 with three specifically arranged target points installed into a rigid body that can be mathematically described precisely, uniquely, and stably. This rigid body becomes a universal "spatial beacon" connecting the laser tracker 1 (world), the robot (flange), and the camera, three different dimensions. It is this carefully designed "beacon" that makes high-precision data fusion across sensors possible.

[0039] Furthermore, based on the joint angles and kinematic model of the first robot 2, the pose transformation matrix of the robot's end flange coordinate system relative to the world coordinate system is calculated using the following mathematical formula:

[0040] In the formula, Let be the pose transformation matrix of the first robot 2 relative to the base coordinate system and the world coordinate system. Joint angle vector Flange coordinate system pose obtained through forward kinematics calculation; This is used to reliably correlate high-precision external measurements with the robot's imperfect internal model, where the pose transformation matrix of the robot's base coordinate system relative to the base coordinate system is used. It is a constant transformation matrix obtained through offline precision calibration; it accurately describes the robot's position in the world coordinate system. Because its calibration process can be carried out independently of the production cycle and repeatedly measured and optimized using high-precision methods, the accuracy and reliability of this matrix are far higher than any pose data reported by the robot online in real time. It is the real-time output of the robot's forward kinematics model, which is based on the current joint angles. The theoretical flange pose is calculated; this calculation is based on the robot's design parameters (DH parameters), but is affected by all internal errors such as robot positioning error, gear backlash, and link flexibility.

[0041] Furthermore, based on the point cloud data acquired by the structured light camera 4, the pose transformation matrix of the calibration board 5 coordinate system relative to the camera coordinate system is calculated. The calculation process is as follows: Point cloud clusters of three spherical mirrors 6 are extracted from the point cloud data of the calibration board 5 acquired by the structured light camera 4. The RANSAC algorithm was used to fit the sphere to obtain the coordinates of the centers of the three spherical mirrors 6. Define the coordinates of the center of the spherical mirror 6 located at the right angle point as the origin; The x-axis direction is defined as the direction pointing from the origin to the center of the other spherical mirror 6. The normal vector perpendicular to the plane containing the coordinates of the centers of the three target points is taken as the z-axis, and the direction of the z-axis follows the right-hand rule; The vector obtained by cross product of the z-axis vector and the x-axis vector is the y-axis direction. Based on the origin, x-axis direction, z-axis direction, and y-axis direction, construct the pose transformation matrix of the calibration board 5 coordinate system relative to the camera coordinate system; The process of determining the pose transformation matrix of the calibration plate 5 coordinate system relative to the camera coordinate system uses the exact same logic as the calculation of the pose transformation matrix of the calibration plate 5 coordinate system relative to the world coordinate system. This consistency in definition ensures that, whether viewed from the world coordinate system or the camera coordinate system, the "calibration plate 5 coordinate system" we are talking about is the same rigid body coordinate system defined by the exact same rules; thus, both sides of the coordinate system chain relationship have the exact same physical meaning, and the equation can be strictly established.

[0042] Example 3 As one embodiment, based on multiple preliminary extrinsic parameter estimates, the optimal extrinsic parameters are obtained using the averaging method, namely the final pose transformation matrix of the structured light camera 4 coordinate system relative to the end flange coordinate system of the first robot 2. The specific process is as follows: Take the arithmetic mean of the translation vectors among multiple extrinsic parameters, where the extrinsic parameter matrix... for:

[0043] In the formula, For rotation matrix, It is a translation vector; The formula for calculating the arithmetic mean of the translation vectors is:

[0044] In the formula, The average of the translation vectors. For extrinsic parameters, For the first Translation vectors in an external parameter matrix.

[0045] The rotation matrix among multiple extrinsic parameters is averaged using spherical linear interpolation. The specific process is as follows: Multiple rotation matrices Convert to unit quaternion form ; Sort the quaternions in descending order, and then calculate the average quaternion using a sequential recursive averaging method. The calculation formula is:

[0046]

[0047]

[0048] In the formula, The first of the average quaternions Step-by-step recursion steps, Let be the unit quaternion of the multiple rotation matrices in descending order. The angular distance between two unit quaternions. The value range is usually in Between radians, it intuitively measures the "difference" or "distance" between two rotations; in Used to represent spherical linear interpolation, i.e. When the parameter When changing from 0 to 1, Output a rotation quaternion Smoothly and uniformly transforming to rotational quaternions The quaternion sequence. This transformation is along the quaternion sphere connected... and The shortest path (great circle arc) is used, while It's the central angle of this arc. This can be understood as the proportion of the distance already traveled on this arc; Transform the average quaternion back into a rotation matrix; The optimal extrinsic parameters are obtained by combining the averaged translation vector with the averaged rotation matrix.

[0049] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for calibrating the extrinsic parameters of a structured light camera based on a laser tracker, characterized in that, include: Using the known positions of the control points in the world coordinate system, the laser tracker (1) is positioned in the world coordinate system; The first robot (2) grabs the structured light camera (4), the second robot (3) grabs the calibration plate (5), plans multiple sets of dual-robot cooperative poses, controls the two robots to move cooperatively in the workspace, and obtains multiple different calibration poses; Under each calibration pose, the joint angles of the first robot (2), the point cloud data of the calibration plate (5) acquired by the structured light camera (4), and the three-dimensional coordinates of the center of the spherical reflector (6) on the calibration plate (5) measured by the laser tracker (1) are collected synchronously; there are three spherical reflectors (6) on the calibration plate (5), and the three spherical reflectors (6) are distributed at right angles; For each calibration pose, based on the measurement data of the laser tracker (1) and the pose of the laser tracker (1) in the world coordinate system, calculate the pose transformation matrix of the calibration plate (5) coordinate system relative to the world coordinate system. Based on the joint angles and kinematic model of the first robot (2), calculate the pose transformation matrix of the robot end flange coordinate system relative to the world coordinate system. Based on the point cloud data acquired by the structured light camera (4), calculate the pose transformation matrix of the calibration plate (5) coordinate system relative to the camera coordinate system. Based on the three pose transformation matrices obtained under each calibration pose, the extrinsic pose transformation matrix of the structured light camera (4) coordinate system relative to the end flange coordinate system of the first robot (2) is solved through the coordinate system chain relationship. The extrinsic pose transformation matrices obtained from multiple poses are averaged and fused to obtain the final pose transformation matrix.

2. The method for calibrating the extrinsic parameters of a structured light camera based on a laser tracker according to claim 1, characterized in that: The process of positioning the laser tracker (1) in the world coordinate system using the known positions of control points in the world coordinate system is as follows: The laser tracker (1) is fixed in the working area, and the laser tracker (1) covers the working space of the two robots respectively; Multiple control points are set up in the working area, and the coordinates of each control point in the coordinate system of the tracker are measured using a laser tracker (1). By registering control points, the transformation relationship between the laser tracker (1) coordinate system and the global world coordinate system is established.

3. The method for calibrating the extrinsic parameters of a structured light camera based on a laser tracker according to claim 1, characterized in that: For each calibration pose, based on the measurement data of the laser tracker (1) and the pose of the laser tracker (1) in the world coordinate system, the pose transformation matrix of the calibration plate (5) coordinate system relative to the world coordinate system is calculated. The calculation process is as follows: The three-dimensional coordinates of the center of the spherical reflector (6) on the calibration plate (5) measured by the laser tracker (1) are transformed to the world coordinate system; Define the spherical mirror (6) located at the right angle point as the origin; The direction from the origin to the other spherical mirror (6) is defined as the x-axis direction; The normal vector perpendicular to the plane containing the three target points is taken as the z-axis, and the direction of the z-axis follows the right-hand rule; The vector obtained by cross product of the z-axis vector and the x-axis vector is the y-axis direction. Based on the origin, x-axis direction, z-axis direction and y-axis direction, construct the pose transformation matrix of the calibration plate (5) coordinate system relative to the world coordinate system.

4. The extrinsic parameter calibration method for a structured light camera based on a laser tracker according to claim 1, characterized in that: Based on the joint angles and kinematic model of the first robot (2), the pose transformation matrix of the robot's end flange coordinate system relative to the world coordinate system is calculated using the following mathematical formula: In the formula, Let be the pose transformation matrix of the known first robot (2) relative to the world coordinate system. Joint angle vector The pose of the flange coordinate system relative to the robot base coordinate system, obtained through forward kinematics calculations.

5. The extrinsic parameter calibration method for a structured light camera based on a laser tracker according to claim 1, characterized in that: The pose transformation matrix of the calibration board (5) coordinate system relative to the camera coordinate system is calculated based on the point cloud data acquired by the structured light camera (4). The calculation process is as follows: Extract the point cloud clusters of the three spherical mirrors (6) from the point cloud data of the calibration board (5) obtained from the structured light camera (4); The RANSAC algorithm was used to fit the sphere to obtain the coordinates of the center of the three spherical mirrors (6); Define the coordinates of the center of the spherical mirror (6) located at the right angle as the origin; The x-axis direction is defined as the direction from the origin to the center coordinates of the other spherical mirror (6). The normal vector perpendicular to the plane containing the coordinates of the centers of the three target points is taken as the z-axis, and the direction of the z-axis follows the right-hand rule; The vector obtained by cross product of the z-axis vector and the x-axis vector is the y-axis direction. Based on the origin, x-axis direction, z-axis direction and y-axis direction, construct the pose transformation matrix of the calibration plate (5) coordinate system relative to the camera coordinate system.

6. The extrinsic parameter calibration method for a structured light camera based on a laser tracker according to claim 1, characterized in that: The three pose transformation matrices calculated based on each calibration pose are used to solve the extrinsic pose transformation matrix of the structured light camera (4) coordinate system relative to the end flange coordinate system of the first robot (2) through the coordinate system chain relationship. The calculation process is as follows: Construct the coordinate system chain relationship and calculate the camera extrinsic parameter estimation formula; For each pose, multiple extrinsic pose transformation matrices are obtained according to the camera extrinsic estimation formula.

7. The extrinsic parameter calibration method for a structured light camera based on a laser tracker according to claim 6, characterized in that: The mathematical formula for constructing the coordinate system chain relationship is as follows: In the formula, The pose transformation matrix of the calibration plate (5) coordinate system relative to the world coordinate system, The pose transformation matrix of the end flange coordinate system of the first robot (2) relative to the world coordinate system; The pose transformation matrix of the calibration board (5) coordinate system relative to the camera coordinate system; Calculate camera extrinsic parameter estimation The formula is: 。 8. The method for calibrating the extrinsic parameters of a structured light camera based on a laser tracker according to claim 7, characterized in that: The process of averaging and fusing the extrinsic pose transformation matrices obtained from multiple poses to obtain the final pose transformation matrix is ​​as follows: Take the arithmetic mean of the translation vectors in multiple extrinsic pose transformation matrices; The rotation matrix in multiple extrinsic pose transformation matrices is averaged using a spherical linear interpolation method. The averaged translation vector is combined with the averaged rotation matrix to obtain the final pose transformation matrix.

9. The extrinsic parameter calibration method for a structured light camera based on a laser tracker according to claim 8, characterized in that: The method of averaging the rotation matrix in multiple extrinsic pose transformation matrices using spherical linear interpolation is as follows: Convert multiple rotation matrices into unit quaternion form; Sort the quaternions in descending order, and then calculate the average quaternion using the sequential recursive averaging method. Transform the average quaternion back into a rotation matrix.

10. The extrinsic parameter calibration method for a structured light camera based on a laser tracker according to claim 9, characterized in that: The average quaternion is calculated using the sequential recursive averaging method. The formula is as follows: In the formula, The unit quaternion for the multiple rotation matrices in descending order. It represents the angular distance between two unit quaternions.

Images