A method for calibrating an extrinsic parameter of a structured light camera based on a laser tracker

Abstract

A kind of structural light camera external parameter calibration method based on laser tracker, it is related to robot calibration technical field, laser tracker is positioned to world coordinate system;First robot grabs structural light camera, second robot grabs calibration board, plans multiple sets of double robot cooperation poses, controls two robots to cooperate movement in workspace, obtains multiple different calibration poses;At each calibration pose, the joint angle of first robot is synchronously collected, the point cloud data of calibration board obtained by structural light camera and the three-dimensional coordinates of spherical mirror ball center measured by laser tracker on calibration board are acquired;For each calibration pose, the pose of structural light camera coordinate system relative to the flange coordinate system of first robot end can be solved by coordinate system conversion relationship, the multiple structural light camera poses obtained by solving multiple sets of poses are averaged and fused, and the final pose transformation matrix is obtained;It is used to solve the problems of low absolute accuracy and poor reliability of traditional calibration results.

Description

TECHNICAL FIELD

[0001] The application relates to a double-arm cooperative calibration technical field, in particular to a structured light camera external parameter calibration method based on a laser tracker. BACKGROUND

[0002] In an industrial robot three-dimensional detection system, a structured light camera is widely used in quality detection, workpiece positioning, three-dimensional reconstruction and robot guidance as a kind of efficient and non-contact visual sensor. Its core function is to project coded light spots and collect object surface images, and then restore three-dimensional point cloud information of the object surface. However, the structured light camera itself can only provide three-dimensional data in its own coordinate system. If these data are to be closely combined with the robot motion system to realize accurate operation or measurement based on vision, it is necessary to accurately obtain the pose transformation relationship between the structured light camera coordinate system and the robot end flange coordinate system, that is, the camera external parameter The calibration accuracy of the external parameter directly determines the accuracy of the visual measurement result in the global coordinate system of the robot, and then affects the final performance of the whole detection or operation system.

[0003] At present, the widely used calibration method in the industry is mainly a hand-eye calibration method. The traditional hand-eye calibration method usually relies on a high-precision calibration board (such as a chessboard or a circular dot array board) fixed in the workspace. Then the camera is moved to a plurality of different poses by the robot, the image of the calibration board is collected and the feature points are extracted, the camera imaging model and the robot kinematics model are used to establish the coordinate system constraint equation under a plurality of poses, and finally the transformation matrix of the camera relative to the robot base or the end is solved. Although this method is relatively mature in theory, it has several significant limitations in practical application:

[0004] Firstly, the calibration process is severely dependent on the absolute positioning accuracy of the robot. The flange pose data provided by the robot controller is directly used in the calibration process. The absolute accuracy error of these data (mainly caused by robot geometric parameter calibration residual error, link deformation, gear backlash and other factors) will be directly introduced into the external parameter calculation, becoming the main bottleneck restricting the calibration accuracy;

[0005] Secondly, the traditional method usually requires the calibration board to be fixed, which restricts the reachable observation poses of the robot, resulting in limited coverage of the spatial point set used for calibration, which may not fully constrain the degrees of freedom of the external parameter, especially in the depth direction.

[0006] In order to improve the spatial distribution of the calibration points, a dual-arm cooperative calibration method is proposed, which utilizes the cooperative motion of two robots, one of which is equipped with a camera at the end and the other is equipped with a calibration board. By planning multiple sets of relative poses, the calibration board presents a variety of poses and positions in the camera's field of view, thereby obtaining more comprehensive constraint information; this method can theoretically improve the robustness of the calibration results and the coverage of the workspace.

[0007] However, a key defect in practice is the lack of independent high-precision true value verification, that is, the entire calibration process still completely relies on the joint encoder readings of the robot itself (calculating the pose through forward kinematics) and the internal measurements of the camera, thus forming a "closed" calibration loop; once there are un-compensated errors in the robot's kinematic parameters or deviations in the camera's internal parameters, these errors will be coupled and transmitted to each other in the calibration process, and eventually absorbed into the external parameters to be solved, resulting in systematic errors in the calibration results that are difficult to detect and quantify; therefore, although the dual-arm cooperation provides more data points, the absolute accuracy and reliability of the calibration results cannot be effectively guaranteed.

[0008] Therefore, we propose a method that can improve the accuracy and reliability of the calibration results. SUMMARY

[0009] The purpose of the present application is to provide a laser tracker-based structured light camera external parameter calibration method, which solves the problems of low absolute accuracy and poor reliability of traditional calibration results.

[0010] The present application is implemented by the following technical solutions:

[0011] A laser tracker-based structured light camera external parameter calibration method, comprising:

[0012] Use the known control points in the world coordinate system to position the laser tracker in the world coordinate system;

[0013] The first robot grasps the structured light camera, the second robot grasps the calibration board, multiple sets of dual-robot cooperative poses are planned, and the two robots are controlled to move cooperatively in the workspace to obtain multiple different calibration poses;

[0014] At each calibration pose, the joint angles of the first robot, the point cloud data of the calibration board obtained by the structured light camera, and the three-dimensional coordinates of the spherical mirror ball center of the calibration board measured by the laser tracker are synchronously collected; wherein the spherical mirror on the calibration board is three, and the three spherical mirrors are distributed at right angles;

[0015] For each calibration pose, based on the measurement data of the laser tracker and the pose of the laser tracker in the world coordinate system, a pose transformation matrix of the calibration board coordinate system relative to the world coordinate system is calculated, based on the joint angles of the first robot and its kinematic model, a pose transformation matrix of the robot end flange coordinate system relative to the world coordinate system is calculated, based on the point cloud data obtained by the structured light camera, a pose transformation matrix of the calibration board coordinate system relative to the camera coordinate system is calculated;

[0016] Based on the three kinds of pose transformation matrices calculated and obtained under each calibration pose, the external parameter pose transformation matrix of the structured light camera coordinate system relative to the first robot end flange coordinate system is solved through the relationship of the coordinate system chain.

[0017] The external parameter pose transformation matrices obtained under multiple poses are averaged and fused to obtain a final pose transformation matrix.

[0018] Further, the laser tracker is positioned in the world coordinate system by using the known positions of the control points in the world coordinate system, and the specific process is as follows:

[0019] The laser tracker is fixed in the working area, and the laser tracker covers the working space of the two robots;

[0020] A plurality of control points are arranged in the working area, and the coordinates of each control point in the tracker coordinate system are measured by the laser tracker;

[0021] The transformation relationship from the laser tracker coordinate system to the global world coordinate system is established through control point registration.

[0022] Further, for each calibration pose, based on the measurement data of the laser tracker and the pose of the laser tracker in the world coordinate system, a pose transformation matrix of the calibration board coordinate system relative to the world coordinate system is calculated, the calculation process is as follows:

[0023] The three-dimensional coordinates of the spherical mirror ball center on the calibration board measured by the laser tracker are converted to the world coordinate system;

[0024] The spherical mirror located at the right angle point is defined as the origin;

[0025] The direction from the origin to the other spherical mirror is defined as the x-axis direction;

[0026] The normal vector perpendicular to the plane where the three target points are located is taken as the z-axis, and the direction of the z-axis follows the right-hand rule;

[0027] The vector obtained by the cross product of the z-axis vector and the x-axis vector is the y-axis direction;

[0028] According to the origin, the x-axis direction, the z-axis direction and the y-axis direction, the pose transformation matrix of the calibration board coordinate system relative to the world coordinate system is constructed.

[0029] Further, based on the joint angle of the first robot and its kinematic model, the pose transformation matrix of the robot end flange coordinate system relative to the world coordinate system is calculated, and the mathematical formula is:

[0030]

[0031] In the formula, is the known pose transformation matrix of the base coordinate system of the first robot relative to the world coordinate system, is the joint angle vector The pose of the flange coordinate system relative to the robot base coordinate system is calculated by forward kinematics.

[0032] Further, based on the point cloud data obtained by the structured light camera, the pose transformation matrix of the calibration board coordinate system relative to the camera coordinate system is calculated, and the calculation process is:

[0033] From the point cloud data of the calibration board obtained by the structured light camera, the point cloud clusters of the three spherical mirrors are extracted;

[0034] The RANSAC algorithm is used to fit the spherical surface to obtain the spherical center coordinates of the three spherical mirrors;

[0035] The spherical center coordinates of the spherical mirror located at the right angle point are defined as the origin;

[0036] The direction from the origin to the spherical center coordinates of the other spherical mirror is defined as the x-axis direction;

[0037] The normal vector perpendicular to the plane where the three target point spherical center coordinates are located is taken as the z-axis, and the direction of the z-axis follows the right-hand rule;

[0038] The vector obtained by the cross product of the z-axis vector and the x-axis vector is the y-axis direction;

[0039] According to the origin, the x-axis direction, the z-axis direction and the y-axis direction, the pose transformation matrix of the calibration board coordinate system relative to the camera coordinate system is constructed.

[0040] Further, based on the three kinds of pose transformation matrices calculated at each calibration pose, the extrinsic pose transformation matrix of the structured light camera coordinate system relative to the first robot end flange coordinate system is solved through the coordinate system chain relationship, and the calculation process is:

[0041] The coordinate system chain relationship is constructed, and the camera extrinsic parameter estimation formula is calculated;

[0042] For each pose, a plurality of extrinsic pose transformation matrices are obtained according to the camera extrinsic parameter estimation formula.

[0043] Further, the coordinate system chain relationship is constructed, and the mathematical formula is:

[0044]

[0045] In the formula, is a pose transformation matrix of the calibration board coordinate system relative to the world coordinate system, is a pose transformation matrix of the first robot end flange coordinate system relative to the world coordinate system; is a pose transformation matrix of the calibration board coordinate system relative to the camera coordinate system;

[0046] Calculate the camera extrinsic parameter estimationThe formula is:

[0047] .

[0048] Further, the obtained extrinsic parameter pose transformation matrices in multiple poses are averaged and fused to obtain a final pose transformation matrix, and the specific process is as follows:

[0049] The translation vectors in the multiple extrinsic parameter pose transformation matrices are arithmetically averaged;

[0050] The rotation matrices in the multiple extrinsic parameter pose transformation matrices are averaged by using a spherical linear interpolation method;

[0051] The averaged translation vectors and the averaged rotation matrices are combined, that is, the final pose transformation matrix.

[0052] Further, the rotation matrices in the multiple extrinsic parameter pose transformation matrices are averaged by using a spherical linear interpolation method, and the specific process is as follows:

[0053] The multiple rotation matrices are converted into unit quaternion form;

[0054] The quaternions are sorted in descending order, and then an order recursive average method is used to calculate an average quaternion;

[0055] The average quaternion is converted back into a rotation matrix.

[0056] Further, the order recursive average method is used to calculate the average quaternion, and the calculation formula is as follows:

[0057]

[0058]

[0059]

[0060] In the formula, is a unit quaternion of the multiple rotation matrices sorted in descending order, is an angular distance between two unit quaternions.

[0061] The technical scheme of the present application has at least the following advantages and beneficial effects:

[0062] The application discloses a kind of based on laser tracker's structure light camera external parameter calibration method, global world coordinate system is established by laser tracker, and high-precision absolute position reference independent of robot body and camera is provided for entire calibration process;This makes the pose of calibration board at each time can be directly and accurately measured by external equipment, breaks the plight that error cannot be quantified and traced in traditional "closed loop" calibration, significantly improves the absolute accuracy and reliability of external parameter calibration result.

[0063] Adopting double-arm cooperation mode, multiple calibration poses with robot workspace, perspective diversity can be planned, and abundant attitude data effectively suppresses random error caused by local singularity, measurement noise and the like, to improve the stability and robustness of calibration algorithm.

[0064] It should be noted that by planning multiple sets of double-arm cooperative observation poses, multi-angle and multiple-time observation can be carried out, so as to effectively smooth the random error introduced by robot motion error, flange coordinate system calibration error and camera measurement noise, and further significantly improve the robustness and accuracy of the calibration result.

[0065] In addition, from laser tracker self-positioning, multi-pose cooperative motion, data synchronous acquisition to final external parameter calculation and optimization, the whole process can be automatically completed by the host computer control system, greatly reducing human intervention, avoiding the uncertainty introduced by manual operation, and ensuring the consistency and repeatability of the calibration process.

[0066] In addition, three spherical mirrors distributed at right angles are used to construct the calibration board coordinate system, which has clear geometric constraints and strong noise resistance;And the spherical mirror coordinates measured by the laser tracker and the sphere center coordinates fitted by the point cloud of the structured light camera are used to construct the calibration board coordinate system in the world coordinate system and the camera coordinate system with the same logic, to ensure the consistency of the two coordinate systems. BRIEF DESCRIPTION OF DRAWINGS

[0067] Fig. 1 A flowchart of a structure light camera external parameter calibration method based on laser tracker of the present application is shown in the figure.

[0068] Fig. 2 A structure diagram of double-arm calibration of the present application is shown in the figure.

[0069] Fig. 3 A structure diagram of a calibration board of the present application is shown in the figure.

[0070] Reference signs: 1, laser tracker;2, first robot;3, second robot;4, structured light camera;5, calibration board;6, spherical mirror. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0072] Example 1

[0073] like Figs. 1-3 The method for calibrating the extrinsic parameters of a structured light camera based on a laser tracker, as shown, includes:

[0074] Using the known positions of control points in the world coordinate system, the laser tracker 1 is positioned in the world coordinate system. Since the world coordinate system is an independent, unified, and high-precision spatial reference frame, the specific process of positioning the laser tracker 1 is as follows:

[0075] Among them, the laser tracker 1 is a device capable of measuring the three-dimensional coordinates of spatial target points with high precision. Fixing it and covering the dual-robot workspace means that it becomes a static, high-precision observation station that can continuously monitor any target point "seen" by it within the work area;

[0076] Multiple control points are set up within the working area, and the coordinates of each control point in the coordinate system of the laser tracker are measured using a laser tracker 1.

[0077] The control points are uniform markers fixed in the work area. Their positions are stable in the physical world and have been accurately calibrated, meaning their positions in the custom world coordinate system are known. The laser tracker 1 measures each of these control points to obtain their coordinates in the laser tracker 1's own coordinate system. At this point, a connection is established between the "stable physical world position" and the "coordinates in the tracker's coordinate system" of the control points.

[0078] By registering control points, the transformation relationship from the laser tracker 1 coordinate system to the global world coordinate system is established;

[0079] The purpose of control point registration is to align the "local temporary coordinate system" of laser tracker 1 with the "global world coordinate system" we want. Specifically, through mathematical algorithms (such as the least squares method), an optimal transformation matrix is ​​found. This matrix can accurately transform any coordinate measured by laser tracker 1 into the global world coordinate system.

[0080] Establish a global world coordinate system based on this transformation relationship;

[0081] The global world coordinate system established by the laser tracker 1 provides an external reference independent of the robot system and the camera system, which has higher precision. The real-time pose of the subsequent calibration board 5 at each moment is directly measured by this external reference, rather than indirectly obtained by calculating through the error robot model or camera measurement, which fundamentally cuts off the transmission path of robot kinematics error and camera internal parameter error to the external parameter. In addition, since all key data (calibration board 5 pose) are ultimately unified in a high-precision global coordinate system, the intermediate results and final external parameters of the entire calibration process can be checked with the same "ruler".

[0082] The first robot 2 grasps the structured light camera 4, and the second robot 3 grasps the calibration board 5. A plurality of double-robot cooperative poses are planned, and the two robots are controlled to move cooperatively in the working space to obtain a plurality of different calibration poses.

[0083] Among them, about 90 double-arm cooperative poses are planned, which provides sufficient sample size in statistics, and balances the actual operation time and benefit. Each pose meets the following conditions:

[0084] 1. The distance between the optical center of the structured light camera 4 and the calibration board 5 is the best working distance, which ensures that the point cloud data obtained has the highest precision and the best integrity, and provides high-quality input for calculating the pose transformation matrix of the structured light camera 4 coordinate system relative to the first robot 2 flange coordinate system, and ensures the imaging quality;

[0085] 2. The calibration board 5 has good pose diversity in the camera field of view, and the change of the calibration board relative to the camera is very small under each cooperative pose. In this way, the three spherical mirrors 6 on the calibration board are basically in the center of the camera, and the point cloud effect of the camera is good. Finally, the precision of extracting the ball center is high. Double-arm cooperation is mainly to balance the errors of the robot and the camera;

[0086] 3. The laser tracker 1 can clearly measure the ball centers of the three spherical mirrors 6 on the calibration board 5, which ensures that high-precision data can be reliably obtained at each pose.

[0087] And a large number of random pose data have the effect of suppressing the robot flange coordinate system calibration and motion error. It should be noted that the error here mainly comes from two aspects: one is the error of the pose calibration relationship between the robot base coordinate system and the world coordinate system; the second is that when the robot performs motion, due to its own absolute positioning accuracy, link flexibility and joint backlash, etc., there is a deviation between the actual flange pose and the command pose. Through 90 independent observations at different spatial configurations, these systematic calibration errors and robot motion errors can be effectively smoothed in statistics, thereby reducing their systematic influence on the final external parameter solution;

[0088] and the effect of random errors of the system and the camera observing the calibration board is inhibited, and the error is mainly from the inherent measurement error of the structured light camera (such as optical distortion, phase solving error) and the random noise of the feature point extraction of the calibration board in each observation. Through a large number of repeated observations at different angles, the influence of random noise on the result can be effectively reduced, and the constraint ability of the system on the inherent error model parameters of the camera can be enhanced.

[0089] In each calibration pose, the joint angle of the first robot 2, the point cloud data of the calibration board 5 obtained by the structured light camera 4, and the three-dimensional coordinates of the ball centers of the spherical mirrors 6 on the calibration board 5 measured by the laser tracker 1 are synchronously collected; wherein the spherical mirrors 6 on the calibration board 5 are three, and the three spherical mirrors 6 are distributed at right angles;

[0090] In addition, the spherical mirror 6 includes a spherical shell, an observation window is formed at the top of the spherical shell, and a mirror is fixedly arranged at the ball center in the spherical shell; the unique design of the spherical mirror 6 creates a physical entity that is "friendly" to both the laser tracker 1 and the structured light camera 4 and can output the same geometric feature; this fundamentally ensures that the two matrices described by the pose transformation matrix of the calibration board 5 coordinate system relative to the world coordinate system and the pose transformation matrix of the calibration board 5 coordinate system relative to the camera coordinate system are the same calibration board 5 coordinate system, eliminating systematic alignment errors introduced due to different measurement objects of the sensors.

[0091] For each calibration pose, based on the measurement data of the laser tracker 1 and the pose of the laser tracker 1 in the world coordinate system, the pose transformation matrix of the calibration board 5 coordinate system relative to the world coordinate system is calculated; this step obtains the "true" pose of the calibration board 5 in the absolute space at the current time, which is the most accurate and most reliable reference value in the entire data chain because it directly comes from the high-precision external measurement equipment;

[0092] Based on the joint angle of the first robot 2 and the kinematics model thereof, the pose transformation matrix of the robot end flange coordinate system relative to the world coordinate system is calculated; this step obtains the "theoretically calculated" pose of the robot end in the absolute space at the current time, and the accuracy is limited by the accuracy of the robot kinematics model (DH parameter error, link deflection, etc.) and the calibration accuracy of the pose transformation matrix of the base coordinate system of the first robot 2 relative to the world coordinate system, which is a key bridge connecting the robot body and the external world;

[0093] Based on the point cloud data obtained by the structured light camera 4, the pose transformation matrix of the calibration board 5 coordinate system relative to the camera coordinate system is calculated; this step obtains the observation pose of the calibration board 5 in the "camera eye" at the current time, and the accuracy is limited by the internal parameter calibration accuracy of the camera, optical distortion, point cloud noise, and the accuracy of the spherical fitting algorithm;

[0094] The three are linked by the coordinate system chain equation, so that the error source becomes clear, and fusion provides the possibility of using high-precision true values to calibrate and constrain the other two links.

[0095] Based on the three kinds of pose transformation matrices calculated under each calibration pose, the pose transformation matrix of the structure light camera 4 coordinate system relative to the first robot 2 end flange coordinate system is solved through the coordinate system chain relationship, which constitutes a complete calculation chain from "data" to "parameters", and the calculation process is:

[0096] The coordinate system chain relationship is constructed, and the mathematical formula is:

[0097]

[0098] In the formula, is the pose transformation matrix of the calibration plate 5 coordinate system relative to the world coordinate system, is the pose transformation matrix of the first robot 2 end flange coordinate system relative to the world coordinate system; is the pose transformation matrix of the calibration plate 5 coordinate system relative to the camera coordinate system;

[0099] The coordinate system chain relationship describes a closed coordinate transformation chain:

[0100] That is, starting from the world coordinate system, the calibration plate 5 coordinate system can be reached through two paths; Path one (left side of the equal sign): directly measured by the laser tracker 1 , which represents the absolute true value.

[0101] Path two (right side of the equal sign): first from the calibration plate to the camera, then from the camera to the robot flange, and finally from the robot flange to the world coordinate system to obtain the pose transformation matrix of the calibration plate in the world coordinate system;

[0102] And this formula "whitens" the external parameter solving process. Each intermediate variable has a clear physical meaning and acquisition method, making the entire calibration process extremely clear in logic, and the calculation result is completely traceable and reproducible;

[0103] The camera external parameter estimation formula is calculated:

[0104] ;

[0105] For each pose, multiple external pose transformation matrices are obtained according to the camera external parameter estimation formula;

[0106] The obtained pose transformation matrices of multiple poses are averaged and fused to obtain a final pose transformation matrix, that is, a final pose transformation matrix of the structured light camera 4 coordinate system relative to the first robot 2 end flange coordinate system; the strategy of "independent preliminary estimation + average value fusion" combines the absolute accuracy advantage brought by the laser tracker 1 and the statistical redundancy advantage brought by the multi-pose dual-arm to output the final high-precision and high-stability pose transformation matrix with the highest efficiency and reliability.

[0107] Embodiment 2

[0108] As an embodiment, based on the measurement data of the laser tracker 1 and the global world coordinate system, a pose transformation matrix of the calibration plate 5 coordinate system relative to the world coordinate system is calculated, and the calculation process is as follows:

[0109] The three-dimensional coordinates of the spherical mirror 6 ball center on the calibration plate 5 measured by the laser tracker 1 are converted to the world coordinate system;

[0110] The spherical mirror 6 located at the right angle point is defined as the origin;

[0111] The direction from the origin to the other spherical mirror 6 is defined as the x-axis direction;

[0112] The normal vector perpendicular to the plane where the three target points are located is taken as the z-axis, and the direction of the z-axis follows the right-hand rule;

[0113] The vector obtained by the cross product of the z-axis vector and the x-axis vector is the y-axis direction;

[0114] According to the origin, the x-axis direction, the z-axis direction and the y-axis direction, the pose transformation matrix of the calibration plate 5 coordinate system relative to the world coordinate system is constructed;

[0115] By directly measuring the absolute coordinates of the centers of the three spherical mirrors 6 by the laser tracker 1, and then calculating the pose by pure geometry, the accuracy of this method directly inherits the measurement accuracy of the laser tracker 1, and the process is determined, the calculation is simple, and there is almost no additional error link, which is one of the key guarantees for realizing overall high precision; and this method changes a physical calibration plate 5 installed with three specifically arranged target points into a rigid body which can be accurately, uniquely and stably described in mathematics, and this rigid body becomes a general "space beacon" connecting the laser tracker 1 (world), robot (flange) and camera of different dimensions; it is this carefully designed "beacon" that makes it possible to fuse high-precision data across sensors.

[0116] In addition, based on the joint angle of the first robot 2 and its kinematic model, a pose transformation matrix of the robot end flange coordinate system relative to the world coordinate system is calculated, and the mathematical formula is as follows:

[0117]

[0118] wherein, is the pose transformation matrix of the first robot 2 base coordinate system relative to the world coordinate system, is the joint angle vector the flange coordinate system pose calculated by forward kinematics;

[0119] for reliably associating high-precision external measurements with the robot's imperfect internal model, wherein the pose transformation matrix of the robot base coordinate system relative to the base coordinate system is a constant transformation matrix obtained through offline precision calibration; it accurately describes the robot's position in the world coordinate system, and since its calibration process can be decoupled from the production rhythm and repeatedly measured and optimized using high-precision means, the accuracy and reliability of this matrix are much higher than any pose data reported by the robot in real time;

[0120] is the real-time output of the robot forward kinematics model, which calculates the theoretical flange pose according to the current joint angle ; this calculation is based on the robot's design parameters (DH parameters), but is affected by all internal errors such as robot positioning errors, gear clearances, and link flexibility.

[0121] In addition, the point cloud data obtained by the structured light camera 4 is used to calculate the pose transformation matrix of the calibration plate 5 coordinate system relative to the camera coordinate system, and the calculation process is as follows:

[0122] Extract the point cloud clusters of the three spherical mirrors 6 from the point cloud data of the calibration plate 5 obtained by the structured light camera 4;

[0123] Fit the spheres using the RANSAC algorithm to obtain the coordinates of the centers of the three spherical mirrors 6;

[0124] Define the center coordinates of the spherical mirror 6 at the right angle point as the origin;

[0125] Define the direction from the origin to the center coordinates of the other spherical mirror 6 as the x-axis direction;

[0126] Take the normal vector perpendicular to the plane on which the center coordinates of the three target points lie as the z-axis, and the direction of the z-axis follows the right-hand rule;

[0127] The vector obtained by the cross product of the z-axis vector and the x-axis vector is the y-axis direction;

[0128] According to the origin, the x-axis direction, the z-axis direction, and the y-axis direction, construct the pose transformation matrix of the calibration plate 5 coordinate system relative to the camera coordinate system;

[0129] The process of determining the pose transformation matrix of the calibration board 5 coordinate system relative to the camera coordinate system uses the same logic as calculating the pose transformation matrix of the calibration board 5 coordinate system relative to the world coordinate system. This consistency in definition ensures that no matter from the world coordinate system or from the camera coordinate system, the "calibration board 5 coordinate system" we are talking about is the same rigid coordinate system defined by the same rules. Thus, the coordinate system chain relationship is strictly established.

[0130] Embodiment 3

[0131] As an embodiment, the average method is used to obtain the optimal external parameter, i.e., the final pose transformation matrix of the structured light camera 4 coordinate system relative to the first robot 2 end flange coordinate system, according to multiple preliminary external parameter estimates. The specific process is as follows:

[0132] The translation vector in the multiple external parameters is taken as an arithmetic mean, where the external parameter matrix is:

[0133]

[0134] In the formula, is a rotation matrix, is a translation vector;

[0135] The calculation formula for the arithmetic average of the translation vector is:

[0136]

[0137] In the formula, is the average of the translation vector, is the external parameter quantity, is the translation vector in the th external parameter matrix.

[0138] The spherical linear interpolation method is used to average the rotation matrix in the multiple external parameters. The specific process is as follows:

[0139] The multiple rotation matrices are converted into unit quaternion form .

[0140] The quaternions are sorted in descending order, and then the sequential recursive average method is used to calculate the average quaternion , and the calculation formula is:

[0141]

[0142]

[0143]

[0144] wherein, is the average quaternion, is the step recursion step, is the unit quaternion of the rotation matrix in descending order, is the angular distance between two unit quaternions, usually ranges between 0 and radians, which intuitively measures the "difference" or "distance" between two rotations;

[0145] wherein is used to denote spherical linear interpolation, i.e. when the parameter varies from 0 to 1, outputs a sequence of quaternions that smoothly and uniformly varies from the rotation quaternion to the rotation quaternion along the shortest path (great circle arc) on the quaternion sphere connecting and , and is the central angle of this circle arc, can be understood as the proportion of the path completed on this circle arc;

[0146] converts the average quaternion back to a rotation matrix;

[0147] combines the averaged translation vector with the averaged rotation matrix, i.e. the optimal extrinsic parameter.

[0148] The above merely describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for calibrating extrinsic parameters of a structured light camera based on a laser tracker, characterized in that, The application relates to a method for calibrating a structured light camera (4) and a laser tracker (1) in a world coordinate system, comprising the following steps: Positioning the laser tracker (1) in the world coordinate system by using the positions of known control points in the world coordinate system; A first robot (2) grabs the structured light camera (4), and a second robot (3) grabs a calibration board (5); a plurality of double-robot cooperative poses are planned; the two robots are controlled to cooperatively move in a working space; and a plurality of different calibration poses are obtained; At each calibration pose, the joint angles of the first robot (2), the point cloud data of the calibration board (5) obtained by the structured light camera (4), and the three-dimensional coordinates of the ball centers of three spherical mirrors (6) on the calibration board (5) measured by the laser tracker (1) are synchronously collected; the three spherical mirrors (6) are arranged at right angles; For each calibration pose, the pose transformation matrix of the calibration board (5) coordinate system relative to the world coordinate system is calculated based on the measurement data of the laser tracker (1) and the pose of the laser tracker (1) in the world coordinate system; the pose transformation matrix of the robot end flange coordinate system relative to the world coordinate system is calculated based on the joint angles of the first robot (2) and the kinematics model of the first robot (2); and the pose transformation matrix of the calibration board (5) coordinate system relative to the camera coordinate system is calculated based on the point cloud data obtained by the structured light camera (4); Based on the three kinds of pose transformation matrices obtained at each calibration pose, the external parameter pose transformation matrix of the structured light camera (4) coordinate system relative to the first robot (2) end flange coordinate system is solved through the coordinate system chain relationship; The external parameter pose transformation matrices obtained at multiple poses are averaged and fused to obtain a final pose transformation matrix, and the calculation process is as follows: The coordinate system chain relationship is constructed, and the mathematical formula is as follows: In the formula, is a pose transformation matrix of the coordinate system of the calibration plate (5) relative to the world coordinate system, is a pose transformation matrix of the coordinate system of the end flange of the first robot (2) relative to the world coordinate system; is a pose transformation matrix of the coordinate system of the calibration plate (5) relative to the camera coordinate system; Computing camera extrinsic parameter estimates The formula is: For each pose, a plurality of external parameter pose transformation matrices are obtained according to the camera external parameter estimation formula; The external parameter pose transformation matrices obtained at multiple poses are averaged and fused to obtain a final pose transformation matrix, and the specific process is as follows: The translation vectors in the plurality of external parameter pose transformation matrices are arithmetically averaged; The rotation matrices in the plurality of external parameter pose transformation matrices are averaged by adopting a spherical linear interpolation method, and the specific process is as follows: The plurality of rotation matrices are converted into unit quaternion forms; The quaternions are sorted in descending order, and then an average quaternion is calculated by adopting a sequential recursive averaging method, and the calculation formula is as follows: wherein is a unit quaternion of the plurality of rotation matrices in descending order, is an angular distance between two unit quaternions, is an extrinsic parameter quantity; The average quaternion is converted back into a rotation matrix; The averaged translation vector and the averaged rotation matrix are combined, and the final pose transformation matrix is obtained. 2.The laser tracker based structure light camera extrinsic calibration method of claim 1, wherein: The specific process of positioning the laser tracker (1) in the world coordinate system by using the positions of known control points in the world coordinate system is as follows: The laser tracker (1) is fixed in a working area, and the laser tracker (1) covers the working spaces of the two robots; A plurality of control points are arranged in the working area, and the coordinates of the control points in the tracker coordinate system are measured by adopting the laser tracker (1); The transformation relationship from the laser tracker (1) coordinate system to the global world coordinate system is established through control point registration.

3. The laser tracker based structure light camera extrinsic calibration method of claim 1, wherein: The pose transformation matrix of the calibration board (5) coordinate system relative to the world coordinate system is calculated based on the measurement data of the laser tracker (1) and the pose of the laser tracker (1) in the world coordinate system for each calibration pose, and the calculation process is as follows: The three-dimensional coordinates of the spherical mirror (6) ball center on the calibration board (5) measured by the laser tracker (1) are converted to the world coordinate system; The spherical mirror (6) located at the right angle point is defined as the origin; The direction from the origin to the ball center of another spherical mirror (6) is defined as the x-axis direction; The normal vector perpendicular to the plane where the three target points are located is taken as the z-axis, and the direction of the z-axis follows the right-hand rule; The vector obtained by the cross product of the z-axis vector and the x-axis vector is the y-axis direction; According to the origin, the x-axis direction, the z-axis direction and the y-axis direction, the pose transformation matrix of the calibration board (5) coordinate system relative to the world coordinate system is constructed.

4. The laser tracker based structure light camera extrinsic calibration method of claim 1, wherein: The pose transformation matrix of the robot flange coordinate system at the end of the robot relative to the world coordinate system is calculated based on the joint angle of the first robot (2) and its kinematic model, and the mathematical formula is as follows: wherein is a pose transformation matrix of the first robot (2) known base coordinate system with respect to the world coordinate system, is a joint angle vector is the pose of the flange coordinate system with respect to the robot base coordinate system calculated by forward kinematics.

5. The laser tracker based structure light camera extrinsic calibration method of claim 1, wherein: The pose transformation matrix of the calibration board (5) coordinate system relative to the camera coordinate system is calculated based on the point cloud data obtained by the structured light camera (4), and the calculation process is as follows: From the point cloud data of the calibration board (5) obtained by the structured light camera (4), the point cloud clusters of the three spherical mirrors (6) are extracted; The RANSAC algorithm is used to fit the sphere to obtain the ball center coordinates of the three spherical mirrors (6); The ball center coordinates of the spherical mirror (6) located at the right angle point are defined as the origin; The direction from the origin to the ball center coordinates of another spherical mirror (6) is defined as the x-axis direction; The normal vector perpendicular to the plane where the three target point ball centers are located is taken as the z-axis, and the direction of the z-axis follows the right-hand rule; The vector obtained by the cross product of the z-axis vector and the x-axis vector is the y-axis direction; According to the origin, the x-axis direction, the z-axis direction and the y-axis direction, the pose transformation matrix of the calibration board (5) coordinate system relative to the camera coordinate system is constructed.

Images