A fast hand-eye calibration method for multi-camera cooperative positioning

Abstract

A kind of fast hand-eye calibration method about multi-camera cooperative positioning, comprising: 1, mobile robot end three times and more, after each movement, detect calibration pattern with the same camera, record the position and pose of calibration pattern relative to camera after each movement and the position and pose of robot end relative to base;2, calculate the homogeneous transformation matrix of robot end relative to base coordinate system after each movement and the homogeneous transformation matrix of camera coordinate system relative to calibration pattern coordinate system after each movement;3, calculate the homogeneous transformation matrix of calibration pattern coordinate system relative to robot end coordinate system;4, calculate the homogeneous transformation matrix of each camera coordinate system relative to base coordinate system;5, calculate the final target point in base under the coordinates of multiple cameras, obtain the accurate position of target point, complete the position calibration of robot and multi-camera coordination.The present application has rapidity when carrying out robot hand-eye calibration task, and can significantly improve calibration efficiency.

Description

Technical Field

[0001] This invention relates to the field of robot position calibration technology, and in particular to a rapid hand-eye calibration method for multi-camera cooperative localization. Background Technology

[0002] Hand-eye calibration is a crucial task in robotics, particularly in dexterous maneuvers, used to determine the relative pose and positional relationship between the robot's hand and its vision system (eyes). When robots perform maneuvers in complex scenes, they often need to locate themselves using vision (camera). The camera acquires the target's pixel coordinates, which are then transformed into camera coordinates using camera intrinsics. However, these coordinates are only the target point coordinates in the camera coordinate system; a certain transformation relationship is needed to obtain the target point coordinates in the robot's base coordinate system before manipulation can proceed. To obtain the target point coordinates in the robot's base coordinate system, we need to calculate this "transformation relationship"—from the camera coordinate system to the robot coordinate system. This "transformation relationship" is hand-eye calibration. Hand-eye calibration can be performed in two different ways depending on the specific scenario: with the eye on the hand and with the eye outside the hand.

[0003] If the experiment requires multiple hand-eye calibrations for different individual scenarios, repeating the traditional hand-eye calibration process is very tedious and wastes a lot of time on a fixed and standardized calibration process.

[0004] Furthermore, traditional calibration methods are suitable for single-camera mission scenarios. However, when performing precise positioning tasks for multi-camera systems, traditional calibration methods are time-consuming, inefficient, and have limitations.

[0005] Furthermore, since the camera position remains unchanged during eye-to-hand calibration (the camera field of view remains unchanged), traditional methods have low reliability when repeated calibration is required if the camera position is improperly deployed or the field of view needs to be adjusted during operation. Summary of the Invention

[0006] This invention provides a rapid hand-eye calibration method for multi-camera cooperative localization to solve the technical problems mentioned in the background art.

[0007] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0008] This invention provides a rapid hand-eye calibration method for multi-camera cooperative localization, comprising the following steps:

[0009] S1. Fix the calibration pattern to the end of the robot, and fix multiple cameras at different positions outside the end of the robot so that the robot's operating space is within the field of view of the cameras; move the end of the robot three or more times, and use the same camera to detect the calibration pattern after each movement, and record the position and pose of the calibration pattern relative to the camera and the position and pose of the end of the robot relative to the base after each movement.

[0010] S2. Based on the position and pose of the robot end effector relative to the base coordinate system after each movement, calculate the homogeneous transformation matrix of the robot end effector coordinate system relative to the base coordinate system after each movement; and based on the position and pose of the calibration pattern relative to the camera after each movement recorded in S1, calculate the homogeneous transformation matrix of the camera coordinate system relative to the calibration pattern coordinate system after each movement.

[0011] S3. Based on the 6D pose data of the end effector and the calibration pattern recorded after each movement of the robot end effector, calculate the homogeneous transformation matrix of the robot end effector coordinate system relative to the base and the homogeneous transformation matrix of the camera coordinate system relative to the calibration pattern coordinate system during each movement. Solve using the least squares method to calculate the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system.

[0012] S4. Calculate the homogeneous transformation matrix of each camera coordinate system relative to the calibration pattern coordinate system in sequence. Then, use the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system obtained in S3, and the homogeneous transformation matrix of the robot end effector coordinate system relative to the base coordinate system calculated from the pose and position relationship between the robot end effector coordinate system and the base coordinate system on the teach pendant. Calculate the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system;

[0013] S5. Using the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system obtained in S4, calculate the coordinates of the target detection point under the base through different cameras, perform multi-camera collaborative localization, and then use the weighted average method to calculate the accurate three-dimensional coordinates of the target point under the base, thereby completing the position calibration of the robot and multiple cameras in coordination.

[0014] Furthermore, step S1 specifically includes the following steps:

[0015] S11. Fix the calibration pattern to the end of the robot, and fix multiple cameras at different positions outside the end of the robot so that the robot's operating space is within the field of view of the cameras.

[0016] S12. Move the robot end effector three times or more, and record the RPY angle of the robot end effector relative to the base and the X, Y, Z coordinates displayed on the teach pendant respectively;

[0017] S13. Simultaneously select one of the cameras, use the camera to take pictures of the calibration image, and calculate and record the RPY angle and X, Y, Z coordinates of the calibration pattern relative to the camera coordinate system after each movement of the robot end effector based on the Canny marker detection, Harris corner detection and PNP pose estimation algorithms.

[0018] Furthermore, step S2 specifically includes the following steps:

[0019] S21. Based on the position and pose angle data recorded in S1, calculate the homogeneous transformation matrix of the robot's end effector coordinate system relative to the base coordinate system after each movement. And the homogeneous transformation matrix of the calibration pattern coordinate system relative to the camera coordinate system after each movement.

[0020] S22, according to Find the homogeneous transformation matrix of the camera coordinate system relative to the calibration pattern coordinate system after each movement.

[0021] Furthermore, the homogeneous transformation matrix of the robot end-effector coordinate system relative to the base coordinate system in S21 The formula is as follows:

[0022]

[0023] in, The rotation matrix of the robot's end-effector coordinate system relative to the base coordinate system; The following relationship must be satisfied:

[0024]

[0025] In the relation, R z (α) represents the rotation matrix after the initial coordinate system is rotated α° around the z-axis; R y (β) represents the rotation matrix after rotating the initial coordinate system about the y-axis by β°; R x (γ) represents the rotation matrix after the initial coordinate system is rotated γ° around the x-axis.

[0026] Furthermore, the R z (α), R y (β) and R x (γ) is expressed by a formula, as follows:

[0027]

[0028]

[0029]

[0030] Furthermore, step S3 specifically includes the following steps:

[0031] S31. The homogeneous transformation matrices of the calibration pattern coordinate system relative to the camera coordinate system obtained during each movement are as follows:

[0032] S32, according to The homogeneous transformation matrices of the camera coordinate system relative to the calibration pattern coordinate system are obtained as follows:

[0033] S33. Simultaneously, record the end-effector data on the teach pendant during each movement, and calculate the homogeneous transformation matrix of the robot's end-effector coordinate system relative to the base coordinate system, respectively.

[0034] S34. Based on the homogeneous transformation matrix of the camera coordinate system relative to the robot's base coordinate system. Unchanged, that is Without changing the matrix equation, we get... The homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system is obtained by using the least squares method.

[0035] Furthermore, step S4 specifically includes the following steps:

[0036] S41. Move the robot end effector again to bring the calibration pattern into the camera's field of view. Take pictures of the calibration pattern with multiple cameras and calculate the position and pose relationship between the multiple cameras and the coordinate system of the calibration pattern using the PNP pose estimation algorithm.

[0037] S42. Then, based on the position and pose relationships between the multiple cameras and the calibration pattern coordinate system obtained in S41, calculate the homogeneous transformation matrix of the calibration pattern coordinate system relative to each camera coordinate system.

[0038] S43, then according to The homogeneous transformation matrix of each camera coordinate system relative to the calibration pattern coordinate system is obtained.

[0039] S44. Based on the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system obtained in S3, combined with the homogeneous transformation matrix of each camera coordinate system relative to the calibration pattern coordinate system obtained in S43, and the homogeneous transformation matrix of the robot end effector coordinate system relative to the base coordinate system calculated through the pose and position relationship between the robot end effector coordinate system and the base coordinate system on the teach pendant. The homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system is calculated.

[0040] Furthermore, step S5 specifically includes the following steps:

[0041] S51. Obtain the approximate three-dimensional coordinates of the target detection point in the camera coordinate system using each camera.

[0042] S52. Then, based on the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system obtained in S4, and using the approximate three-dimensional coordinates obtained in S51, construct the equation.

[0043] S53. Using the equation constructed in S52, and using the weighted average method to calculate the coordinates of the target detection point obtained by different cameras, multi-camera collaborative localization is performed to obtain the final accurate three-dimensional coordinates of the target point on the substrate, thereby completing the position calibration of the robot and multiple cameras in coordination.

[0044] Furthermore, the equation in S52 is as follows:

[0045]

[0046] in, This represents the three-dimensional coordinates of the target detection point t in the base coordinate system.

[0047] Furthermore, S53 is expressed by a formula, as follows:

[0048]

[0049]

[0050] Among them, P i These are the 3D coordinates of the point obtained under the i-th camera, respectively. Specifically, P... i This represents the coordinates of point p in one of the x, y, or z directions, while wi is the weight value of the corresponding point's 3D coordinates obtained under the i-th camera. This represents the precise position of the target point in the robot's base coordinate system, where i represents the nth camera and n represents the number of cameras.

[0051] The beneficial effects of this invention are:

[0052] 1. This invention offers speed in robot hand-eye calibration tasks, significantly improving calibration efficiency. When the robot faces different operational scenarios, this invention can execute repetitive yet efficient hand-eye calibration processes. Compared to traditional hand-eye calibration methods, this invention simplifies the calibration steps and overcomes the shortcomings of complex and low repeatability in traditional hand-eye calibration processes. Furthermore, this invention solves the problem of time-consuming multiple hand-eye calibrations required in actual engineering tasks, thereby significantly improving the robot's speed in overall operational tasks.

[0053] 2. This invention captures environmental information to the greatest extent possible, ensuring that the camera's wide field of view covers the entire robot's operating space. Simultaneously, this invention uses calibration QR codes instead of calibration patterns for experiments. This ensures that the robot's operating space remains within the camera's field of view during calibration, avoiding the limitations imposed by the robot's end effector and the position of the calibration pattern in traditional methods. This ensures the camera can acquire as much information as possible that is helpful for robot operation, thereby improving the accuracy and efficiency of the operation.

[0054] 3. This invention features flexible and variable camera position, enabling real-time hand-eye calibration. Through a single hand-eye relationship solution, this invention can instantly calculate the hand-eye matrix corresponding to changes in camera position, thus meeting the needs of practical applications with variable camera positions. For dynamic changes in camera position, this invention ensures flexible adjustment of the camera's field of view during calibration to maximize calibration results. This feature allows the robot to quickly adapt to different operating environments, guaranteeing the real-time performance and accuracy of hand-eye calibration.

[0055] 4. This invention possesses the capability to perform simultaneous multi-camera hand-eye calibration tasks, unrestricted by the limitations of traditional single-camera calibration. In robot operation scenarios composed of multi-camera vision systems, this invention can effectively perform synchronous hand-eye calibration tasks for multiple cameras, without being limited to the calibration of a single camera. Furthermore, this invention has the advantage of rapidly and in real-time completing the simultaneous calibration of multiple cameras, providing reliable hand-eye calibration support for the efficient operation of robots in complex multi-view environments. Attached Figure Description

[0056] Figure 1 This is a flowchart of the present invention;

[0057] Figure 2 This is a calibration diagram illustrating the relationship between the multi-camera system and the robot's base coordinate system in this invention. Detailed Implementation

[0058] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0059] Reference Figure 1 and Figure 2 This application provides a fast hand-eye calibration method for multi-camera cooperative localization, including the following steps:

[0060] S1. Fix the calibration pattern to the end of the robot, and fix multiple cameras at different positions outside the end of the robot so that the robot's operating space is within the field of view of the cameras; move the end of the robot three or more times, and use the same camera to detect the calibration pattern after each movement, and record the position and pose of the calibration pattern relative to the camera and the position and pose of the end of the robot relative to the base after each movement.

[0061] Preferably, the number of cameras is three or more. In this application, three cameras are preferred. The three cameras are named camera1, camera2, and camera3 respectively. The invention will be described in detail below with reference to three cameras.

[0062] S2. Based on the position and pose of the robot end effector relative to the base coordinate system after each movement, calculate the homogeneous transformation matrix of the robot end effector coordinate system relative to the base coordinate system after each movement; and based on the position and pose of the calibration pattern relative to the camera after each movement recorded in S1, calculate the homogeneous transformation matrix of the camera coordinate system relative to the calibration pattern coordinate system after each movement.

[0063] S3. Based on the 6D pose data of the end effector and the calibration pattern recorded after each movement of the robot end effector, calculate the homogeneous transformation matrix of the robot end effector coordinate system relative to the base and the homogeneous transformation matrix of the camera coordinate system relative to the calibration pattern coordinate system during each movement. Solve using the least squares method to calculate the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system.

[0064] S4. Calculate the homogeneous transformation matrices of the three camera coordinate systems relative to the calibration pattern coordinate system in sequence. Then, using the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system obtained in S3, and the homogeneous transformation matrix of the robot end effector coordinate system relative to the base coordinate system calculated from the pose and position relationship between the robot end effector coordinate system and the base coordinate system on the teach pendant,... Calculate the homogeneous transformation matrices of the three camera coordinate systems relative to the base coordinate system;

[0065] S5. Using the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system obtained in S4, calculate the coordinates of the target detection point under the base through different cameras, perform multi-camera collaborative localization, and then use the weighted average method to calculate the accurate three-dimensional coordinates of the target point under the base, thereby completing the position calibration of the robot and multiple cameras in coordination.

[0066] In this invention, the precise three-dimensional coordinates are the actual three-dimensional coordinates accurate to three decimal places.

[0067] In some embodiments, S1 specifically includes the following steps:

[0068] S11. Fix the calibration pattern to the end of the robot, and fix three cameras at different positions outside the end of the robot so that the robot's operating space is within the field of view of the cameras.

[0069] S12. Move the robot end effector three times or more, and record the RPY angle of the robot end effector relative to the base and the X, Y, and Z coordinates of the robot end effector relative to the base as displayed on the teach pendant respectively;

[0070] Based on the RPY angle and X, Y, and Z coordinates of the robot's end effector relative to the base displayed on the teach pendant, data was recorded. The specific data recorded is as follows:

[0071] The position and pose data of the end relative to the base after the first movement: X1, Y1, Z1, α1, β1, γ1;

[0072] The position and pose data of the end relative to the base after the second movement: X2, Y2, Z2, α2, β2, Y2;

[0073] The position and pose data of the end relative to the base after the third movement: X3, Y3, Z3, α3, β3, γ3;

[0074] ...

[0075] S13. Simultaneously select one of the cameras, such as camera1, and use this camera to take pictures of the calibration pattern. Calculate and record the RPY angle of the calibration pattern relative to the camera coordinate system and the X, Y, and Z coordinates of the calibration pattern relative to the camera coordinate system after each movement of the robot's end effector, based on the Canny marker detection, Harris corner detection, and PNP pose estimation algorithms. The recorded data are as follows:

[0076] After the first movement of the end effector, the position and pose angle of the calibration pattern relative to the camera are: X11, Y11, Z11, α11, β11, γ11;

[0077] After the second end-effector movement, the position and pose angle of the calibration pattern relative to the camera are: X22, Y22, Z22, α22, β22, γ22;

[0078] After the third end-effector movement, the position and pose angle of the calibration pattern relative to the camera are: X33, Y33, Z33, α33, β33, γ33;

[0079] ...

[0080] In some embodiments, S2 specifically includes the following steps:

[0081] S21. Based on the position and pose angle data recorded in S1, calculate the homogeneous transformation matrix of the robot's end effector coordinate system relative to the base coordinate system after each movement. And the homogeneous transformation matrix of the calibration pattern coordinate system relative to the camera coordinate system after each movement.

[0082] S22, according to Find the homogeneous transformation matrix of the camera coordinate system relative to the calibration pattern coordinate system after each movement. The calibration pattern is fixed to the end of the robot, ensuring that the calibration pattern is clearly imaged within the camera's field of view and that the relative position of the calibration pattern and the robot's end remains fixed; the camera position is fixed outside the robot.

[0083] In some embodiments, the homogeneous transformation matrix of the robot end-effector coordinate system relative to the base coordinate system in S21 The formula is as follows:

[0084]

[0085] in, The rotation matrix of the robot's end-effector coordinate system relative to the base coordinate system; The following relationship must be satisfied:

[0086]

[0087] In the relation, R z (α) represents the rotation matrix after rotating the initial coordinate system about the z-axis by α°; Ry(β) represents the rotation matrix after rotating the initial coordinate system about the y-axis by β°; R x (γ) represents the rotation matrix after the initial coordinate system is rotated γ° around the x-axis.

[0088] In some embodiments, the R z (α), R y (β) and R x (γ) is expressed by a formula, as follows:

[0089]

[0090]

[0091]

[0092] In some embodiments, S3 specifically includes the following steps:

[0093] S31. The homogeneous transformation matrices of the calibration pattern coordinate system relative to the camera coordinate system obtained during each movement are as follows:

[0094] S32, according to The homogeneous transformation matrices of the camera coordinate system relative to the calibration pattern coordinate system are obtained as follows:

[0095] S33. Simultaneously, record the end-effector data on the teach pendant during each movement, and calculate the homogeneous transformation matrix of the robot's end-effector coordinate system relative to the base coordinate system, respectively.

[0096] S34. Based on the homogeneous transformation matrix of the camera coordinate system relative to the robot's base coordinate system. Unchanged, that is Without changing the matrix equation, we get... The homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system is obtained by using the least squares method.

[0097] In some embodiments, S4 specifically includes the following steps:

[0098] S41. Move the robot's end effector again to bring the calibration pattern within the camera's field of view. Take pictures of the calibration pattern using cameras 1, 2, and 3 respectively. Calculate the position and pose relationship between the three cameras and the coordinate system of the calibration pattern using the PNP pose estimation algorithm, obtaining the following data in sequence:

[0099] The relative position and pose data of Camera1 and the calibration pattern are X111, Y111, Z111, α111, β111, γ111, respectively;

[0100] The relative position and pose data of Camera2 and the calibration pattern are X222, Y222, Z222, α222, β222, γ222, respectively;

[0101] The relative position and pose data of Camera3 and the calibration pattern are X333, Y333, Z333, α333, β333, γ333, respectively;

[0102] S42. Then, based on the position and pose relationships between the multiple cameras and the calibration pattern coordinate system obtained in S41, calculate the homogeneous transformation matrix of the calibration pattern coordinate system relative to each camera coordinate system.

[0103] S43, then according to The homogeneous transformation matrix of each camera coordinate system relative to the calibration pattern coordinate system is obtained.

[0104] S44. Based on the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system obtained in S3, combined with the homogeneous transformation matrix of each camera coordinate system relative to the calibration pattern coordinate system obtained in S43, and the homogeneous transformation matrix of the robot end effector coordinate system relative to the base coordinate system calculated through the pose and position relationship between the robot end effector coordinate system and the base coordinate system on the teach pendant. The homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system is calculated. And the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system The following relationship must be satisfied:

[0105]

[0106]

[0107]

[0108] In S4, camera1 is used to perform the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot's end effector coordinate system. Solve, and then use the solution It can quickly solve for the homogeneous transformation matrices of camera2 and camera3 relative to the base coordinate system.

[0109] In some embodiments, S5 specifically includes the following steps:

[0110] S51. Obtain the approximate three-dimensional coordinates of the target detection point in the camera coordinate system using each camera.

[0111] S52. Then, based on the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system obtained in S4, and using the approximate three-dimensional coordinates obtained in S51, construct the equation.

[0112] S53. Using the equation constructed in S52, and using the weighted average method to calculate the coordinates of the target detection point obtained by different cameras, multi-camera collaborative localization is performed to obtain the final accurate three-dimensional coordinates of the target point on the substrate, thereby completing the position calibration of the robot and multiple cameras in coordination.

[0113] In some embodiments, the equation in S52 is specifically as follows:

[0114]

[0115] in, This represents the three-dimensional coordinates of the target detection point t in the base coordinate system.

[0116] In some embodiments, S53 is expressed by a formula, as follows:

[0117]

[0118]

[0119] Among them, P i These are the 3D coordinates of the point obtained under the i-th camera, respectively. Specifically, P... i This represents the coordinates of point p in one of the x, y, or z directions, while wi is the weight value of the corresponding point's 3D coordinates obtained under the i-th camera. This represents the precise position of the target point in the robot's base coordinate system, where i represents the nth camera and n represents the number of cameras.

[0120] This invention has the following advantages: 1. It is fast in performing robot hand-eye calibration tasks, significantly improving calibration efficiency. When the robot faces different operating scenarios, this invention can execute repetitive yet efficient hand-eye calibration processes. Compared with traditional hand-eye calibration methods, this invention simplifies the calibration steps and overcomes the shortcomings of complexity and low repeatability in traditional hand-eye calibration processes. Furthermore, this invention solves the problem of time-consuming multiple hand-eye calibrations required in actual engineering tasks, thereby significantly improving the robot's speed in overall operation tasks.

[0121] 2. This invention captures environmental information to the greatest extent possible, ensuring that the camera's wide field of view covers the entire robot's operating space. Simultaneously, this invention uses calibration QR codes instead of calibration patterns for experiments. This ensures that the robot's operating space remains within the camera's field of view during calibration, avoiding the limitations imposed by the robot's end effector and the position of the calibration pattern in traditional methods. This ensures the camera can acquire as much information as possible that is helpful for robot operation, thereby improving the accuracy and efficiency of the operation.

[0122] 3. This invention features flexible and variable camera position, enabling real-time hand-eye calibration. Through a single hand-eye relationship solution, this invention can instantly calculate the hand-eye matrix corresponding to changes in camera position, thus meeting the needs of practical applications with variable camera positions. For dynamic changes in camera position, this invention ensures flexible adjustment of the camera's field of view during calibration to maximize calibration results. This feature allows the robot to quickly adapt to different operating environments, guaranteeing the real-time performance and accuracy of hand-eye calibration.

[0123] 4. This invention possesses the capability to perform simultaneous multi-camera hand-eye calibration tasks, unrestricted by the limitations of traditional single-camera calibration. In robot operation scenarios composed of multi-camera vision systems, this invention can effectively perform synchronous hand-eye calibration tasks for multiple cameras, without being limited to the calibration of a single camera. Furthermore, this invention has the advantage of rapidly and in real-time completing the simultaneous calibration of multiple cameras, providing reliable hand-eye calibration support for the efficient operation of robots in complex multi-view environments.

[0124] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A rapid hand-eye calibration method for multi-camera cooperative localization, characterized in that, Includes the following steps: S1. Fix the calibration pattern to the end of the robot, and fix multiple cameras at different positions outside the end of the robot so that the robot's operating space is within the field of view of the cameras. The mobile robot end effector performs three or more movements, and after each movement, the same camera is used to detect the calibration pattern. The position and pose of the calibration pattern relative to the camera and the position and pose of the robot end effector relative to the base are recorded after each movement. S2. Based on the position and pose of the robot end effector after moving relative to the base coordinate system, calculate the homogeneous transformation matrix of the robot end effector coordinate system relative to the base coordinate system after each movement; then calculate the homogeneous transformation matrix of the camera coordinate system relative to the calibration pattern coordinate system after each movement. S3. Based on the 6D pose data of the end effector and the calibration pattern recorded after each movement of the robot end effector, calculate the homogeneous transformation matrix of the robot end effector coordinate system relative to the base and the homogeneous transformation matrix of the camera coordinate system relative to the calibration pattern coordinate system during each movement. Solve using the least squares method to calculate the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system. S4. Calculate the homogeneous transformation matrix of each camera coordinate system relative to the calibration pattern coordinate system in sequence. Using the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end coordinate system obtained in S3, as well as the position and pose information of the robot end relative to the base, calculate the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system. S5. Using the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system obtained in S4, calculate the coordinates of the target point under the base through different cameras, perform multi-camera collaborative positioning, and then use the weighted average method to calculate the accurate three-dimensional coordinates of the target point under the base, thereby completing the position calibration of the robot and multiple cameras in coordination.

2. The rapid hand-eye calibration method according to claim 1, characterized in that, S1 specifically includes the following steps: S11. Fix the calibration pattern to the end of the robot, and fix multiple cameras at different positions outside the end of the robot so that the robot's operating space is within the field of view of the cameras. S12. Move the robot end effector three times or more, and record the RPY angle of the robot end effector relative to the base and the X, Y, Z coordinates displayed on the teach pendant respectively; S13. Simultaneously select one of the cameras, use the camera to take pictures of the calibration pattern, and calculate and record the RPY angle and X, Y, Z coordinates of the calibration pattern relative to the camera coordinate system after each movement of the robot end effector based on the Canny marker detection, Harris corner detection and PNP attitude estimation algorithms.

3. The rapid hand-eye calibration method according to claim 1, characterized in that, S2 specifically includes the following steps: S21. Based on the position and pose angle data recorded in S1, calculate the homogeneous transformation matrix of the robot's end effector coordinate system relative to the base coordinate system after each movement. And the homogeneous transformation matrix of the calibration pattern coordinate system relative to the camera coordinate system after each movement. ; S22, according to The homogeneous transformation matrix of the camera coordinate system relative to the calibration pattern coordinate system after each movement is obtained. .

4. The rapid hand-eye calibration method according to claim 3, characterized in that, The homogeneous transformation matrix of the robot end effector coordinate system relative to the base coordinate system in S21 The formula is as follows: in, The rotation matrix of the robot's end-effector coordinate system relative to the base coordinate system; The following relationship must be satisfied: ； In the relation, This represents the rotation matrix after the initial coordinate system is rotated α° around the z-axis; This represents the rotation matrix after rotating the initial coordinate system by β° around the y-axis; This indicates that the initial coordinate system rotates around the x-axis. The rotation matrix after °.

5. The rapid hand-eye calibration method according to claim 4, characterized in that, The , and The formula is as follows: 。 6. The rapid hand-eye calibration method according to claim 5, characterized in that, S3 specifically includes the following steps: S31. The homogeneous transformation matrices of the calibration pattern coordinate system relative to the camera coordinate system obtained during each movement are as follows: , , ......; S32, according to The homogeneous transformation matrices of the camera coordinate system relative to the calibration pattern coordinate system are obtained as follows: , , ......; S33. Simultaneously, record the end-effector data on the teach pendant during each movement, and calculate the homogeneous transformation matrix of the robot's end-effector coordinate system relative to the base coordinate system, respectively. , , ......; S34. Based on the homogeneous transformation matrix of the camera coordinate system relative to the robot's base coordinate system. Unchanged, that is = * * Without changing the matrix equation, we get... * * = * * = * * =......;The homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system is obtained by using the least squares method. .

7. The rapid hand-eye calibration method according to claim 6, characterized in that, S4 specifically includes the following steps: S41. Move the robot end effector again to bring the calibration pattern into the camera's field of view. Take pictures of the calibration pattern with multiple cameras and calculate the position and pose relationship between the multiple cameras and the coordinate system of the calibration pattern using the PNP pose estimation algorithm. S42. Then, based on the position and pose relationships between the multiple cameras and the calibration pattern coordinate system obtained in S41, calculate the homogeneous transformation matrix of the calibration pattern coordinate system relative to each camera coordinate system. , , ......; S43, then according to The homogeneous transformation matrix of each camera coordinate system relative to the calibration pattern coordinate system is obtained. , , ......; S44. Based on the homogeneous transformation matrix of the calibration pattern coordinate system relative to the robot end effector coordinate system obtained in S3, combined with the homogeneous transformation matrix of each camera coordinate system relative to the calibration pattern coordinate system obtained in S43, and the homogeneous transformation matrix of the robot end effector coordinate system relative to the base coordinate system calculated through the pose and position relationship between the robot end effector coordinate system and the base coordinate system on the teach pendant. Then, the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system is calculated. , , ......

8. The rapid hand-eye calibration method according to claim 7, characterized in that, S5 specifically includes the following steps: S51. Obtain the approximate three-dimensional coordinates of the target point in the camera coordinate system using each camera. , , ......; S52. Then, based on the homogeneous transformation matrix of each camera coordinate system relative to the base coordinate system obtained in S4, and using the approximate three-dimensional coordinates obtained in S51, construct the equation. S53. Using the equation constructed in S52, and using the weighted average method to calculate the coordinates of the target point obtained by different cameras, multi-camera collaborative positioning is performed to obtain the final accurate three-dimensional coordinates of the target point on the substrate, thereby completing the position calibration of the robot and multiple cameras in coordination.

9. The rapid hand-eye calibration method according to claim 8, characterized in that, The specific equation in S52 is as follows: = * ， = * ， = * ......； in, This represents the three-dimensional coordinates of the target point t in the base coordinate system.

10. The rapid hand-eye calibration method according to claim 9, characterized in that, S53 is expressed by a formula, as follows: in, These are the 3D coordinates of the point obtained under the i-th camera, respectively. The specific solution... It only represents the coordinates of point p in one of the x, y, or z directions. These are the weight values ​​of the 3D coordinates of the corresponding point obtained under the i-th camera. This represents the precise position of the target point in the robot's base coordinate system, where i represents the nth camera and n represents the number of cameras.

Images