A method and apparatus for hand-eye calibration of a robotic arm based on a point laser emitter

Abstract

This invention discloses a hand-eye calibration method and apparatus for a robotic arm based on a point laser emitter, relating to the field of camera calibration. The calibration method includes: calibrating the required calibration plane using a camera, calibration plate, and camera-robotic arm; adjusting the point laser emitter to the required pose using the laser emitter robotic arm, projecting the point laser beam onto the calibration plane, and acquiring the image through the camera to obtain the coordinates of the projection point of the point laser beam on the calibration plane in the camera coordinate system; considering the two-dimensional pose of the point laser beam facing downwards in the point laser emitter coordinate system, using the PnP algorithm to solve the transformation matrix, thereby obtaining the calibration result. This invention can effectively control the robotic arm to make the laser emitter emit a laser beam in a specific pose, and calculate the hand-eye calibration matrix from the coordinates of the point laser beam on the calibration plane, thereby simplifying the calibration process to the greatest extent and improving calibration accuracy.

Description

Technical Field

[0001] This invention relates to the field of camera calibration, and in particular to a method and apparatus for calibrating the hand and eye of a robotic arm based on a point laser emitter. Background Technology

[0002] Hand-eye calibration is a crucial concept in robotics, playing a vital role in achieving precise position and posture control during robot task execution. In hand-eye calibration, the "hand" refers to the robot's robotic arm or other actuators, responsible for performing actual physical operations such as grasping, moving, and rotating; while the "eye" refers to the camera or other sensing devices used to acquire images or sensor data from the environment. Data acquired by the camera cannot be directly used as parameters to control the robotic arm's movement because the transformation relationship between the robot's coordinate system and the camera's coordinate system is unknown. Therefore, the hand-eye calibration process involves solving for the coordinate transformation relationship between the robot's end-effector coordinate system and the camera's coordinate system, or between the robot's base coordinate system and the camera's coordinate system. There are two types of hand-eye calibration: The first type is where the camera (eye) is fixed to the end of the robotic arm (hand), and the camera moves with the robotic arm. This type of hand-eye calibration is called eye-in-hand. The second type is where the camera (eye) is separated from the robotic arm (hand), and the camera is fixed to the robot's base. The movement of the robotic arm has no effect on the camera. This type of hand-eye calibration is called eye-to-hand.

[0003] Traditional calibration methods involve fixing a calibration plate to the end of a robotic arm, with the camera fixed outside the arm, and the camera and the robotic arm base remaining relatively stationary. The coordinate system involved is the camera coordinate system C. camera Calibration plate coordinate system C chessboard The coordinate system C of the end effector of the robotic arm end The coordinate system of the robot arm base C base The transformation relationship from the camera coordinate system to the robot arm base coordinate system This refers to the hand-eye calibration result that needs to be solved. It takes the form of a homogeneous transformation matrix, containing a rotation matrix and a translation vector. The specific steps involve controlling the robot arm to move to different positions and orientations to obtain C. base To C end The transformation relationship is obtained by simultaneously capturing image data of the calibration board using a camera, thus obtaining C. camera To C chessboard The transformation relationship can be established. Based on these two transformation relationships, C can be calculated. camera To C base The transformation relationship between them.

[0004] However, when the calibration board is outside the camera's field of view, the above calibration method becomes very difficult, or even impossible. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for hand-eye calibration of a robotic arm based on a point laser emitter. This invention can effectively control the robotic arm to make the laser emitter emit a point laser beam in a specific posture, and calculate the hand-eye calibration matrix from the coordinates of the projection points of the point laser beam on the calibration plane, so as to simplify the calibration process to the greatest extent and improve the calibration accuracy.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] In a first aspect, the present invention provides a method for robotic arm hand-eye calibration based on a point laser emitter, comprising:

[0008] The camera is calibrated to obtain the camera's intrinsic parameter matrix, and a first control command is determined based on the camera's intrinsic parameter matrix; the first control command is used to fix the camera.

[0009] A calibration plane is selected to obtain the homogeneous transformation matrix from the camera coordinate system to the calibration board coordinate system, and a second control command is determined based on the homogeneous transformation matrix from the camera coordinate system to the calibration board coordinate system; the second control command is used to fix the calibration board; the calibration plane is the plane determined based on the calibration board.

[0010] The motor rotation angle when the point laser beam is projected onto the calibration plane and the pixel coordinates of the projection point of the point laser beam on the calibration plane within the actual camera image are obtained; the point laser beam is the laser beam emitted by the point laser emitter; the motor is used to control the rotation of the point laser emitter.

[0011] Based on the motor rotation angle, the pixel coordinates of the projection point of the point laser beam on the calibration plane within the virtual camera image are calculated; the virtual camera is an imaginary camera with its optical center at the origin of the point laser emitter's coordinate system, oriented coinciding with the z-axis of the point laser emitter, and its intrinsic parameter matrix is ​​fixed.

[0012] Based on the pixel coordinates of the projection point of the point laser beam on the calibration plane within the actual camera image, the three-dimensional coordinates of the projection point of the point laser beam on the calibration plane in the actual camera coordinate system are calculated; the actual camera coordinate system is a coordinate system established based on the camera.

[0013] The PnP algorithm is used to calculate the transformation matrix from the actual camera coordinate system to the virtual camera coordinate system; the virtual camera coordinate system is a coordinate system established based on the point laser emitter.

[0014] Optionally, the formula for calculating the pixel coordinates of the projection point of the point laser beam on the calibration plane within the virtual camera image is:

[0015] (x′ i ,y′ i )=(100(1+tan(θ xi ),100(1-tan(θ yi )));

[0016] Among them, (x' i ,y' i Let be the pixel coordinates of the projection point of the i-th laser beam on the calibration plane within the virtual camera image, where i is a positive integer, (θ) xi θ yi ) represents the rotation angle of the motor corresponding to the projection point of the i-th laser beam on the calibration plane.

[0017] Optionally, the formula for calculating the three-dimensional coordinates of the projection point of the point laser beam on the calibration plane in the actual camera coordinate system is as follows:

[0018]

[0019] Among them, P ci Let be the three-dimensional coordinates of the projection point of the i-th laser beam onto the calibration plane in the actual camera coordinate system, where i is a positive integer, (x... i y i Let z be the pixel coordinates of the center of the projection point of the i-th point laser beam on the calibration plane in the camera image. i Let K be the image depth of the pixel where the center of the projection point of the i-th point laser beam is located on the calibration plane, and K is the intrinsic parameter matrix.

[0020] Optionally, the transformation relationship of the transformation matrix from the actual camera coordinate system to the virtual camera coordinate system is as follows:

[0021]

[0022] in, Let be the homogeneous transformation matrix from the camera coordinate system to the point laser emitter coordinate system. The rotation matrix for the calibration results. P is the translation vector of the calibration result. c Let P be the coordinates of the projection point of the point laser beam onto the calibration plane in the camera coordinate system. l The coordinates of the projection point of the point laser beam onto the calibration plane in the coordinate system of the point laser emitter.

[0023] Optionally, the number of projection points of the point laser beam on the calibration plane is greater than or equal to 4, and all point laser beams are not completely collinear.

[0024] Optionally, before calculating the three-dimensional coordinates of the projection point of the point laser beam on the calibration plane in the actual camera coordinate system based on the pixel coordinates of the projection point of the point laser beam on the calibration plane in the actual camera image, the method further includes:

[0025] Obtaining the pixel coordinates of the projection point of the point laser beam onto the calibration plane, specifically including:

[0026] Convert the camera image from RGB space to HSV space using OpenCV.

[0027] In the HSV space, pixels containing the projection points of the point laser beam on the calibration plane are obtained by setting a threshold.

[0028] Based on the pixel points of the projection points of the point laser beam on the calibration plane, the contour of the projection points of the point laser beam on the calibration plane is obtained through the findContours() function in OpenCV.

[0029] The centroid coordinates of the projection point contour of the point laser beam on the calibration plane can be obtained using the moments() function in OpenCV.

[0030] The centroid coordinates of the projection point contour of the point laser beam on the calibration plane are used as the pixel coordinates of the projection point of the point laser beam on the calibration plane.

[0031] Secondly, the present invention provides a robotic arm hand-eye calibration device based on a point laser emitter, comprising:

[0032] Control module, camera robotic arm, calibration board, laser emitter robotic arm, camera mechanism and laser emitter mechanism.

[0033] The control module is used to execute the steps of the robotic arm hand-eye calibration method based on a point laser emitter as described in the first aspect.

[0034] The camera robotic arm is used to receive a first control command to fix the camera mechanism; the camera mechanism includes a camera.

[0035] The calibration plate is used to receive a second control command to fix the calibration plane.

[0036] A laser emitter robotic arm is used to fix the laser emitter mechanism; the laser emitter mechanism includes a point laser emitter; the point laser emitter is used to emit a point laser beam.

[0037] Optionally, the camera mechanism further includes:

[0038] A first motor and a second motor; the first motor and the second motor are placed vertically, and the first motor and the second motor enable the camera to move in two coordinate axis planes, and the rotation axes of the first motor and the second motor intersect at a point.

[0039] Optionally, the laser emitter mechanism further includes:

[0040] A third motor and a fourth motor; the third motor and the fourth motor are placed vertically, and the third motor and the fourth motor enable the point laser emitter to move in two coordinate axis planes, and the rotation axes of the third motor and the fourth motor intersect at a point.

[0041] The coordinate system of the point laser emitter is fixed relative to the coordinate system of the camera robotic arm base and does not rotate with the rotation of the laser emitter robotic arm.

[0042] Optionally, the camera is a depth camera.

[0043] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0044] This invention provides a method and apparatus for hand-eye calibration of a robotic arm based on a point laser emitter. The invention selects the desired calibration plane using a camera and a calibration plate; the robotic arm adjusts the point laser emitter to the desired pose, projects the point laser beam onto the calibration plane, and captures the image to obtain the coordinates of the projection point of the point laser beam on the calibration plane in the camera coordinate system; considering the two-dimensional pose of the point laser beam facing downwards in the laser emitter coordinate system, the PnP algorithm is used to solve the transformation matrix, thereby obtaining the calibration result. Compared with existing technologies, this invention simplifies the calibration steps and improves calibration accuracy. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating a robotic arm hand-eye calibration method based on a point laser emitter, as provided in Embodiment 1 of the present invention.

[0047] Figure 2 This is a schematic diagram of a robotic arm hand-eye calibration device based on a point laser emitter, provided in Embodiment 2 of the present invention.

[0048] Figure 3This is a schematic diagram illustrating the system coordinate system specification provided in Embodiment 2 of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] The purpose of this invention is to provide a method and apparatus for hand-eye calibration of a robotic arm based on a point laser emitter. This invention can effectively control the robotic arm to make the laser emitter emit a point laser beam in a specific posture, and calculate the hand-eye calibration matrix from the coordinates of the projection points of the point laser beam on the calibration plane, so as to simplify the calibration process to the greatest extent and improve the calibration accuracy.

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Example 1:

[0053] like Figure 1 As shown, this embodiment provides a robotic arm hand-eye calibration method based on a point laser emitter, including:

[0054] S1: The camera is calibrated to obtain the camera's intrinsic parameter matrix, and a first control command is determined based on the camera's intrinsic parameter matrix; the first control command is used to fix the camera.

[0055] S2: Select a calibration plane, obtain the homogeneous transformation matrix from the camera coordinate system to the calibration board coordinate system, and determine a second control command based on the homogeneous transformation matrix from the camera coordinate system to the calibration board coordinate system; the second control command is used to fix the calibration board; the calibration plane is the plane determined based on the calibration board.

[0056] S3: Obtain the motor rotation angle when the point laser beam is projected onto the calibration plane and the pixel coordinates of the projection point of the point laser beam on the calibration plane within the actual camera image; the point laser beam is the laser beam emitted by the point laser emitter; the motor is used to control the rotation of the point laser emitter.

[0057] S4: Based on the motor rotation angle, calculate the pixel coordinates of the projection point of the point laser beam on the calibration plane within the virtual camera image; the virtual camera is an imaginary camera with its optical center at the origin of the point laser emitter's coordinate system, oriented coinciding with the z-axis of the point laser emitter, and its intrinsic parameter matrix is ​​fixed.

[0058] S5: Based on the pixel coordinates of the projection point of the point laser beam on the calibration plane in the actual camera image, calculate the three-dimensional coordinates of the projection point of the point laser beam on the calibration plane in the actual camera coordinate system; the actual camera coordinate system is a coordinate system established based on the camera.

[0059] S6: Using the PnP algorithm, the transformation matrix from the actual camera coordinate system to the virtual camera coordinate system is calculated; the virtual camera coordinate system is a coordinate system established based on the point laser emitter.

[0060] As an optional implementation method provided in this embodiment, in step S1, after taking multiple pictures of the calibration plate in different poses, calibrating using MATLAB to obtain the intrinsic parameter matrix of the camera, the camera is fixed by the first control command.

[0061] As an optional implementation method provided in this embodiment, in step S2, the calibration of the calibration plane requires the camera to be calibrated first using a calibration plate. Then, the calibration plate and the camera are fixed, and the checkerboard surface of the calibration plate is used as the calibration plane. The calibration plane is then calibrated again using the camera to obtain its plane equation in the camera coordinate system.

[0062] Specifically, after fixing the camera, fix the calibration plate and use its surface as the calibration plane. The extrinsic parameters, i.e., the homogeneous transformation matrix from the camera coordinate system to the calibration plate coordinate system, can then be obtained through calibration. Let the calibration plane be the plane formed by the x and y axes of the calibration plate coordinate system. Then, the plane equation of the calibration plane in the camera coordinate system can be obtained by... The result is obtained through the following process:

[0063] Using homogeneous transformation matrix rotation matrix in Translation vector The unit vectors of the x and y axes of the calibration plate coordinate system can be obtained. Representation in camera coordinate system:

[0064]

[0065] The normal vector of the calibration plane can be obtained through Represented by the cross product:

[0066]

[0067] A general plane equation can be expressed in the form Ax + By + Cz + d = 0, where the method vector is... The equation of the calibration plane is then:

[0068] n1x+n2y+n3z+d=0 (3);

[0069] Where d is an unknown constant.

[0070] The fact that the origin of the coordinate system of the calibration plate must lie on the calibration plane is the basis for the coordinate system. If the plane equation satisfies equation (3), the constant d can be obtained, and thus the calibration plane equation is obtained.

[0071] It should be noted that all the above calculation results are expressed in the camera coordinate system.

[0072] Next, adjust the rotation angle of the motor of the point laser emitter structure so that the point laser beam of the point laser emitter is on the calibration plane and within the camera's field of view.

[0073] At this point, it is necessary to obtain the pixel coordinates of the projection point of the point laser beam on the calibration plane. The steps are as follows:

[0074] ① Convert the camera image from RGB space to HSV space using OpenCV.

[0075] ② In the HSV space, pixels containing the projection points of the point laser beam on the calibration plane are obtained by setting a threshold.

[0076] ③ Based on the pixel points of the projection points of the point laser beam on the calibration plane, the contour of the projection points of the point laser beam on the calibration plane is obtained through the findContours() function in OpenCV.

[0077] ④ The centroid coordinates of the projection point contour of the point laser beam on the calibration plane are obtained by using the moments() function of OpenCV.

[0078] ⑤ The centroid coordinates of the projection point contour of the point laser beam on the calibration plane are used as the pixel coordinates of the projection point of the point laser beam on the calibration plane.

[0079] It should be noted that the pixel coordinates of the projection points of the laser beam onto the calibration plane are determined using computer vision for automatic laser beam localization. Considering that different visual effects of the laser point on the calibration plane under varying ambient lighting conditions and different observation angles will produce different imaging results in the final camera image, using a traditional vision algorithm with a single threshold for laser point recognition and localization may lead to biased or even erroneous recognition results. Therefore, a target detection neural network based on the YOLOv5s network model is first trained to recognize the contour of the laser point in the camera-acquired image. Then, the centroid of the contour is calculated using the image moment method to accurately locate the coordinates of the laser point center, meeting the needs of different environments and observation angles in practical use.

[0080] Let P be the coordinates of the projection point of the laser beam onto the calibration plane at this point in the camera coordinate system. c The coordinates of the point laser emitter in the coordinate system are P. l Then there is a transformation relationship between them:

[0081]

[0082] in Let be the homogeneous transformation matrix from the camera coordinate system to the point laser emitter coordinate system. The rotation matrix for the calibration results. P is the translation vector of the calibration result. c Let P be the coordinates of the projection point of the point laser beam onto the calibration plane in the camera coordinate system. l The coordinates of the projection point of the point laser beam onto the calibration plane in the coordinate system of the point laser emitter.

[0083] Therefore, after unfolding, there is This refers to the transformation relationship we need to solve. Record the rotation angle θ of the motor at this point. x1 θ y1 The pixel coordinates (x1, y1) of the center of the projection point of the point laser beam on the calibration plane in the camera image, and the image depth z1 of the pixel where the center of the projection point of the point laser beam on the calibration plane is located.

[0084] Then continue adjusting the motor angle so that the position of the projection point of the point laser beam on the calibration plane changes but remains on the calibration plane and within the camera's view, and repeat this process until at least four sets of data are obtained [θ]. xi ,θ yi ,(x i ,y i (i = 1, 2, 3, ..., n, where n is the total number of data groups), and the laser beams of all data groups are not all collinear.

[0085] For each calibration data point in step S1, the three-dimensional coordinates P of the projection point of the point laser beam on the calibration plane in the actual camera coordinate system can be calculated based on the camera model. ci ,Right now:

[0086]

[0087] Among them, P ci Let be the three-dimensional coordinates of the projection point of the i-th laser beam onto the calibration plane in the actual camera coordinate system, where i is a positive integer, (x... i y i Let z be the pixel coordinates of the center of the projection point of the i-th laser beam on the calibration plane in the camera image. iLet K be the image depth of the pixel where the center of the projection point of the i-th point laser beam on the calibration plane is located, and K is the intrinsic parameter matrix.

[0088] Assume there also exists a virtual camera in the laser emitter coordinate system, with an intrinsic parameter matrix... The optical center coincides with the origin of the laser emitter's coordinate system. This allows us to determine the pixel coordinates of each point laser beam's projection onto the calibration plane within the image captured by the virtual camera.

[0089] (x' i ,y' i )=(100(1+tan(θ xi ),100(1-tan(θ yi ))) (6);

[0090] Among them, (x' i ,y' i Let be the pixel coordinates of the projection point of the i-th laser beam on the calibration plane within the virtual camera image, where i is a positive integer, (θ) xi θ yi ) represents the rotation angle of the motor corresponding to the projection point of the i-th laser beam on the calibration plane.

[0091] Therefore, the problem is transformed into: There is a virtual camera with known parameters on a robotic arm, and a fixed camera with known parameters outside the robotic arm. Several (at least four) point laser beams project onto a calibration plane, and their projection points are simultaneously within the images of both cameras. Given the coordinates of these projection points on the calibration plane in the camera coordinate system and their pixel coordinates in the virtual camera image, find the transformation matrix between the two camera coordinate systems. This is a classic PnP problem. The `solvePnP()` function in OpenCV can be used to find the translation and rotation vectors. After transformation, the translation vector and rotation matrix of the calibration result can be obtained. Thus, the calibration is complete, and the calibration matrix is ​​obtained.

[0092] Compared with existing technologies, this invention simplifies the calibration process, requiring only a minimum of four sets of points to complete the calibration, and improves calibration accuracy by simultaneously calculating the rotation matrix and translation vector through the PnP algorithm.

[0093] Example 2:

[0094] like Figure 2 As shown, this embodiment provides a robotic arm hand-eye calibration device based on a point laser emitter, comprising:

[0095] Control module, camera robotic arm, calibration board, laser emitter robotic arm, camera mechanism and laser emitter mechanism.

[0096] The control module is used to execute the steps of the robotic arm hand-eye calibration method based on a point laser emitter described in Embodiment 1. The control module is equipped with a development board for closed-loop control of the rotation angle of each motor in the robotic arm and data communication with the host computer, so that it can accurately position itself to the required pose, while achieving a certain degree of compatibility and scalability.

[0097] The camera robotic arm is used to receive a first control command to fix the camera mechanism; the camera mechanism includes a camera used to transmit and capture the projection of the point laser emitter onto the calibration plane in real time, and to obtain data for calculating the transformation matrix for calibration. The camera robotic arm has at least two rotational degrees of freedom and is used to control the position of the camera.

[0098] The calibration plate is used to receive a second control command to fix the calibration plane.

[0099] A laser emitter robotic arm is provided to fix the laser emitter mechanism, which includes a point laser emitter for emitting a point laser beam. The point laser emitter is a high-power type, achieving high-brightness laser indication, improving system robustness, and reducing the difficulty of image analysis. The laser emitter robotic arm has at least two rotational degrees of freedom and consists of multiple motors and connecting components, enabling the point laser emitter to rotate within its coordinate system. During rotation, the emitted point laser beam always passes through the center of the coordinate system, improving calibration accuracy.

[0100] Among them, the camera robotic arm and the laser emitter robotic arm enable flexible pose transformation of the camera and the point laser emitter, providing more diverse and accurate data for calibration.

[0101] As an optional implementation provided in this embodiment, the camera mechanism further includes:

[0102] A first motor and a second motor; the first motor and the second motor are placed vertically, and the first motor and the second motor enable the camera to move in two coordinate axis planes, and the rotation axes of the first motor and the second motor intersect at a point.

[0103] As an optional implementation provided in this embodiment, the laser emitter mechanism further includes:

[0104] A third motor and a fourth motor; the third motor and the fourth motor are placed vertically, and the third motor and the fourth motor enable the point laser emitter to move in two coordinate axis planes, and the rotation axes of the third motor and the fourth motor intersect at a point.

[0105] As an optional implementation method provided in this embodiment, the camera is a D435 depth camera, which adds depth measurement function to the basic camera. Based on the depth data, visual data can be further compensated to improve the calibration accuracy.

[0106] The coordinate system of the camera and the point laser emitter is defined as follows: Figure 3 As shown, the camera coordinate system O c Fixed to the camera, the camera robotic arm's end effector coordinate system O c_end Fixed to the end effector of the camera robotic arm, the camera robotic arm base coordinate system O c_base The laser emitter is fixed to the camera arm base. Its coordinate system is fixed relative to the camera arm base coordinate system and does not rotate with the laser emitter arm. It uses the rotation axis of the third motor as the y-axis and the rotation axis of the fourth motor as the x-axis. Through structural design, the two rotation axes can be made to intersect perpendicularly at point O. l Let point O be the origin of the laser emitter's coordinate system. It is stipulated that when both motors of the laser emitter's robotic arm rotate at 0 degrees, the laser beam emission direction coincides with the positive z-axis. Thus, regardless of the rotation angles of the two motors, the laser beam always passes through point O. l point.

[0107] The motor is controlled via a CAN bus, using a RoboMaster C board as the control device, which communicates with the computer via a UART serial port. The control algorithm is a cascaded PID control algorithm, which uses Hall effect sensors to detect the motor's rotation angle and performs closed-loop control through a cascaded PID system consisting of current loop, acceleration loop, speed loop, and position loop.

[0108] This invention can be calibrated on a gimbal mechanism with two rotational degrees of freedom, and can also be extended to robot structures with more degrees of freedom, making it widely applicable. At the same time, the required equipment is simple, with the main equipment being a control module, a camera robotic arm, a calibration board, a laser emitter robotic arm, a camera mechanism, and a laser emitter mechanism, without the need for additional equipment.

[0109] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for hand-eye calibration of a robotic arm based on a point laser emitter, characterized in that, include: The camera is calibrated to obtain the camera's intrinsic parameter matrix, and the first control command is determined based on the camera's intrinsic parameter matrix. The first control command is used to fix the camera; Select a calibration plane, obtain the homogeneous transformation matrix from the camera coordinate system to the calibration board coordinate system, and determine the second control command based on the homogeneous transformation matrix from the camera coordinate system to the calibration board coordinate system; the second control command is used to fix the calibration board. The calibration plane is the plane determined based on the calibration plate; The motor rotation angle when the point laser beam is projected onto the calibration plane and the pixel coordinates of the projection point of the point laser beam on the calibration plane within the actual camera image are obtained; the point laser beam is the laser beam emitted by the point laser emitter; the motor is used to control the rotation of the point laser emitter; Based on the motor rotation angle, the pixel coordinates of the projection point of the point laser beam on the calibration plane within the virtual camera image are calculated; the virtual camera is an imaginary camera with its optical center at the origin of the point laser emitter's coordinate system, oriented coinciding with the z-axis of the point laser emitter, and its intrinsic parameter matrix is ​​fixed. Based on the pixel coordinates of the projection point of the point laser beam on the calibration plane within the actual camera image, the three-dimensional coordinates of the projection point of the point laser beam on the calibration plane in the actual camera coordinate system are calculated; the actual camera coordinate system is a coordinate system established based on the camera. The PnP algorithm is used to calculate the transformation matrix from the actual camera coordinate system to the virtual camera coordinate system; the virtual camera coordinate system is a coordinate system established based on the point laser emitter.

2. The robotic arm hand-eye calibration method based on a point laser emitter according to claim 1, characterized in that, The formula for calculating the pixel coordinates of the projection point of the point laser beam on the calibration plane within the virtual camera image is as follows: (x′ i ,and' i )=(100(1+tan(θ xi ),100(1-tan(θ yi ))); Among them, (x′ i ,y′ i Let be the pixel coordinates of the projection point of the i-th laser beam on the calibration plane within the virtual camera image, where i is a positive integer, (θ) xi θ yi ) represents the rotation angle of the motor corresponding to the projection point of the i-th laser beam on the calibration plane.

3. The robotic arm hand-eye calibration method based on a point laser emitter according to claim 1, characterized in that, The formula for calculating the three-dimensional coordinates of the projection point of the point laser beam on the calibration plane in the actual camera coordinate system is as follows: Among them, P ci Let be the three-dimensional coordinates of the projection point of the i-th laser beam onto the calibration plane in the actual camera coordinate system, where i is a positive integer, (x... i y i Let z be the pixel coordinates of the center of the projection point of the i-th laser beam on the calibration plane in the camera image. i Let K be the image depth of the pixel where the center of the projection point of the i-th point laser beam on the calibration plane is located, and K is the intrinsic parameter matrix.

4. The robotic arm hand-eye calibration method based on a point laser emitter according to claim 1, characterized in that, The transformation relationship of the transformation matrix from the actual camera coordinate system to the virtual camera coordinate system is as follows: in, Let be the homogeneous transformation matrix from the camera coordinate system to the point laser emitter coordinate system. The rotation matrix for the calibration results. P is the translation vector of the calibration result. c Let P be the coordinates of the projection point of the point laser beam onto the calibration plane in the camera coordinate system. l The coordinates of the projection point of the point laser beam onto the calibration plane in the coordinate system of the point laser emitter.

5. The robotic arm hand-eye calibration method based on a point laser emitter according to claim 1, characterized in that, The number of projection points of the point laser beam on the calibration plane is greater than or equal to 4, and all point laser beams are not collinear.

6. The robotic arm hand-eye calibration method based on a point laser emitter according to claim 1, characterized in that, Before calculating the three-dimensional coordinates of the projection point of the point laser beam on the calibration plane in the actual camera coordinate system based on the pixel coordinates of the projection point of the point laser beam on the calibration plane in the actual camera image, the process further includes: Obtaining the pixel coordinates of the projection point of the point laser beam onto the calibration plane, specifically including: Convert the camera image from RGB space to HSV space using OpenCV; In the HSV space, pixels containing the projection points of the point laser beam on the calibration plane are obtained by setting a threshold. Based on the pixel points of the projection points of the point laser beam on the calibration plane, the contour of the projection points of the point laser beam on the calibration plane is obtained through the findContours() function in OpenCV. The centroid coordinates of the projection point contour of the point laser beam on the calibration plane can be obtained by using the moments() function of OpenCV. The centroid coordinates of the projection point contour of the point laser beam on the calibration plane are used as the pixel coordinates of the projection point of the point laser beam on the calibration plane.

7. A robotic arm hand-eye calibration device based on a point laser emitter, characterized in that, include: Control module, camera robotic arm, calibration board, laser emitter robotic arm, camera mechanism and laser emitter mechanism; The control module is used to execute the steps of the robotic arm hand-eye calibration method based on a point laser emitter as described in any one of claims 1-6; The camera robotic arm is used to receive a first control command to fix the camera mechanism; the camera mechanism includes a camera; The calibration plate is used to receive a second control command to fix the calibration plane; A laser emitter robotic arm is used to fix the laser emitter mechanism; the laser emitter mechanism includes a point laser emitter; the point laser emitter is used to emit a point laser beam.

8. The robotic arm hand-eye calibration device based on a point laser emitter according to claim 7, characterized in that, The camera mechanism also includes: A first motor and a second motor; the first motor and the second motor are placed vertically, and the first motor and the second motor enable the camera to move in two coordinate axis planes, and the rotation axes of the first motor and the second motor intersect at a point.

9. The robotic arm hand-eye calibration device based on a point laser emitter according to claim 7, characterized in that, The laser emitter mechanism also includes: The third motor and the fourth motor are placed vertically, and the third motor and the fourth motor enable the point laser emitter to move in two coordinate axis planes, and the rotation axes of the third motor and the fourth motor intersect at one point; The coordinate system of the point laser emitter is fixed relative to the coordinate system of the camera robotic arm base and does not rotate with the rotation of the laser emitter robotic arm.

10. A robotic arm hand-eye calibration device based on a point laser emitter according to claim 7, characterized in that, The camera is a depth camera.

Images