Visual calibration method and system for calibration-free automatic mechanical arm

Abstract

The invention relates to a visual calibration method and system for an uncalibrated automatic mechanical arm, the method comprises two calibration modes of'eye on hand 'and'eye outside hand', the'eye on hand 'calculates and determines a hand-eye matrix through a camera and a marker mounted at the tail end of the mechanical arm so as to obtain pose transformation between the tail end of the mechanical arm and a flange in real time. In the calibration of'eyes outside hands', a global camera and a marker installed on the mechanical arm are utilized to determine the position and posture relation between the tail end of the mechanical arm and a target object through coordinate transformation, and therefore accurate calibration of the mechanical arm is achieved. The calibration method is suitable for scenes where D-H parameter precision is poor, a view blind area exists, and real-time deviation correction needs to be conducted through a visual servo system during work of the mechanical arm.

Description

Technical Field

[0001] The present invention relates to the field of vision calibration, and in particular to a vision calibration method and system for an uncalibrated automated robotic arm. Background Art

[0002] Robotic arms are widely used in aerospace, medical, hazardous operations and other fields due to their flexibility, efficiency and safety. In order to meet diverse needs, some performance of the robotic arm, such as repeatability and rigidity, may be sacrificed, resulting in large errors in the DH parameter calibration of the robotic arm. Currently, commonly used robotic arms can ensure safe locking and sealing after docking with the flange. The end rod has good rigidity and is used to install visual equipment. In addition, some axes may be affected by weight and produce small deformations, resulting in unstable DH parameters. The kinematic forward solution matrix cannot accurately represent the actual position of the end. Therefore, real-time correction is required through visual servoing, especially when approaching the flange position. A global camera is needed to reduce the impact of blind spots. The calibration of this type of robotic arm has become a problem that needs to be solved urgently. Summary of the invention

[0003] The purpose of the present invention is to provide a visual calibration method and system for an uncalibrated automated robotic arm, so as to solve the problem of visual system calibration of the uncalibrated automated robotic arm and improve the calibration accuracy of the robotic arm.

[0004] The technical solution to achieve the purpose of the present invention is:

[0005] A visual calibration method for an uncalibrated automated robotic arm, comprising:

[0006] Eye-on-hand calibration: when the uncalibrated automated robotic arm is in a blind spot without field of view, the transformation matrix of the target flange posture converted to the robotic arm end coordinate system is obtained for calibration. The objects involved in the calibration include the camera on the arm, the manifold flange, and the fixed Aruco tag set on the manifold flange. The camera installation position is relatively fixed to the end of the robotic arm.

[0007] Eye-out-of-hand calibration, when the uncalibrated automated robotic arm is in the blind spot of the field of view, obtains the transformation matrix of the target flange posture converted to the robotic arm end coordinate system for calibration. The objects involved in the eye-out-of-hand calibration include the global camera, the manifold flange, the fixed Aruco tag set on the manifold flange, and the fixed Aruco tag set on the robotic arm. The global camera is fixed on the frame.

[0008] Furthermore, the eye-on-hand calibration specifically includes:

[0009] Step 1-1, determine the hand-eye matrix. When only the flange recognition camera is used for positioning, obtain the transformation matrix T=T1*T2 between the flange and the robot end coordinate system, where T1 is the hand-eye matrix to be calibrated, and T2 is the transformation matrix between the camera and the flange coordinate system;

[0010] Step 1-2, using a camera to capture images of the Aruco tags within the field of view, including an image of the Aruco tag f concentric with the flange and an image of a marker f1 rigidly connected to the flange;

[0011] Step 1-3, obtain the transformation matrix T3 between the camera and flange coordinate systems, and the transformation matrix T4 between the camera and f1 coordinate systems, and obtain the transformation matrix T5 between the flange and marker coordinate systems through coordinate transformation;

[0012] Step 1-4, using the locking device at the end of the robot arm to make the end of the robot arm concentric and coplanar with the flange surface, that is, the coordinate system of the end of the robot arm coincides with the coordinate system of the flange;

[0013] Step 1-5, the camera captures the image of the marker f1 within the field of view, and decodes it to obtain the transformation matrix T6 between the coordinate system of the camera and the marker f1 at this time. Through coordinate transformation, the transformation matrix between the end of the robot arm and the camera coordinate system, that is, the required hand-eye matrix T1, is obtained to complete the eye-on-hand calibration.

[0014] Furthermore, in step 1-3, the transformation matrix T5=T4 -1 *T3.

[0015] Furthermore, the hand-eye matrix T1=T5 obtained in step 1-5 -1 *T6 -1 .

[0016] Furthermore, the eye-out-of-hand calibration specifically includes:

[0017] Step 2-1, determine the marker-manipulator calibration matrix, introduce the global camera and marker f2 to participate in the calibration, and obtain the transformation matrix T7=T8*T9 between the flange and the end coordinate system of the manipulator -1 *T 10 , where T8 is the calibration matrix of the marker-robot arm to be calibrated, T9 is the transformation matrix between the coordinate system of the camera and the marker f2 on the arm, and T 10 is the transformation matrix between the global camera and flange coordinate systems;

[0018] Step 2-2, using a global camera to capture images of the ARUCO tags within the field of view, including an image f concentric with the flange and an image f1 of a marker rigidly connected to the flange;

[0019] Step 2-3, obtaining the transformation matrix T3 between the camera and flange coordinate systems, and the transformation matrix T4 between the camera and the marker f1 coordinate system, and obtaining the transformation matrix T5 between the flange and marker coordinate systems through coordinate transformation;

[0020] Step 2-4, using the locking device at the end of the robot arm to make the end of the robot arm concentric and coplanar with the flange surface, that is, the coordinate system of the end of the robot arm coincides with the coordinate system of the flange;

[0021] Step 2-5: The global camera captures the images of the landmarks f1 and f2 within the field of view, and decodes them to obtain the transformation matrix T between the global camera and the f1 coordinate system and the f2 coordinate system at this time. 11 , T9, obtain the calibration matrix T8 through coordinate transformation to complete the eye-outside-hand calibration.

[0022] Furthermore, in step 2-3, the transformation matrix T5=T4 -1 *T3.

[0023] Furthermore, the calibration matrix T8 in step 5 is (T5) -1 *(T 11 ) -1 *T9.

[0024] A visual calibration system for a calibration-free automated robotic arm, comprising:

[0025] The eye-on-hand calibration unit obtains the transformation matrix of the target flange posture converted to the end coordinate system of the robotic arm for calibration when the uncalibrated automated robotic arm is in a blind spot without vision. The objects involved in the calibration include the camera on the arm, the manifold flange, and the fixed Aruco tag set on the manifold flange. The camera installation position is relatively fixed to the end of the robotic arm.

[0026] The eye-in-hand calibration unit obtains the transformation matrix of the target flange posture converted to the robot end coordinate system for calibration when the uncalibrated automated robot arm is in the blind spot of the field of view. The objects involved in the eye-in-hand calibration include the global camera, the manifold flange, the fixed Aruco label set on the manifold flange, and the fixed Aruco label set on the robot arm. The global camera is fixed on the frame.

[0027] A visual calibration device for an uncalibrated automated robotic arm comprises: a memory, a processor and a computer program stored in the memory, wherein the processor implements the steps of the visual calibration method according to any one of claims 1 to 7 when executing the computer program.

[0028] A computer storage medium stores an executable program, and the executable program is executed by a processor to implement the steps of the visual calibration method.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows: the visual calibration method for an uncalibrated automated robotic arm of the present invention, including the "eyes on the hand" calibration relying on the camera on the arm and the "eyes outside the hand" calibration relying on the global camera, can realize the visual calibration of an uncalibrated automated robotic arm; the present invention adopts aruco tags as markers, and does not need to obtain the robotic arm posture through solution to participate in the calibration process, and can well ensure the calibration accuracy; the present invention is suitable for scenarios where the DH parameter accuracy is poor and there are blind spots in the field of view, and the operation of the robotic arm requires real-time correction through a visual servo system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The figure is a flow chart of the visual calibration method of the present invention.

[0031] Figure 2 This is a schematic diagram of the coordinate conversion of the hand-eye calibration of the camera on the arm in the present invention, where: Figure 2 (a) is a schematic diagram of the relative position of the flange and the marker f1 obtained by the camera on the arm. Figure 2 (b) in the figure is a schematic diagram of obtaining the hand-eye matrix.

[0032] Figure 3 Schematic diagram of global camera hand-eye calibration coordinate transformation in the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention is further explained and illustrated in detail below based on the accompanying drawings and specific implementation methods.

[0034] Combination Figure 1 The present invention proposes a visual calibration method for an uncalibrated automated robotic arm, with the aim of solving the problem that the uncalibrated automated robotic arm has a large DH parameter error and cannot be calibrated directly through the cooperation of the robotic arm's kinematic model and the visual system. Without the need for the robotic arm to participate, the hand-eye calibration is performed relying on the visual system, including calibration based only on visual methods for two situations: eye on hand and eye outside the hand.

[0035] The eye-on-hand calibration is aimed at the situation that when the uncalibrated automated robotic arm is in a blind spot without vision, it is necessary to directly obtain the target flange posture through the visual system and convert it into the robotic arm end coordinate system. The objects involved in the calibration are the camera on the arm, the manifold flange, and the fixed Aruco tags set on the manifold flange; combined with Figure 2 , specifically including the following steps:

[0036] Step 1: Determine the hand-eye matrix. When only the flange recognition camera is used for positioning, it is necessary to obtain the transformation matrix T=T1*T2 between the flange and the robot end coordinate system in real time, where T1 is the hand-eye matrix that needs to be calibrated, and T2 is the transformation matrix between the camera and flange coordinate systems.

[0037] Step 2: The camera captures images of the Aruco tags within the field of view, including the image of the Aruco tag f concentric with the flange and the image of the Aruco tag f1 rigidly connected to the flange;

[0038] Step 3, obtain the transformation matrix T3 between the camera and flange coordinate systems, and the transformation matrix T4 between the camera and f1 coordinate systems, and obtain the transformation matrix T5 between the flange and marker coordinate systems through coordinate transformation;

[0039] T5=T4 -1 *T3

[0040] Step 4: Use the locking device at the end of the robot arm to make the end of the robot arm concentric and coplanar with the flange surface, that is, the coordinate system of the end of the robot arm coincides with the coordinate system of the flange;

[0041] Step 5, obtain the transformation matrix T3 between the camera and flange coordinate systems, and the transformation matrix T6 between the camera and the marker f1 coordinate systems after the end of the robotic arm is attached to the flange surface, and obtain the transformation matrix between the end of the robotic arm and the camera coordinate system through coordinate transformation, that is, the required hand-eye matrix T1;

[0042] T1=T5 -1 *T6 -1

[0043] The global camera hand-eye calibration aims to solve the problem of blind spots in the vision system of the uncalibrated automated robotic arm when acquiring target information in real time. The target flange position needs to be indirectly obtained through the visual system and converted to the robotic arm end coordinate system. The objects involved in the calibration are the camera on the arm, the manifold flange, the fixed Aruco label set on the manifold flange, and the fixed Aruco label f2 set on the robotic arm. The eye-out-of-hand calibration also only requires the participation of the visual system, combined with Figure 3 , comprising the following steps;

[0044] Step 1: Determine the marker-robot calibration matrix. When only using the global camera for positioning, since it is impossible to directly obtain the transformation matrix between the flange and the end coordinate system of the robot in real time, it is necessary to introduce a global camera for subsequent guidance. Similarly, a marker f2 is introduced to participate in the calibration of the visual system; the transformation matrix between the flange and the end coordinate system of the robot is obtained. Transformation matrix T7 = T8*T9 -1 *T 10 , where T8 is the calibration matrix of the marker-robot arm to be calibrated, T9 is the transformation matrix between the coordinate system of the camera and the marker f2 on the arm, and T 10 is the transformation matrix between the camera and flange coordinate systems;

[0045] Step 2: The camera captures the image of the Aruco tag within the field of view, including the image f concentric with the flange and the image f1 rigidly connected to the flange;

[0046] Step 3, obtain the transformation matrix T3 between the camera and flange coordinate systems, and the transformation matrix T4 between the camera and f1 coordinate systems, and obtain the transformation matrix T5 between the flange and marker coordinate systems through coordinate transformation;

[0047] T5=T4 -1 *T3

[0048] Step 4: Use the locking device at the end of the robot arm to make the end of the robot arm concentric and coplanar with the flange surface, that is, the coordinate system of the end of the robot arm coincides with the coordinate system of the flange;

[0049] Step 5: The global camera captures the images of the markers f1 and f2 within the field of view, and decodes them to obtain the transformation matrix T between the global camera and the marker f1 coordinate system after the end of the robot arm is attached to the flange surface. 11 And the transformation matrix T9 between the camera and the f2 coordinate system, the calibration matrix T8 is obtained by coordinate transformation = (T5) -1 *(T 11 ) -1 *T9.

[0050] A visual calibration system for a calibration-free automated robotic arm, comprising:

[0051] The eye-on-hand calibration unit obtains the transformation matrix of the target flange posture converted to the end coordinate system of the robotic arm for calibration when the uncalibrated automated robotic arm is in a blind spot without vision. The objects involved in the calibration include the camera on the arm, the manifold flange, and the fixed Aruco tag set on the manifold flange. The camera installation position is relatively fixed to the end of the robotic arm.

[0052] The eye-in-hand calibration unit obtains the transformation matrix of the target flange posture converted to the robot end coordinate system for calibration when the uncalibrated automated robot arm is in the blind spot of the field of view. The objects involved in the eye-in-hand calibration include the global camera, the manifold flange, the fixed Aruco label set on the manifold flange, and the fixed Aruco label set on the robot arm. The global camera is fixed on the frame.

[0053] A visual calibration device for an uncalibrated automated robotic arm comprises: a memory, a processor and a computer program stored in the memory, wherein the processor implements the steps of the visual calibration method according to any one of claims 1 to 7 when executing the computer program.

[0054] A computer storage medium stores an executable program, and the executable program is executed by a processor to implement the steps of the visual calibration method.

[0055] The present invention does not involve parts that are the same as the prior art or can be implemented by using the prior art. The visual calibration method for the calibration-free automated robotic arm of the present invention includes the "eyes on the hand" calibration relying on the camera on the arm and the "eyes outside the hand" calibration relying on the global camera, which can realize the visual calibration of the calibration-free automated robotic arm. In addition, the present invention uses aruco tags as markers, and does not need to obtain the robotic arm posture through solution to participate in the calibration process, which can well ensure the calibration accuracy.

[0056] The above are only preferred embodiments of the present application, and the present invention is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the invention should be considered to be included in the scope of protection of the invention.

Claims

1. A visual calibration method for an uncalibrated automated robotic arm, characterized in that: include: Eye-on-hand calibration: when the uncalibrated automated robotic arm is in a blind spot without field of view, the transformation matrix of the target flange posture converted to the robotic arm end coordinate system is obtained for calibration. The objects involved in the calibration include the camera on the arm, the manifold flange, and the fixed Aruco tag set on the manifold flange. The camera installation position is relatively fixed to the end of the robotic arm. Eye-out-of-hand calibration, when the uncalibrated automated robotic arm is in the blind spot of the field of view, obtains the transformation matrix of the target flange posture converted to the robotic arm end coordinate system for calibration. The objects involved in the eye-out-of-hand calibration include the global camera, the manifold flange, the fixed Aruco tag set on the manifold flange, and the fixed Aruco tag set on the robotic arm. The global camera is fixed on the frame.

2. The visual calibration method of a calibration-free automated robotic arm according to claim 1, characterized in that: Eye-on-hand calibration specifically includes: Step 1-1, determine the hand-eye matrix. When only the flange recognition camera is used for positioning, obtain the transformation matrix T=T1*T2 between the flange and the robot end coordinate system, where T1 is the hand-eye matrix to be calibrated, and T2 is the transformation matrix between the camera and the flange coordinate system; Step 1-2, using a camera to capture images of the Aruco tags within the field of view, including an image of the Aruco tag f concentric with the flange and an image of a marker f1 rigidly connected to the flange; Step 1-3, obtain the transformation matrix T3 between the camera and flange coordinate systems, and the transformation matrix T4 between the camera and f1 coordinate systems, and obtain the transformation matrix T5 between the flange and marker coordinate systems through coordinate transformation; Step 1-4, using the locking device at the end of the robot arm to make the end of the robot arm concentric and coplanar with the flange surface, that is, the coordinate system of the end of the robot arm coincides with the coordinate system of the flange; Step 1-5, the camera captures the image of the marker f1 within the field of view, and decodes it to obtain the transformation matrix T6 between the coordinate system of the camera and the marker f1 at this time. Through coordinate transformation, the transformation matrix between the end of the robot arm and the camera coordinate system, that is, the required hand-eye matrix T1, is obtained to complete the eye-on-hand calibration.

3. The visual calibration method of a calibration-free automated robotic arm according to claim 2, characterized in that: In step 1-3, the transformation matrix T5=T4 -1 *T3.

4. The visual calibration method of a calibration-free automated robotic arm according to claim 3, characterized in that: The hand-eye matrix T1=T5 obtained in step 1-5 -1 *T6 -1 .

5. The visual calibration method of a calibration-free automated robotic arm according to claim 1, characterized in that: Eye-out-of-hand calibration specifically includes: Step 2-1, determine the marker-manipulator calibration matrix, introduce the global camera and marker f2 to participate in the calibration, and obtain the transformation matrix T7=T8*T9 between the flange and the end coordinate system of the manipulator -1 *T 10 , where T8 is the calibration matrix of the marker-robot arm to be calibrated, T9 is the transformation matrix between the coordinate system of the camera and the marker f2 on the arm, and T 10 is the transformation matrix between the global camera and flange coordinate systems; Step 2-2, using a global camera to capture images of the ARUCO tags within the field of view, including an image f concentric with the flange and an image f1 of a marker rigidly connected to the flange; Step 2-3, obtaining the transformation matrix T3 between the camera and flange coordinate systems, and the transformation matrix T4 between the camera and the marker f1 coordinate system, and obtaining the transformation matrix T5 between the flange and marker coordinate systems through coordinate transformation; Step 2-4, using the locking device at the end of the robot arm to make the end of the robot arm concentric and coplanar with the flange surface, that is, the coordinate system of the end of the robot arm coincides with the coordinate system of the flange; Step 2-5: The global camera captures the images of the landmarks f1 and f2 within the field of view, and decodes them to obtain the transformation matrix T between the global camera and the f1 coordinate system and the f2 coordinate system at this time. 11 , T9, obtain the calibration matrix T8 through coordinate transformation to complete the eye-outside-hand calibration.

6. The visual calibration method of a calibration-free automated robotic arm according to claim 5, characterized in that: In step 2-3, the transformation matrix T5=T4 -1 *T3.

7. The visual calibration method of a calibration-free automated robotic arm according to claim 6, characterized in that: The calibration matrix T8 in step 5 is (T5) -1 *(T 11 ) -1 *T9.

8. A visual calibration system for an uncalibrated automated robotic arm based on the method of any one of claims 1 to 7, characterized in that: include: The eye-on-hand calibration unit obtains the transformation matrix of the target flange posture converted to the end coordinate system of the robotic arm for calibration when the uncalibrated automated robotic arm is in a blind spot without vision. The objects involved in the calibration include the camera on the arm, the manifold flange, and the fixed Aruco tag set on the manifold flange. The camera installation position is relatively fixed to the end of the robotic arm. The eye-in-hand calibration unit obtains the transformation matrix of the target flange posture converted to the robot end coordinate system for calibration when the uncalibrated automated robot arm is in the blind spot of the field of view. The objects involved in the eye-in-hand calibration include the global camera, the manifold flange, the fixed Aruco label set on the manifold flange, and the fixed Aruco label set on the robot arm. The global camera is fixed on the frame.

9. A visual calibration device for a non-calibrated automated robotic arm, characterized in that: include: A memory, a processor and a computer program stored in the memory, wherein the processor implements the steps of the visual calibration method according to any one of claims 1 to 7 when executing the computer program.

10. A computer storage medium, characterized in that: The computer storage medium stores an executable program, and the executable program is executed by a processor to implement the steps of the visual calibration method according to any one of claims 1 to 7.

Images