Mechanical arm teaching method based on teaching gun

Abstract

The invention relates to a mechanical arm teaching method based on a teaching gun. The mechanical arm teaching method comprises the following steps that a CAD model of the teaching gun is obtained; based on parameters in the CAD model, a conversion relation from the needle tip to a 2D calibration plate is obtained, a conversion relation from the needle tip to a 3D calibration object is obtained, and the pose of the camera in the base coordinate system is obtained; when the teaching gun is moved to a designated position i, the pose of the 2D calibration plate in a camera coordinate system is obtained, and meanwhile, the pose of the 3D calibration object in the camera coordinate system is obtained; and respectively calculating the poses from the needle point to the base coordinate system through the 2D calibration plate and the 3D calibration object, and when the relative offset of the two poses is smaller than a preset threshold b, taking the average pose as the final pose estimation. Compared with the prior art, the design and application of the teaching gun, the dual calibration technology, the convenient teaching process and the flexible teaching mode are innovated, so that the teaching method has higher efficiency, lower operation difficulty and wider application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic arms, and in particular to a robotic arm teaching method based on a teaching gun. Background Art

[0002] In robotic arm operation, teaching is a common motion control method used to identify key points along a movement path. For efficiency reasons, collaborative robotic arms offer a manual drag-and-drop method for teaching positioning. However, since industrial robotic arms cannot reach the teaching point by direct dragging, the teaching process requires continuous operation of the teach pendant, which is extremely time-consuming and labor-intensive.

[0003] To this end, there is an urgent need for a portable teaching gun to conveniently tell the robotic arm the target position and complete the teaching positioning. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a robotic arm teaching method based on a teaching gun. Innovations have been made in the design and application of the teaching gun, dual calibration technology, convenient teaching process, and flexible teaching methods, so that the teaching method has higher efficiency, lower operating difficulty and broader application prospects.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The present invention provides a teaching method for a robotic arm based on a teaching gun, comprising the following steps:

[0007] S1: Obtain a CAD model of a teaching gun, wherein the teaching gun is provided with a 2D calibration plate, a 3D calibration object, and a needle tip;

[0008] S2: Based on the parameters in the CAD model, the transformation relationship T from the needle tip to the 2D calibration plate is measured. point2board , and at the same time obtain the transformation relationship T from the needle tip to the 3D calibration object pointobj , and obtain the camera's pose T in the base coordinate system eye2base ;

[0009] S3: When the teaching gun is moved to the specified position i, the pose T of the 2D calibration plate in the camera coordinate system is obtained. board2eye , and obtain the pose T of the 3D calibration object in the camera coordinate system obj2eye ;

[0010] S4: Calculate the pose of the needle tip to the base coordinate system using the 2D calibration plate and the 3D calibration object. When the relative offset of the two poses is less than the preset threshold b, take the average pose as the final pose estimate T point2base ;

[0011] S5: Using the obtained T point2base , control the robotic arm to move to the specified position and complete the teaching process.

[0012] Furthermore, in S1, the 2D calibration plate is selected from one of a center plate, a checkerboard, and ArUco.

[0013] Furthermore, in S1, the 3D calibration object is a geometric body with a preset three-dimensional shape, and the CAD model of the 3D calibration object can be integrated with the CAD model of the teaching gun into the same model.

[0014] Furthermore, in S2, the conversion relationship from the needle tip to the 2D calibration plate is: a mapping relationship from the coordinates of the needle tip of the teaching gun to each calibration point on the 2D calibration plate in the same coordinate system.

[0015] Furthermore, in S2, the conversion relationship from the needle tip to the 3D calibration object is: a mapping relationship from the coordinates of the needle tip of the teaching gun to each calibration point on the 3D calibration object.

[0016] Furthermore, in S3, the needle tip of the teaching gun is moved to the designated position i by one of manual dragging, remote control, and programming instructions.

[0017] Furthermore, in S3, the position T of the 2D calibration plate in the camera coordinate system is obtained through the 2D calibration plate recognition algorithm. board2eye ;

[0018] The 2D calibration plate recognition algorithm includes image preprocessing, feature extraction, calibration plate recognition, camera parameter correction, and pose estimation.

[0019] Furthermore, in S3, the pose T of the 3D calibration object in the camera coordinate system is obtained by the ICP alignment algorithm. obj2eye .

[0020] Furthermore, in S4, the tip pose estimated by the 2D calibration plate is The pose estimation obtained by 3D calibration object is When the offset between the two is less than the threshold b, the average is taken as the final pose estimate T point2base ,in:

[0021]

[0022]

[0023]

[0024] Furthermore, in S5, when the teaching stage i is greater than 1, S3 to S5 are repeated until the teaching of each stage is completed.

[0025] Compared with the prior art, the present invention has the following technical advantages:

[0026] 1) Design and Application of a Teaching Gun: Traditionally, the teaching process for robotic arms is time-consuming and labor-intensive, requiring constant manipulation via a teach pendant. This invention introduces a teaching gun that combines a 2D calibration plate and a 3D calibration object, allowing the user to intuitively teach the robotic arm the target pose, greatly simplifying the teaching process.

[0027] 2) Dual Calibration: This invention utilizes not only a 2D calibration plate but also a 3D calibration object. This dual calibration technique improves the accuracy and robustness of pose estimation. The pose of the needle tip is calculated using both the 2D calibration plate and the 3D calibration object. The average pose is taken when the relative offset between the two is less than a preset threshold. This method reduces errors and improves teaching accuracy.

[0028] 3) Flexible teaching methods: The teaching gun can be moved to a specified position in a variety of ways, such as manual dragging, remote control, or programmed instructions. This flexibility allows users to choose the appropriate teaching method based on specific application scenarios and needs.

[0029] 4) Wide range of applications: Since the teaching method of the present invention is efficient, convenient and flexible, it can be widely used in various scenarios requiring robotic arm teaching, such as industrial production, automated assembly, robot programming, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a structural diagram of the robot arm teaching system based on the teaching gun in the present invention.

[0031] In the figure: 1. Camera, 2. Robotic arm, 3. 2D calibration plate, 4. 2D calibration plate, 5. 3D calibration object. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Component models, material names, connection structures, control methods, algorithms, and other features not explicitly described in this technical solution are considered common technical features disclosed in the prior art.

[0033] Example 1

[0034] The present invention proposes a teaching method for a robotic arm based on a teaching gun. The method uses both 2D alignment and 3D alignment in the alignment stage. Figure 1 It is a structural diagram of the robot arm teaching system based on the teaching gun in the present invention.

[0035] Figure 1 The various components involved in this system are: camera 1, robotic arm 2, 2D calibration plate 3, 2D calibration plate 4, and 3D calibration object 5.

[0036] The specific teaching steps are as follows:

[0037] Get the CAD design model of the teaching gun. The teaching gun is equipped with a 2D calibration board (such as a center plate, checkerboard, ArUco, etc.) and a 3D calibration object (obj). The conversion relationship T from the needle tip (point) to the 2D calibration board is obtained by measuring the parameters in the CAD model. point2board Similarly, the transformation relationship T from the needle tip to the 3D calibration object is obtained through the parameters in the CAD model. point2obj .

[0038] The CAD model accurately captures the geometric parameters and relative positions of the teaching gun and its components (including the 2D calibration plate, 3D calibration object, and stylus tip). These precise parameters provide a solid foundation for subsequent calculations of the stylus tip's position in the base coordinate system, thereby improving teaching accuracy.

[0039] In traditional teaching systems, a complex calibration process may be required to determine the position and orientation of the tool or sensor. In the present invention, the conversion relationship is directly obtained through the CAD model, which greatly simplifies the calibration steps and reduces the errors introduced by inaccurate calibration. Using a CAD model means that the precise size and position of all components are taken into account during the design phase, which helps to reduce system instability caused by manufacturing tolerances or assembly errors. This robustness is particularly important when the system needs to operate in different working environments or configurations. The CAD model can be easily modified and updated to adapt to different teaching gun designs or calibration object configurations. This flexibility enables this method to adapt to a variety of application scenarios and different user needs. The conversion relationship defined by the CAD model can be more easily integrated with other systems, such as machine vision systems, robotic control systems, etc. In addition, it also helps to standardize the teaching process, thereby improving work efficiency and reducing training costs.

[0040] Use the eye-outside-hand / on-hand calibration to get the camera's pose T in the base coordinate system. eye2base The camera (eye) is calibrated using the eye outside the hand / on the hand to obtain the camera's position in the base coordinate system (base).

[0041] When the teaching gun is moved to the specified position i, the 2D calibration plate recognition algorithm is used to obtain the position and pose of the calibration plate in the camera coordinate system. Similarly, the ICP alignment algorithm is used to obtain the position and pose of the 3D calibration object in the camera coordinate system.

[0042] The specific implementation of the 2D calibration plate recognition algorithm can include the following steps: Image preprocessing: Preprocess the image captured by the camera, including operations such as denoising, grayscale conversion, and binarization, to highlight the feature points or patterns on the calibration plate. Feature extraction: Use image processing techniques to extract feature points on the calibration plate. These feature points can be center points, corner points, edges, etc., depending on the type of calibration plate (such as a center plate, checkerboard, ArUco, etc.). For example, for a center plate, the Hough transform can be used to detect circles; for a checkerboard, a corner detection algorithm (such as Harris corner detection) can be used to extract corners. Calibration plate recognition: Based on the extracted feature points, the position and posture of the calibration plate in the image are identified. This can be achieved through methods such as template matching and contour analysis. If a calibration plate with a specific pattern (such as an ArUco marker) is used, the encoded information of this pattern can be used to identify the calibration plate. Camera parameter correction: After identifying the calibration plate, the camera's internal parameters (such as focal length, principal point coordinates, distortion coefficient, etc.) can be corrected using the known calibration plate size and the detected feature point positions. This step usually requires the use of a camera calibration algorithm, such as Zhang's Method. Pose estimation: Based on the corrected camera parameters and the identified calibration plate position, the pose of the calibration plate (and the teaching gun needle tip fixed to it) in the camera coordinate system is estimated. This can be achieved by solving the perspective transformation (Perspective-n-Point, PnP) problem. Common PnP solution methods include EPnP (Efficient PnP), DLS (Direct Least Squares), etc.

[0043] When the teaching gun is moved to the specified position i, the 2D calibration plate recognition algorithm is used to obtain the position T of the calibration plate in the camera coordinate system. board2eye Similarly, the ICP alignment algorithm is used to obtain the pose T of the 3D calibration object in the camera coordinate system. obj2eye .

[0044] This method combines a 2D calibration plate and a 3D calibration object to estimate the pose of the teaching gun. The 2D calibration plate recognition algorithm is used to obtain the pose of the calibration plate in the camera coordinate system, while the ICP (Iterative Closest Point) alignment algorithm is used to obtain the pose of the 3D calibration object in the camera coordinate system. These two methods complement each other and improve the robustness and accuracy of pose estimation.

[0045] By combining camera calibration with visual recognition algorithms, this invention simplifies the steps that require manual input or complex calculations in the traditional teaching process. The operator only needs to move the teaching gun to the target position, and the system will automatically complete the subsequent posture estimation and robotic arm control, greatly reducing the difficulty of operation and training costs.

[0046] The method of the present invention is not only applicable to specific robot arm and camera configurations, but can also be expanded and adapted as needed. By replacing different types of calibration plates or 3D calibration objects, or adjusting the relative position of the camera and robot arm, it can be adapted to different application scenarios and work requirements.

[0047] The pose of the needle tip of the teaching gun to the base coordinate system is calculated by the 2D calibration plate and the 3D calibration object. Among them, the pose of the needle tip obtained by the 2D calibration plate is estimated to be The pose estimation obtained by 3D calibration object is When the offset between the two is less than the threshold b, the average is taken as the final pose estimate T point2base .

[0048]

[0049]

[0050]

[0051] Using the obtained T point2base , control the robotic arm to move to the specified position.

[0052] By using two different dimensional calibration tools—a 2D calibration plate and a 3D calibration object—the pose information of the teaching gun tip can be acquired from multiple angles. These two calibration methods are independent of each other, each providing a set of pose estimates. When the offset between the two pose estimates is less than a preset threshold, the two pose estimates are consistent, and averaging them can further improve pose estimation accuracy. Setting an offset threshold to determine whether the two pose estimates are consistent and, based on this, to determine whether to average the final pose estimate simplifies the decision-making process. This simple decision rule enables the system to quickly and efficiently process the teaching gun's position information, thereby improving the efficiency of the entire teaching process. The offset threshold b can be adjusted according to the actual application scenario and requirements. By adjusting this threshold, a balance between accuracy and robustness can be found to adapt to different working environments and task requirements.

[0053] The above description of the embodiments is intended to facilitate understanding and use of the invention by those skilled in the art. It will be apparent that those skilled in the art can readily make various modifications to these embodiments and apply the general principles described herein to other embodiments without requiring inventive effort. Therefore, the present invention is not limited to the above-described embodiments. Improvements and modifications made by those skilled in the art based on the disclosure of the present invention, without departing from the scope of the present invention, should be within the scope of protection of the present invention.

Claims

1. A teaching method for a robotic arm based on a teaching gun, characterized in that: The following steps are involved: S1: Obtaining a CAD model of a teaching gun (3), wherein the teaching gun (3) is provided with a 2D calibration plate (4), a 3D calibration object (5), and a needle tip (6); S2: Based on the parameters in the CAD model, the conversion relationship T from the needle tip (6) to the 2D calibration plate (4) is measured. point2board , and at the same time obtain the conversion relationship T from the needle tip (6) to the 3D calibration object (5) point2obj , and obtain the pose T of camera (1) in the base coordinate system eye2base ; S3: When the teaching gun is moved to the specified position i, the position T of the 2D calibration plate (4) in the camera coordinate system is obtained. board2eye , and simultaneously obtain the pose T of the 3D calibration object (5) in the camera coordinate system obj2eye ; S4: Calculate the position and posture of the needle tip (6) in the base coordinate system respectively through the 2D calibration plate (4) and the 3D calibration object (5). When the relative offset of the two postures is less than the preset threshold b, take the average posture as the final posture estimate T point2base ; S5: Using the obtained T point2base , control the robot arm (2) to move to the specified position and complete the teaching process.

2. A teaching method for a robotic arm based on a teaching gun according to claim 1, characterized in that: In S1, the 2D calibration plate (4) is selected from one of a center plate, a checkerboard, and ArUco.

3. A teaching method for a robotic arm based on a teaching gun according to claim 1, characterized in that: In S1, the 3D calibration object (5) is a geometric body with a preset three-dimensional shape, and the CAD model of the 3D calibration object (5) can be integrated with the CAD model of the teaching gun (3) into the same model.

4. A teaching method for a robotic arm based on a teaching gun according to claim 1, characterized in that: In S2, the conversion relationship from the needle tip (6) to the 2D calibration plate (4) is: in the same coordinate system, the mapping relationship from the coordinates of the needle tip (6) of the teaching gun to each calibration point on the 2D calibration plate (4).

5. A teaching method for a robotic arm based on a teaching gun according to claim 1, characterized in that: In S2, the conversion relationship from the needle tip (6) to the 3D calibration object (5) is: a mapping relationship from the coordinates of the needle tip (6) of the teaching gun to each calibration point on the 3D calibration object (5).

6. A teaching method for a robotic arm based on a teaching gun according to claim 1, characterized in that: In S3, the needle tip (6) of the teaching gun is moved to the designated position i by one of manual dragging, remote control, and programming instructions.

7. A teaching method for a robotic arm based on a teaching gun according to claim 1, characterized in that: In S3, the position T of the 2D calibration plate (4) in the camera coordinate system is obtained through the 2D calibration plate recognition algorithm. board2eye ; The 2D calibration plate recognition algorithm includes: image preprocessing, feature extraction, calibration plate recognition, camera parameter correction, and pose estimation.

8. The robot arm teaching method based on a teaching gun according to claim 1, characterized in that: In S3, the pose T of the 3D calibration object (5) in the camera coordinate system is obtained by the ICP alignment algorithm. obj2eye .

9. The teaching method for a robot arm based on a teaching gun according to claim 1, characterized in that: In S4, the pose of the needle tip (6) obtained by the 2D calibration plate (4) is estimated to be The pose estimation obtained by 3D calibration object is When the offset between the two is less than the threshold b, the average is taken as the final pose estimate T point2base ,in:

10. A teaching method for a robotic arm based on a teaching gun according to claim 1, characterized in that: In S5, when the teaching stage i is greater than 1, S3 to S5 are repeated until the teaching of each stage is completed.

Images