Industrial mechanical arm automatic calibration method and system based on depth camera

Abstract

The invention relates to the technical field of industrial automation, in particular to an industrial mechanical arm automatic calibration method and system based on a depth camera, and the system comprises a mechanical arm, the depth camera, a recognition patch and a controller. The method comprises installing a depth camera at the tail end of the mechanical arm; pasting an identification patch on the standard position of the target object; the depth camera collects RGB images and depth information; the data processing module extracts linear features through Hough transformation; calculating a spatial analytic expression of the straight line by utilizing the spatial geometrical relationship and the depth information; reversely solving the spatial position and attitude of the tail end of the mechanical arm by using the solved linear spatial analytic expression, and estimating errors in all directions; according to errors in all directions, the position of the next step of the mechanical arm is adjusted through a path planner on the controller; and the DH coordinate system of the mechanical arm is used for calculating the overall accurate position and posture of the mechanical arm. Compared with the prior art, the positioning precision is improved, the automation degree is improved, and the time consumption of positioning is reduced.

Description

Technical Field

[0001] The present invention relates to the field of industrial automation technology, and in particular to an automatic calibration method and system for an industrial robot arm based on a depth camera. Background Art

[0002] As the global economy accelerates its transformation toward digitalization and intelligence, the field of industrial manufacturing is undergoing profound changes. Especially in industries such as automobile manufacturing, electronic assembly, and precision machining, the application of industrial robotic arms is becoming more and more widespread, which not only improves production efficiency, but also significantly reduces human errors and reduces production costs. However, in these industries that require high-precision operations, the positioning accuracy of the robotic arm has become one of the key factors affecting the final quality and production efficiency of the product. Especially in application scenarios where the work location needs to be changed frequently, how to quickly and accurately determine the position and posture of the robotic arm has become an urgent problem to be solved.

[0003] At present, the positioning technology of industrial robotic arms mainly relies on various sensors, including visual sensors, lidar, ultrasonic sensors, etc. Among them, depth cameras are favored because they can provide RGB images and depth information at the same time. With the advancement of depth camera technology, such as the development of structured light, time of flight (ToF) and other technologies, the ability to obtain three-dimensional information of objects has been greatly improved, which provides new possibilities for the automatic positioning of robotic arms. However, the existing positioning methods based on depth cameras still have some shortcomings, such as limited positioning accuracy in complex environments and long calibration time. In addition, most positioning systems rely on fixed markers. When the robotic arm needs to change the working location, it needs to be recalibrated, which is not only time-consuming but also reduces production efficiency.

[0004] Therefore, it is particularly important to develop a method and system that can quickly and accurately determine the position and posture of an industrial robot arm. Especially for those application scenarios that require frequent changes in work locations, this method must not only be able to achieve sub-millimeter positioning accuracy, but also be able to adapt to different working environments, thereby reducing calibration time and costs and improving production efficiency. In response to this demand, the present invention proposes an automatic calibration method and system for an industrial robot arm based on a depth camera. This method and system can significantly improve the positioning accuracy of the industrial robot arm after changing the work location, thereby meeting the requirements of high-precision operations. Summary of the invention

[0005] In response to the above technical problems, the present invention proposes an automatic calibration method and system for industrial robotic arms based on a depth camera, which can solve the problem of intelligent positioning of non-fixed robotic arms after the work site is changed. It can significantly improve the positioning accuracy of industrial robotic arms after the work site is changed.

[0006] The technical problem to be solved by the present invention is achieved by adopting the following technical solutions:

[0007] Industrial robot arm automatic calibration system based on depth camera, including:

[0008] The robotic arm is used to control the system to achieve precise positioning and motion execution, including the end effector;

[0009] Depth camera, installed at the end of the robotic arm, used to collect RGB images and depth information;

[0010] Identification patch, a standard patch used to calibrate the position, helps the system complete the calculation of position and posture;

[0011] The controller is connected to the robotic arm and the depth camera, and is used to identify and process the recognition information fed back by the depth camera, control the robotic arm to make path planning adjustments, and achieve the purpose of automatic calibration of the base coordinates.

[0012] As a further improvement of the present invention, the controller includes a data processing module, a feedback path planner, and an actuator control unit.

[0013] As a further improvement of the present invention, the data processing module is responsible for processing the image and depth information obtained from the depth camera, extracting straight line features, and calculating the spatial position and posture of the robotic arm through an inverse solution method.

[0014] As a further improvement of the present invention, the feedback path planner dynamically adjusts the motion path of the robotic arm based on the positioning error, so as to ensure that the robotic arm accurately approaches the target position.

[0015] As a further improvement of the present invention, the actuator control unit realizes the motion control and path planning of the robot arm, and adjusts the moving path of the robot arm in real time based on the positioning error.

[0016] As a further improvement of the present invention, the identification patch includes a specific straight line pattern.

[0017] The automatic calibration method of an industrial robot arm based on a depth camera, using the automatic calibration system of an industrial robot arm based on a depth camera, comprises the following steps:

[0018] (1) Install a depth camera at the end of the robotic arm;

[0019] (2) attaching an identification patch containing a specific straight line pattern to the standard position of the target object;

[0020] (3) The depth camera collects RGB images and depth information and transmits them to the data processing module on the controller in real time;

[0021] (4) The data processing module extracts straight line features through Hough transform;

[0022] (5) Calculate the spatial analytical expression of the straight line using spatial geometric relationships and depth information;

[0023] (6) Use the obtained linear space analytical expression to reverse the spatial position and posture of the end of the robot arm and estimate the errors in all directions;

[0024] (7) According to the errors in each direction, the path planner on the controller adjusts the position of the robot arm for the next step;

[0025] (8) Use the DH coordinate system of the robot arm to calculate the overall precise position and posture of the robot arm.

[0026] As a further improvement of the present invention, the position and posture errors of the robotic arm do not exceed 1 mm and 0.1 degrees respectively.

[0027] The beneficial effects of the present invention are:

[0028] Improved positioning accuracy: The present invention enables the robot arm to quickly position itself after changing the work location, with a positioning accuracy of less than 1 mm and a posture error of no more than 0.1 degrees.

[0029] Improved automation: Automatic calibration of the robotic arm is achieved, reducing the need for human intervention.

[0030] Reduced positioning time: Through the feedback path planner, the movement path of the robot arm is dynamically adjusted, effectively improving positioning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention is further described below in conjunction with the accompanying drawings and embodiments:

[0032] Figure 1 It is a schematic diagram of the system framework;

[0033] Figure 2 It is the system flow chart;

[0034] Figure 3 To illustrate the process of moving and shooting Figure 1 ;

[0035] Figure 4 To illustrate the process of moving and shooting Figure 2 . DETAILED DESCRIPTION

[0036] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0037] like Figure 1As shown, the industrial robot arm automatic calibration system based on the depth camera includes a robot arm, a depth camera, a recognition patch, and a controller.

[0038] The robotic arm, including the end effector, can achieve precise positioning and motion execution through the control system.

[0039] The depth camera, installed at the end of the robotic arm, is used to collect RGB images and depth information, serving as the core perception module of the entire system.

[0040] Identification patch, a standard patch used for calibrating position, which contains a specific straight line pattern to help the system complete the calculation of position and posture.

[0041] The controller is connected to the robot arm and the depth camera to identify and process the recognition information fed back by the depth camera, control the robot arm to adjust the path planning, and achieve the purpose of automatic calibration of the base coordinates. It consists of three parts: data processing module, feedback path planner, and actuator control unit.

[0042] The data processing module is responsible for processing the image and depth information obtained from the depth camera, extracting straight line features, and calculating the spatial position and posture of the robotic arm through the inverse solution method.

[0043] The feedback path planner dynamically adjusts the motion path of the robot arm based on the positioning error to ensure that the robot arm approaches the target position accurately. As the robot arm approaches the image patch, the image and the corresponding depth information are captured each time. The estimation error in each direction is calculated, and the feedback path planner is designed based on the current screenshot posture. The movement path of the robot arm is adjusted according to the positioning error to maximize the positioning accuracy of the next capture. The path planner calculates the optimal path point based on the current positioning error and the current state of the robot arm to guide the robot arm to gradually approach and accurately position.

[0044] The actuator control unit realizes the motion control and path planning of the robot arm, and adjusts the movement path of the robot arm in real time based on the positioning error.

[0045] like Figure 2 As shown in the figure, the workflow of the automatic calibration system is as follows: the depth camera collects data and transmits it to the data processing module in real time. The data processing module extracts features from the collected images and depth information, and calculates the position and posture of the robot arm. The control system adjusts the motion path of the robot arm in real time through the feedback path planner according to the current posture and positioning error to optimize the positioning result. The robot arm performs corresponding movement and adjustment actions until it reaches the ideal posture and position. Figure 3 The moving and shooting process is demonstrated.

[0046] The automatic calibration method of an industrial robot arm based on a depth camera, using the automatic calibration system of an industrial robot arm based on a depth camera, comprises the following steps:

[0047] (1) Install a depth camera at the end of the robotic arm.

[0048] (2) A recognition patch containing a specific straight line pattern is pasted on the standard position of the target object.

[0049] (3) The depth camera collects RGB images and depth information and transmits them to the data processing module on the controller in real time.

[0050] (4) The data processing module extracts straight line features through Hough transform to improve the accuracy of feature extraction.

[0051] (5) Use spatial geometric relationships and depth information to calculate the spatial analytical expression of the line.

[0052] (6) Use the obtained linear space analytical expression to inversely calculate the spatial position and posture of the end of the robot arm and estimate the errors in each direction.

[0053] (7) According to the errors in each direction, the path planner on the controller adjusts the position of the robot arm for the next step to improve the accuracy of the next positioning result.

[0054] (8) Use the DH coordinate system of the robot arm to calculate the overall precise position and attitude of the robot arm. The position and attitude errors of the robot arm are no more than 1 mm and 0.1 degrees respectively.

[0055] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An industrial robot arm automatic calibration system based on a depth camera, characterized by: include: The robotic arm is used to control the system to achieve precise positioning and motion execution, including the end effector; Depth camera, installed at the end of the robotic arm, used to collect RGB images and depth information; Identification patch, a standard patch used to calibrate the position, helps the system complete the calculation of position and posture; The controller is connected to the robotic arm and the depth camera, and is used to identify and process the recognition information fed back by the depth camera, control the robotic arm to make path planning adjustments, and achieve the purpose of automatic calibration of the base coordinates.

2. The industrial robot arm automatic calibration system based on a depth camera according to claim 1, characterized in that: The controller includes a data processing module, a feedback path planner, and an actuator control unit.

3. The industrial robot arm automatic calibration system based on a depth camera according to claim 1, characterized in that: The data processing module is responsible for processing the image and depth information obtained from the depth camera, extracting straight line features, and calculating the spatial position and posture of the robotic arm through the inverse solution method.

4. The industrial robot arm automatic calibration system based on a depth camera according to claim 1, characterized in that: The feedback path planner dynamically adjusts the motion path of the robot arm based on the positioning error to ensure that the robot arm accurately approaches the target position.

5. The industrial robot arm automatic calibration system based on a depth camera according to claim 1, characterized in that: The actuator control unit realizes the motion control and path planning of the robot arm, and adjusts the movement path of the robot arm in real time based on the positioning error.

6. The industrial robot arm automatic calibration system based on a depth camera according to claim 1, characterized in that: The identification patch contains a specific pattern of straight lines.

7. An automatic calibration method for an industrial robot arm based on a depth camera, characterized in that: The industrial robot arm automatic calibration system based on a depth camera according to any one of claims 1 to 6 comprises the following steps: (1) Install a depth camera at the end of the robotic arm; (2) attaching an identification patch containing a specific straight line pattern to the standard position of the target object; (3) The depth camera collects RGB images and depth information and transmits them to the data processing module on the controller in real time; (4) The data processing module extracts straight line features through Hough transform; (5) Calculate the spatial analytical expression of the straight line using spatial geometric relationships and depth information; (6) Use the obtained linear space analytical expression to reverse the spatial position and posture of the end of the robot arm and estimate the errors in all directions; (7) According to the errors in each direction, the path planner on the controller adjusts the position of the robot arm for the next step; (8) Use the DH coordinate system of the robot arm to calculate the overall precise position and posture of the robot arm.

8. The method for automatic calibration of an industrial robot arm based on a depth camera according to claim 7, characterized in that: The position and attitude errors of the robotic arm do not exceed 1 mm and 0.1 degrees respectively.

Images