Three-dimensional camera posture online calibration device and method for industrial robots

Abstract

The present invention discloses an online calibration device and method for the position and posture of a three-dimensional camera for an industrial robot, including an industrial robot driving a calibration stereo target to move to a calibration point in a certain posture; the three-dimensional camera can measure the three-dimensional point cloud data in space, and output the calibration stereo target and the local three-dimensional point cloud data of the nearby structural parts to the calibration control module of the industrial control computer, the calibration control module processes and solves to obtain the three-dimensional coordinates of the visible calibration feature vertices in the three-dimensional camera coordinate system; identifies and retrieves to obtain the three-dimensional coordinates of each visible calibration feature vertex in the industrial robot coordinate system; solves to obtain the coordinate conversion matrix between the three-dimensional camera and the industrial robot. The present invention does not require additional calibration tooling accessories, has the characteristics of high efficiency and high precision, and can also verify or calibrate the coordinate conversion matrix of the three-dimensional camera and the industrial robot online after the manipulator is installed and during use.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent industrial robots, and in particular relates to an online position and posture calibration device and method for a three-dimensional stereo camera of an intelligent industrial robot. Background Art

[0002] Traditional industrial robots use teaching programming to accurately position single-type materials or products with known specifications and dimensions and postures in advance, and have been widely used in production, palletizing, loading and unloading, and other scenarios in the chemical, metal processing, food, beverage, tobacco, and electronics industries. However, they cannot adapt to the flexible production of small batches and multiple varieties of materials or products in the Industrial 4.0 era where the operating objects cannot be accurately positioned.

[0003] With the development and maturity of machine vision technology and products, industrial robots with visual perception capabilities have provided solutions for intelligent industrial robots. Based on visual perception technology, this solution can identify the environment, operating objects and the status of the equipment itself in dynamic scenes, intelligently plan the operating procedures and processes applicable to the current scene, and realize intelligent production and manufacturing, palletizing, loading and unloading, picking and other operations in variable scenes.

[0004] The visual perception sensors of intelligent industrial robots have now evolved from 2D industrial cameras to 3D stereo cameras. 3D stereo cameras can output 3D point cloud data based on the 3D camera coordinate system. When controlling the operation of the industrial robot, the coordinates under the 3D camera must be converted to the industrial robot coordinate system. This requires calibrating the rotation and translation transformation matrix of the 3D camera and the industrial robot coordinates before using the intelligent industrial robot.

[0005] In the case where the 3D camera is fixedly installed on the base outside the robot, since the components of the 3D camera include one or more 2D industrial cameras, the industry currently mostly adopts the traditional calibration method of the coordinate conversion matrix of the 2D industrial camera and the industrial robot: before installing the operating manipulator unit, a planar cooperative target such as a chessboard is installed on the robot flange, and the robot drives the planar cooperative target to run multiple points. The 2D industrial camera collects and processes the planar cooperative target image at each point, extracts the feature corner points, and uses the affine transformation principle to solve the coordinate conversion matrix of the 2D industrial camera and the industrial robot, and replaces the coordinate conversion matrix of the 3D camera and the industrial robot. However, the 2D coordinates of the pixel space of the 2D industrial camera and the 3D coordinates of the object space are not linearly related. The 3D camera usually needs to nonlinearly revise the 3D point cloud data obtained by the initial calculation to obtain higher-precision 3D point cloud data. Therefore, this calibration method will cause the 3D camera coordinates to be converted to the industrial robot coordinates, and the deviation from the 3D coordinates of the actual space is large, and the accuracy is low. Moreover, this calibration method requires additional calibration tooling accessories, which is time-consuming. When installing the operating manipulator on the robot flange, the planar cooperative target needs to be removed. The process is cumbersome and it is impossible to online verify or calibrate the coordinate transformation matrix of the three-dimensional stereo camera and the industrial robot during use.

[0006] At present, the industry has also adopted the method of installing spherical, spire and other three-dimensional cooperative targets with significant three-dimensional features on the robot flange before installing the operating manipulator unit. The robot drives the three-dimensional cooperative target to run multiple points. The three-dimensional camera collects and processes the three-dimensional point cloud data of the three-dimensional cooperative target at each point. The general algorithm in the PCL point cloud library is used to extract the coordinates of the center of the sphere or the corner of the spire feature in the three-dimensional camera coordinate system. Combined with the known coordinates of the center of the sphere or the spire in the industrial robot coordinate system, the coordinate transformation matrix of the three-dimensional camera and the industrial robot can be solved. This calibration method uses a general algorithm to extract the center of the sphere or the spire feature, which is heavily dependent on the three-dimensional point cloud quality of the stereoscopic cooperative target. It often requires manual removal of poor-quality three-dimensional point cloud data, and can only achieve offline calibration; at the same time, it requires additional calibration tooling accessories, which is time-consuming. When installing the operating manipulator on the robot flange, the three-dimensional cooperative target needs to be removed. The process is cumbersome and it is impossible to verify or calibrate the coordinate transformation matrix of the three-dimensional camera and the industrial robot online during use.

[0007] Therefore, the study proposes an online calibration method for the three-dimensional camera pose for intelligent industrial robots, which does not require additional calibration tooling accessories, is efficient and high-precision, and can also online verify or calibrate the coordinate transformation matrix of the three-dimensional camera and the industrial robot after the installation of the manipulator and during use. It has great significance and value. Summary of the invention

[0008] In order to solve the above-mentioned defects existing in the prior art, the purpose of the present invention is to provide an online calibration method for the position and posture of a three-dimensional camera for an intelligent industrial robot, aiming to solve the problems in the prior art that additional calibration tooling accessories are required, the accuracy is low, the time is long, and when the working manipulator is installed on the robot flange, the planar cooperation target needs to be removed, the process is cumbersome, and the coordinate transformation matrix of the three-dimensional camera and the industrial robot cannot be checked or calibrated online during use.

[0009] The present invention is achieved through the following technical solutions.

[0010] The present invention provides an online calibration method for a three-dimensional camera posture of an industrial robot, comprising the following steps:

[0011] Step 1: The calibration control module of the industrial control computer issues a calibration command, and the industrial robot drives the calibration stereo target to move to the calibration point in a certain posture;

[0012] Step 2: The 3D camera collects 3D point cloud data in the measurable space, and outputs local 3D point cloud data of the calibration stereo target and its nearby structural parts to the calibration control module of the industrial control computer;

[0013] Step 3: The calibration control module processes the local 3D point cloud data and calculates the 3D coordinates of the visible calibration feature vertices in the 3D camera coordinate system;

[0014] Step 4: the calibration control module of the industrial control computer identifies the visible calibration feature vertices, and retrieves and obtains the three-dimensional coordinates of each visible calibration feature vertex in the industrial robot coordinate system;

[0015] Step 5: Using the visible calibration feature vertices that correspond one to one between the three-dimensional camera coordinate system and the industrial robot coordinate system, calculate and obtain the coordinate transformation matrix between the three-dimensional camera and the industrial robot.

[0016] For the above technical solution, the present invention has a further preferred solution:

[0017] Preferably, in step 1, determining the calibration point and the calibration stereo target posture includes the following steps:

[0018] 1a) Establish a basic coordinate system for calibration: The industrial robot base coordinate system is used as the basic coordinate system for calibration; the coordinate system is an XY plane formed by the X-axis and Y-axis approximately located in the horizontal plane and the Z-axis perpendicular to the X-axis and Y-axis;

[0019] 1b) Determine the range of the calibration plane along the XY plane: the calibration plane is parallel to the XY plane of the basic coordinate system for calibration; the range of the calibration plane along the XY plane covers the extreme positions along the X-axis and Y-axis in the working space of the industrial robot;

[0020] 1c) Determine the height of the calibration plane along the Z axis: The height of the calibration plane along the Z axis is selected at a certain height in the working space of the industrial robot;

[0021] 1d) Determine the calibration point: select the vertex of the calibration plane as the calibration point;

[0022] 1e) Determine the posture of the calibration stereo target: the calibration feature vertex of the calibration stereo target is tilted upward.

[0023] Preferably, in step 2, the method in which the three-dimensional camera collects three-dimensional point cloud data in the measurable space and outputs local three-dimensional point cloud data of the calibration stereo target and its nearby structural parts to the calibration control module of the industrial control computer comprises the following steps:

[0024] 2a) The calibration control module issues a 3D point cloud data acquisition instruction, and the 3D stereo camera acquires 3D point cloud data in the measurable space;

[0025] 2b) The three-dimensional stereo camera processes the three-dimensional point cloud data collected in step 2a) using a straight-through filtering method to obtain local three-dimensional point cloud data of the calibration stereo target and its nearby structural parts.

[0026] Preferably, in step 3, the method in which the calibration control module processes the local three-dimensional point cloud data comprises the following steps:

[0027] 3a) adopting the straight-through filtering method to segment and extract the 3D point cloud data containing only the calibration stereo target from the local 3D point cloud data;

[0028] 3b) performing plane segmentation and fitting on the obtained three-dimensional point cloud data of the calibration stereo target to obtain the area and characteristic normal vector of the plane;

[0029] 3c) performing an intersection operation on the obtained fitting planes to obtain the three-dimensional coordinates of the visible calibration feature vertices in the three-dimensional camera coordinate system.

[0030] Preferably, in step 4, the method for identifying the visible calibration feature vertices comprises the following steps:

[0031] 4a) calculating the visible calibration feature vertex spacing according to the coordinates of the visible calibration feature vertices;

[0032] 4b) identifying the numbers of the calibration feature vertices by combining the visible calibration feature vertex spacing and the topological relationship between the calibration feature vertices;

[0033] 4c) according to the number of the characteristic vertex for calibration, retrieve the three-dimensional coordinates of the vertex in the three-dimensional target for calibration in a known database;

[0034] 4d) converting the coordinates in the calibration stereo target into three-dimensional coordinates in the industrial robot flange coordinate system;

[0035] 4e) Convert the three-dimensional coordinates in the industrial robot flange coordinate system to the three-dimensional coordinates in the industrial robot base coordinate system.

[0036] Preferably, in step 5, the method for solving the coordinate transformation matrix between the three-dimensional camera and the industrial robot is as follows:

[0037] 5a) The same-name points in the three-dimensional space satisfy the following conditions in the three-dimensional camera coordinate system and the industrial robot coordinate system:

[0038]

[0039] Among them, P b1 ,P b2 ,P bi The three-dimensional coordinates of the visible calibration feature vertex in the industrial robot coordinate system; RT cb is the coordinate transformation matrix between the 3D camera and the industrial robot; Pc1, Pc2, Pci are the 3D coordinates of the visible calibration feature vertices in the 3D camera coordinate system; the subscript i represents the total number of visible calibration feature vertices extracted from each calibration point;

[0040] 5b) Using the three-dimensional coordinates of the multiple visible calibration feature vertices obtained in steps 3 and 4 in the three-dimensional camera coordinate system and the industrial robot coordinate system, the least squares method is used to solve and obtain the coordinate transformation matrix between the three-dimensional camera and the industrial robot.

[0041] The present invention further provides a three-dimensional camera posture online calibration device for an industrial robot adopted by the method, comprising:

[0042] The industrial control computer includes a calibration control module, which controls the industrial robot to drive the calibration stereo target to move to the calibration point and then triggers the three-dimensional camera to collect and process three-dimensional point cloud data; the three-dimensional point cloud data processing module in the calibration control module processes the three-dimensional point cloud data to obtain the three-dimensional coordinates of the visible calibration feature vertices in the three-dimensional camera coordinate system;

[0043] A three-dimensional stereo camera is used to collect and process three-dimensional point cloud data of a calibration stereo target, and output local three-dimensional point cloud data of the calibration stereo target and its nearby structural parts to a calibration control module;

[0044] Industrial robot, used to drive the calibration stereo target to move to the calibration point in a certain posture;

[0045] The stereo target for calibration is used to provide feature corner points in space;

[0046] The industrial control computer is respectively connected to the three-dimensional camera and the industrial robot, and the industrial robot is connected to a calibration stereo target to achieve online position and posture calibration between the three-dimensional camera and the industrial robot.

[0047] Preferably, an operating manipulator and a calibration stereo target are installed on the industrial robot, and the calibration stereo target is provided with calibration feature vertices required for collecting and processing point clouds by a three-dimensional camera.

[0048] Preferably, the operating manipulator is fastened to the industrial robot via an adapter flange, and the calibration stereoscopic target is mounted on the adapter flange connecting the industrial robot and the operating manipulator.

[0049] Preferably, the calibration stereo target is a polygonal frame structure, in which a pair of symmetrical trapezoidal structures and a rectangular structure are provided in the polygonal inner frame, and the vertices of the inner frame of the pair of symmetrical trapezoidal structures and the vertices of the polygonal outer frame are the characteristic vertices for calibration required for the three-dimensional camera to collect and process point clouds.

[0050] The present invention adopts the above technical solution, which has the following beneficial effects:

[0051] 1. The present invention adopts a calibration stereo target installed between the flange of the industrial robot and the operating manipulator, which has a plurality of calibration feature vertices formed by the intersection of planes; by extracting visible calibration feature vertices at the calibration points and combining their coordinates in the industrial robot coordinate system, the coordinate transformation matrix of the three-dimensional camera and the industrial robot is directly solved, which has the significant advantages of high precision, simple and fast calibration process.

[0052] 2. In the present invention, the three-dimensional target for calibration is installed between the flange of the industrial robot and the operating manipulator. The installation, use and disassembly of the operating manipulator will not change the position and posture of the three-dimensional target for calibration, so that the intelligent industrial robot can realize the functions of online verification and calibration after the operating manipulator unit is installed and during use.

[0053] 3. The calibration stereo target in the present invention has a plurality of calibration feature vertices formed by plane intersection. The three-dimensional point cloud data of the calibration stereo target collected by the three-dimensional camera is solved by plane segmentation, fitting and plane intersection, which can greatly reduce the calibration feature vertex extraction error caused by edge noise, residual edges, holes, etc., and further ensure the reliability and stability of online verification or calibration.

[0054] 4. In the present invention, what are extracted at the calibration points are visible feature vertices for calibration, the visible feature vertices for calibration are identified, and the three-dimensional coordinates of each visible feature vertex for calibration in the industrial robot coordinate system are retrieved, thereby improving the robustness of the algorithm and being able to adapt to the problem of inconsistent quality of three-dimensional point cloud data caused by changes in ambient light. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute an improper limitation of the present invention. In the drawings:

[0056] Figure 1 A block diagram of a three-dimensional camera pose online calibration system for an intelligent industrial robot provided by an embodiment of the present invention is shown;

[0057] Figure 2a , 2b A schematic diagram of the structure of an online calibration system for a three-dimensional camera posture of an intelligent industrial robot provided by an embodiment of the present invention is shown;

[0058] Figure 3 A flowchart of a three-dimensional camera pose online calibration method for an intelligent industrial robot provided by an embodiment of the present invention is shown;

[0059] Figure 4 A flowchart of determining the calibration points and the posture of a calibration stereo target provided by an embodiment of the present invention is shown.

[0060] Figure 5 A schematic diagram of a model point cloud of a stereo target for calibration provided in an embodiment of the present invention is shown.

[0061] In the figure: 1. Stereo target for calibration; 2. Feature vertices for calibration; 3. Calibration control module; 4. 3D point cloud data processing module; 5. Feature vertex database for calibration; 6. 3D camera; 7. Bracket; 8. Industrial robot; 9. Operating manipulator; 10. Industrial robot control cabinet; 11. Industrial control computer; 12. Local 3D point cloud data; 13. Visible feature vertices for calibration; 14. Working space; 15. Adapter flange; 16. Connecting bolts; 17. Locating pins. DETAILED DESCRIPTION

[0062] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. The exemplary embodiments and descriptions of the present invention are used to explain the present invention but are not intended to limit the present invention.

[0063] like Figure 1 , Figure 2a , 2b As shown, a three-dimensional stereo camera posture online calibration device for an intelligent industrial robot includes a calibration stereo target 1, calibration feature vertices 2 on the calibration stereo target 1, a calibration control module 3, a three-dimensional point cloud data processing module 4 in the calibration control module 3, and a calibration feature vertex database 5 in the calibration control module 3.

[0064] like Figure 2a As shown, a schematic structural diagram of a three-dimensional camera pose online calibration device for an intelligent industrial robot is shown, including a three-dimensional camera 6, a bracket 7, an industrial robot 8, an operating manipulator 9, an industrial robot control cabinet 10 and an industrial control computer 11.

[0065] An operating manipulator 9 is installed on the industrial robot 8, an industrial control computer 11 is placed in an industrial robot control cabinet 10, a three-dimensional camera 6 is mounted on a bracket 7, and an operating space 14 of the industrial robot 8 is located within the field of view of the three-dimensional camera 6; the industrial robot 8 and the three-dimensional camera 6 are connected to the industrial control computer 11 by signal.

[0066] like Figure 2b As shown, the operating manipulator 9 is fastened to the industrial robot 8 through the adapter flange 15, and the calibration stereo target 1 is horizontally connected below the adapter flange 15 through the connecting bolts 16 and the positioning pins 17. The calibration stereo target 1 is a polygonal frame structure, and a pair of symmetrical trapezoidal structures and a rectangular structure are provided in the polygonal inner frame. The vertices of the inner frames of the pair of symmetrical trapezoidal structures and the vertices of the polygonal outer frame are the calibration feature vertices 2 required for the three-dimensional camera 6 to collect and process the point cloud.

[0067] The three-dimensional coordinates of the calibration feature vertex 2 in the calibration stereo target 1 are stored in the calibration feature vertex database 5 , and the three-dimensional coordinates can be obtained according to the serial number index of the calibration feature vertex 2 .

[0068] The calibration control module 3 is installed in the industrial control computer 11. The calibration control module 3 is used to control the movement position and posture of the industrial robot 8; control the industrial robot 8 to drive the calibration stereo target 1 to move to the calibration points P1, P2, P3, and P4, and then trigger the three-dimensional stereo camera to collect three-dimensional point cloud data.

[0069] The three-dimensional point cloud data processing module 4 processes the local three-dimensional point cloud data 12 including the calibration stereo target 1 to obtain the three-dimensional coordinates of the visible calibration feature vertices 13 in the three-dimensional camera coordinate system.

[0070] The calibration control module 3 uses the visible calibration feature vertices 13 in one-to-one correspondence between the three-dimensional camera coordinate system and the industrial robot coordinate system to calculate and obtain the coordinate transformation matrix RT between the three-dimensional camera and the industrial robot. cb .

[0071] The three-dimensional camera 6 is installed on a bracket 7 on one side of the industrial robot 1; the bracket 7 is not limited to being formed as an integrated structural part by welding, but can also be formed as a component by assembling profiles; the bracket 7 is not limited to being installed on the ground, but can also be hoisted; the bracket should have mechanical stability to ensure that the coordinate system of the three-dimensional camera 6 is stable for a long time.

[0072] The measuring range of the three-dimensional camera 6 can cover the working space 14 of the industrial robot; the three-dimensional camera 2 is connected to the industrial control computer 4 by installing a communication control cable.

[0073] The industrial robot 8 is a large-range motion actuator; the arm span range of the industrial robot 8 covers the working space 14 requirements in specific application scenarios.

[0074] The industrial robot 8 is connected to the industrial robot control cabinet 10 by installing a communication control cable; the industrial robot control cabinet 10 is connected to the industrial control computer 11 by installing a communication control cable.

[0075] like Figure 3 As shown, the present invention accordingly provides a three-dimensional stereo camera pose online calibration method for an intelligent industrial robot, and the steps are as follows:

[0076] Step 1: The calibration control module of the industrial control computer issues a calibration command, and the industrial robot drives the calibration stereo target to move to the calibration point P in a certain posture. b .

[0077] like Figure 4 As shown, the determination of the calibration point and the calibration stereo target posture includes the following steps:

[0078] 1a) Establish the basic coordinate system for calibration: The industrial robot base coordinate system is used as the basic coordinate system OXYZ for calibration. The X-axis and Y-axis are approximately located in the horizontal plane, and the Z-axis is perpendicular to the XY plane formed by the X-axis and Y-axis.

[0079] 1b) Determine the range of the calibration plane along the XY plane: The calibration plane is parallel to the XY plane of the calibration base coordinate system. The range of the calibration plane along the XY plane covers the extreme positions along the X-axis and Y-axis in the working space of the industrial robot; the shape of the calibration plane is preferably a rectangle.

[0080] In step 1b), preferably, the limit position x along the X-axis and the Y-axis in the working space of the industrial robot is bmin 、x bmax ,y bmin ,y bmax 1.1 times or more of the range of the calibration plane along the XY plane, that is, [(x bmax ,y bmax )、(x bmax ,y bmin )、(x bmin ,y bmax )、(x bmin ,y bmin )] is 1.1 times.

[0081] 1c) Determine the height of the calibration plane along the Z axis: The height of the calibration plane along the Z axis is selected at a certain height in the working space of the industrial robot; the height of the calibration plane along the Z axis is preferably at a distance of 1 / 3 times the working range from the highest working space height of the industrial robot, that is, z bmax -(z bmax -z bmin )×1 / 3.

[0082] 1d) Determine the calibration point: Select the vertex of the calibration plane as the calibration point.

[0083] 1e) Determine the posture of the calibration stereo target: The calibration feature vertex of the calibration stereo target is tilted upward. The posture of the calibration stereo target is preferably such that the top surface of the calibration stereo target forms an angle of 45 degrees with the XY plane, that is, the posture component is (A b ,B b ,C b ), constraining the top surface of the calibration stereo target to form an angle of 45 degrees with the XY plane.

[0084] Step 2: The 3D camera collects 3D point cloud data in the measurable space and outputs local 3D point cloud data of the calibration stereo target and its nearby structural parts to the calibration control module.

[0085] The method for collecting and outputting three-dimensional point cloud data of a calibration stereo target and its surrounding structural parts by a three-dimensional stereo camera comprises the following steps:

[0086] 2a) The calibration control module issues a 3D point cloud data acquisition instruction, and the 3D stereo camera acquires 3D point cloud data in the measurable space.

[0087] 2b) The three-dimensional camera uses a through-filter method to process the three-dimensional point cloud data collected in step 2a) to obtain local three-dimensional point cloud data of the calibration stereo target and its nearby structural parts. The filtering parameters in the through-filter method are determined by the calibration point where the three-dimensional point cloud data is collected by the three-dimensional camera and the size of the calibration stereo target, and the three-dimensional point cloud data of the calibration stereo target must be completely included.

[0088] Step 3: The 3D point cloud data processing module in the calibration control module processes the local 3D point cloud data and calculates the 3D coordinates of the visible calibration feature vertices in the 3D camera coordinate system.

[0089] The method for processing local three-dimensional point cloud data by the calibration control module comprises the following steps:

[0090] 3a) A straight-through filtering method is used to segment and extract the 3D point cloud data containing only the calibration stereo target from the local 3D point cloud data.

[0091] 3b) performing plane segmentation and fitting on the three-dimensional point cloud data of the calibration stereo target obtained in step 3a) to obtain the area and characteristic normal vector of the plane.

[0092] 3c) performing an intersection operation on the fitting plane obtained in step 3b) to obtain the three-dimensional coordinates of the visible calibration feature vertices in the three-dimensional camera coordinate system.

[0093] Step 4: The calibration control module identifies the visible calibration feature vertices, and retrieves the three-dimensional coordinates of each visible calibration feature vertex in the industrial robot coordinate system.

[0094] The method for identifying visible calibration feature vertices comprises the following steps:

[0095] 4a) Calculate the distances between the visible calibration feature vertices based on the coordinates of the visible calibration feature vertices.

[0096] 4b) Combining the distances between the visible calibration feature vertices and the topological relationships between the calibration feature vertices, the numbers of the calibration feature vertices are identified.

[0097] 4c) According to the number of the feature vertex, the three-dimensional coordinates of the vertex in the calibration stereo target are retrieved from a known database.

[0098] 4d) Convert the coordinates of the calibration stereo target into three-dimensional coordinates in the industrial robot flange coordinate system.

[0099] 4e) Convert the coordinates in the flange coordinate system of the industrial robot into three-dimensional coordinates in the base coordinate system of the industrial robot.

[0100] Step 5: Using the visible calibration feature vertices that correspond one to one between the three-dimensional camera coordinate system and the industrial robot coordinate system, calculate and obtain the coordinate transformation matrix between the three-dimensional camera and the industrial robot.

[0101] The method for solving the coordinate transformation matrix between the three-dimensional camera and the industrial robot is as follows:

[0102] 5a) The same-name points in space satisfy the following conditions in the three-dimensional camera coordinate system and the industrial robot coordinate system:

[0103]

[0104] Among them, P b1 ,P b2 ,P bi The three-dimensional coordinates of the visible calibration feature vertex in the industrial robot coordinate system. RT cb is the coordinate transformation matrix between the 3D camera and the industrial robot. Pc1, Pc2, Pci are the 3D coordinates of the visible calibration feature vertices in the 3D camera coordinate system. The subscript i represents the total number of visible calibration feature vertices extracted from each calibration point.

[0105] 5b) In step 3 and step 4), the three-dimensional coordinates of multiple visible calibration feature vertices in the three-dimensional camera coordinate system and the industrial robot coordinate system are obtained respectively, and the coordinate transformation matrix between the three-dimensional camera and the industrial robot is obtained by solving the least squares method.

[0106] The present invention is further illustrated by a specific embodiment below.

[0107] As shown in Figure 2, the calibration base coordinate system OXYZ is established to b X b Y b Z b The basic coordinate system OXYZ is used for calibration. The XY plane formed by the X-axis and the Y-axis is affected by the horizontality of the base installation of the depalletizing robot, and is approximately located in the horizontal plane. The Z axis is perpendicular to the XY plane formed by the X-axis and the Y-axis.

[0108] The limit position x along the X axis in the working space of the depalletizing robot bmin -600mm,x bmax The limit position y along the Y axis is 600 mm. bmin 800mm,y bmaxThe calibration plane is 1600mm. The shape of the calibration plane is a rectangle. The range of the calibration plane along the XY plane is 1.1 times the limit position along the X-axis and Y-axis in the working space of the depalletizing robot, that is, the four vertices of the calibration plane [P1, P2, P3, P4] are [(-660, 720), (660, 720), (660, 1680), (-660, 1680)].

[0109] The maximum working space height of the depalletizing robot is 1800mm, and the working range is 1200mm. The height of the calibration plane along the Z axis is selected at a distance of 1 / 3 times the working range from the maximum working space height of the industrial robot, that is, 1400mm. The four calibration points are determined as [(-660,720,1400), (660,720,1400), (660,1680,1400), (-660,1680,1400)].

[0110] The posture of the calibration stereo target is selected. The top surface of the calibration stereo target is at a 45-degree angle to the XY plane, that is, the flange of the depalletizing robot is around the Y b The axis rotates 45 degrees clockwise, and the attitude component is (A b ,B b ,C b ) is (0,45,0). The poses of the four calibration points are determined to be [(-660,720,1400,0,45,0), (660,720,1400,0,45,0), (660,1680,1400,0,45,0), (-660,1680,1400,0,45,0)].

[0111] The calibration control module issues a calibration command, and the depalletizing robot drives the calibration stereo target to move to the first calibration point P in a certain posture. b =(-660,720,1400,0,45,0). The 3D camera collects 3D point cloud data in the measurable space, and adopts the straight-through filtering method. The range of the straight-through filtering is 400mm in the X-axis, 300mm in the Y-axis, and 400mm in the Z-axis. The local 3D point cloud data of the calibration stereo target and its nearby structural parts are output to the calibration control module.

[0112] The 3D point cloud data processing module in the calibration control module processes the local 3D point cloud data and uses straight-through filtering to segment and extract the 3D point cloud data containing only the calibration stereo target from the local 3D point cloud data. The 3D point cloud data of the calibration stereo target is as follows: Figure 5 shown.

[0113] As shown in Table 1, plane segmentation and fitting are used to obtain the plane area and characteristic normal vector. Intersection operation is performed on the fitted plane to obtain the visible calibration feature vertex Pc1 ,P c2 ,…,P c14 The 3D coordinates in the 3D stereo camera coordinate system.

[0114] Table 1 The three-dimensional coordinates of the feature vertices used for calibration in the three-dimensional camera coordinate system

[0115]

[0116]

[0117] The calibration control module calculates the distance between the visible calibration feature vertices and uses the topological relationship between the feature vertices to identify the number of the calibration feature vertices. According to the number of the feature vertex, the visible calibration feature vertex P is retrieved from the known database. c1 ,P c2 ,…,P c14 The three-dimensional coordinate P in the coordinate system of the depalletizing robot b1 ,P b2 ,…,P b14 .

[0118] The calibration control module issues a calibration command, and the depalletizing robot drives the calibration stereo target to move to the second to fourth calibration points P in a certain posture. b =((660,720,1400,0,45,0),(660,1680,1400,0,45,0),(-660,1680,1400,0,45,0)]. Obtain the visible calibration feature vertices P at the second to fourth calibration points. c15 ,P c16 ,…,P c64 The 3D coordinates in the 3D camera coordinate system. The visible calibration feature vertex P c15 ,P c16 ,…,P c64 The three-dimensional coordinate P in the coordinate system of the depalletizing robot b15 ,P b16 ,…,P b64 .

[0119] Using the visible calibration feature vertices in one-to-one correspondence between the 3D camera coordinate system and the industrial robot coordinate system, the coordinate transformation matrix RT between the 3D camera and the industrial robot is obtained by least squares solution. cb .

[0120]

[0121] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A three-dimensional camera pose online calibration method for an industrial robot, characterized in that: The steps include: Step 1: The calibration control module of the industrial control computer issues a calibration command, and the industrial robot drives the calibration stereo target to move to the calibration point in a certain posture; Step 2: The 3D camera collects 3D point cloud data in the measurable space, and outputs local 3D point cloud data of the calibration stereo target and its nearby structural parts to the calibration control module of the industrial control computer; Step 3: The calibration control module processes the local 3D point cloud data and calculates the 3D coordinates of the visible calibration feature vertices in the 3D camera coordinate system; Step 4: the calibration control module of the industrial control computer identifies the visible calibration feature vertices, and retrieves and obtains the three-dimensional coordinates of each visible calibration feature vertex in the industrial robot coordinate system; Step 5, using the visible calibration feature vertices that correspond one to one in the three-dimensional camera coordinate system and the industrial robot coordinate system, calculate and obtain the coordinate transformation matrix between the three-dimensional camera and the industrial robot; In step 1, the determination of the calibration point and the calibration stereo target posture includes the following steps: 1a) Establish a basic coordinate system for calibration: The industrial robot base coordinate system is used as the basic coordinate system for calibration; the coordinate system is an XY plane formed by the X-axis and Y-axis approximately located in the horizontal plane and the Z-axis perpendicular to the X-axis and Y-axis; 1b) Determine the range of the calibration plane along the XY plane: the calibration plane is parallel to the XY plane of the basic coordinate system for calibration; the range of the calibration plane along the XY plane covers the extreme positions along the X-axis and Y-axis in the working space of the industrial robot; 1c) Determine the height of the calibration plane along the Z axis: The height of the calibration plane along the Z axis is selected at a certain height in the working space of the industrial robot; 1d) Determine the calibration point: select the vertex of the calibration plane as the calibration point; 1e) Determine the posture of the calibration stereo target: the calibration feature vertex of the calibration stereo target is tilted upward.

2. The method for online calibration of a three-dimensional camera pose for an industrial robot according to claim 1, characterized in that: In step 2, the method of collecting three-dimensional point cloud data in the measurable space with a three-dimensional stereo camera and outputting local three-dimensional point cloud data of a calibration stereo target and its nearby structural parts to a calibration control module of an industrial control computer comprises the following steps: 2a) The calibration control module issues a 3D point cloud data acquisition instruction, and the 3D stereo camera acquires 3D point cloud data in the measurable space; 2b) The three-dimensional stereo camera processes the three-dimensional point cloud data collected in step 2a) using a straight-through filtering method to obtain local three-dimensional point cloud data of the calibration stereo target and its nearby structural parts.

3. The method for online calibration of a three-dimensional camera pose for an industrial robot according to claim 1, characterized in that: In step 3, the method for the calibration control module to process the local three-dimensional point cloud data includes the following steps: 3a) Using the straight-through filtering method, the 3D point cloud data containing only the calibration stereo target is segmented and extracted from the local 3D point cloud data; 3b) Perform plane segmentation and fitting on the obtained three-dimensional point cloud data of the calibration stereo target to obtain the area and characteristic normal vector of the plane; 3c) Perform an intersection operation on the obtained fitting planes to obtain the three-dimensional coordinates of the visible calibration feature vertices in the three-dimensional stereo camera coordinate system.

4. The method for online calibration of a three-dimensional camera pose for an industrial robot according to claim 1, characterized in that: In step 4, the method for identifying the visible calibration feature vertices comprises the following steps: 4a) calculating the visible calibration feature vertex spacing according to the coordinates of the visible calibration feature vertices; 4b) Combining the visible calibration feature vertex spacing and the topological relationship between the calibration feature vertices, the numbers of the calibration feature vertices are identified; 4c) According to the number of the characteristic vertex for calibration, the three-dimensional coordinates of the vertex in the three-dimensional target for calibration are retrieved from a known database; 4d) Convert the coordinates of the calibration stereo target into three-dimensional coordinates in the industrial robot flange coordinate system; 4e) Convert the three-dimensional coordinates in the industrial robot flange coordinate system to the three-dimensional coordinates in the industrial robot base coordinate system.

5. The method for online calibration of a three-dimensional camera pose for an industrial robot according to claim 1, characterized in that: In step 5, the method for solving the coordinate transformation matrix between the three-dimensional camera and the industrial robot is as follows: 5a) The same-name points in the three-dimensional space satisfy the following conditions in the three-dimensional camera coordinate system and the industrial robot coordinate system: Among them, P b1 ,P b2 ,P bi The three-dimensional coordinates of the visible calibration feature vertex in the industrial robot coordinate system; RT cb is the coordinate transformation matrix between the 3D camera and the industrial robot; Pc1, Pc2, Pci are the 3D coordinates of the visible calibration feature vertices in the 3D camera coordinate system; the subscript i represents the total number of visible calibration feature vertices extracted from each calibration point; 5b) Using the 3D coordinates of the multiple visible calibration feature vertices obtained in steps 3 and 4 in the 3D camera coordinate system and the industrial robot coordinate system, the least squares method is used to solve and obtain the coordinate transformation matrix between the 3D camera and the industrial robot.

6. A three-dimensional camera pose online calibration device for an industrial robot used in the method according to any one of claims 1 to 5, characterized in that: include: The industrial control computer includes a calibration control module, which controls the industrial robot to drive the calibration stereo target to move to the calibration point and then triggers the three-dimensional camera to collect three-dimensional point cloud data; the three-dimensional point cloud data processing module in the calibration control module processes the three-dimensional point cloud data to obtain the three-dimensional coordinates of the visible calibration feature vertices in the three-dimensional camera coordinate system; A three-dimensional stereo camera is used to collect three-dimensional point cloud data in the measurable space and output local three-dimensional point cloud data of the calibration stereo target and its nearby structural parts to the calibration control module; Industrial robot, used to drive the calibration stereo target to move to the calibration point in a certain posture; The stereo target for calibration is used to provide feature corner points in space; The industrial control computer is respectively connected to the three-dimensional camera and the industrial robot, and the industrial robot is connected to a calibration stereo target to achieve online position and posture calibration between the three-dimensional camera and the industrial robot.

7. The online calibration device for a three-dimensional camera posture of an industrial robot according to claim 6, characterized in that: An operating manipulator and a calibration stereo target are installed on the industrial robot. The calibration stereo target is provided with calibration feature vertices required for collecting and processing point clouds by a three-dimensional camera.

8. The three-dimensional camera pose online calibration device for an industrial robot according to claim 7, characterized in that: The operating manipulator is fastened to the industrial robot via an adapter flange, and a calibration stereoscopic target is mounted on the adapter flange connecting the industrial robot and the operating manipulator.

9. The device for online calibration of a three-dimensional camera posture of an industrial robot according to claim 6, characterized in that: The calibration stereo target is a polygonal frame structure, in which a pair of symmetrical trapezoidal structures and a rectangular structure are arranged in the polygonal inner frame. The vertices of the inner frame of the pair of symmetrical trapezoidal structures and the vertices of the polygonal outer frame are the characteristic vertices for calibration required for the three-dimensional camera to collect and process point clouds.

Images