A method for coordinate system calibration and operation of an industrial robot 3D vision workstation

Abstract

This invention discloses a method for calibrating and operating the coordinate system of an industrial robot 3D vision workstation. The method includes: identifying the kinematic model of the industrial robot; performing hand-eye calibration between the industrial robot and the 3D vision camera, and calibration between the 3D vision camera and a single-axis turntable; obtaining the homogeneous transformation matrix between the single-axis turntable, the 3D vision camera, and the industrial robot; placing the workpiece to be processed on the single-axis turntable; using the 3D vision camera to capture images of the workpiece for 3D reconstruction and to identify processing features; and generating a processing path for the industrial robot based on the processing features and the homogeneous transformation matrix to complete the processing of the workpiece. This invention significantly reduces calibration difficulty and improves calibration accuracy. It can be widely applied to various robot workstations and can also be extended to applications involving 3D vision cameras and multi-axis turntables, possessing a very broad market application prospect and extremely important practical significance for improving the digitalization and intelligentization level of the manufacturing industry.

Description

TECHNICAL FIELD

[0001] The application relates to a work station coordinate system calibration and operation method, in particular to an industrial robot three-dimensional vision work station coordinate system calibration and operation method. BACKGROUND

[0002] With the rapid development of intelligent manufacturing, the coordinate system calibration technology is increasingly widely applied in various automatic devices. In a work station composed of an industrial robot, a positioner and a vision sensor, high-precision coordinate system calibration between the components is a key link to ensure the collaborative work of the device. Through accurate calibration of the coordinate system and determination of the pose relationship between each other, the components can be accurately controlled and instructed based on the respective and mutual coordinate system, so that the execution efficiency and processing precision of the work station are further improved.

[0003] At present, the calibration technologies mainly applied in automatic devices mainly include the following: (1) calibration of an industrial robot and a rotary table, mainly used for various welding work stations with rotary tables; (2) calibration of a robot and a vision sensor, mainly used for robot vision systems with eyes outside or on the hand; (3) calibration of a vision sensor and a rotary table, mainly used for various three-dimensional reconstruction systems. With the continuous development of robot offline programming and online programming technologies, the requirement for the calibration accuracy of the coordinate system in the robot work station is higher and higher. To achieve high-precision coordinate system calibration, the calibration error introduced by the low positioning accuracy of the industrial robot must be solved, and the simplicity and ease of use of the calibration method must also be considered.

[0004] In summary, the current industrial robot three-dimensional vision work station coordinate system calibration and operation method has the following problems: 1) the calibration error introduced by the low positioning accuracy of the industrial robot cannot be effectively solved, and the coordinate system calibration accuracy is not high; 2) the calibration cost and complexity are high, and if the positioning accuracy of the industrial robot is to be improved, an expensive laser tracker must be used, which is difficult to be widely applied in industrial production lines. SUMMARY

[0005] In order to solve the problems in the background art, the application provides an industrial robot three-dimensional vision work station coordinate system calibration and operation method. For an industrial robot three-dimensional vision work station composed of an industrial robot, a single-axis rotary table, a three-dimensional vision camera and a base, a new robot positioning accuracy compensation method and a coordinate system calibration and operation method of the three-dimensional vision camera and the single-axis rotary table are designed, and high-precision and easy-to-operate industrial robot three-dimensional vision work station coordinate system calibration is achieved.

[0006] The technical scheme adopted by the application is:

[0007] The industrial robot three-dimensional vision work station coordinate system calibration and operation method of the application comprises:

[0008] 1) Install a circle dot calibration board on an industrial robot three-dimensional vision work station composed of a single-axis turntable, a three-dimensional vision camera, an industrial robot and a work station base, the circle dot calibration board is installed on the end of the industrial robot; drive the circle dot calibration board to move multiple times by the industrial robot, and obtain the two-dimensional image and three-dimensional point cloud of the circle dot calibration board by the three-dimensional vision camera, and simultaneously obtain the joint angle values of the industrial robot.

[0009] 2) Establish a calibration board coordinate system of the circle dot calibration board, and obtain the homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board relative to the vision coordinate system of the three-dimensional vision camera according to the two-dimensional image and three-dimensional point cloud of each circle dot calibration board.

[0010] 3) Establish an industrial robot kinematics model of the industrial robot, identify the industrial robot kinematics model to obtain an identified industrial robot kinematics model, input the joint angle values of the industrial robot in step 1) into the identified industrial robot kinematics model, calculate to obtain the pose parameters of the industrial robot, and obtain the homogeneous transformation matrix of the vision coordinate system of the three-dimensional vision camera relative to the base coordinate system of the industrial robot according to the pose parameters of the industrial robot and the homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board relative to the vision coordinate system of the three-dimensional vision camera in step 2).

[0011] 4) Remove the circle dot calibration board, install an end effector on the end of the industrial robot, drive the end effector to move multiple times to the same preset position with different poses by the industrial robot, and obtain the joint angle values of the industrial robot each time when the end effector moves to the preset position, so as to calculate the coordinate transformation of the tool center point of the end effector relative to the flange coordinate system of the industrial robot.

[0012] 5) Place a flat plate on a single-axis turntable, drive the flat plate to rotate multiple times by the single-axis turntable while changing the inclination angle of the flat plate, and use a three-dimensional vision camera to obtain the point cloud data of the flat plate, so as to establish a turntable coordinate system of the single-axis turntable, and establish the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable relative to the vision coordinate system of the three-dimensional vision camera according to the point cloud data of the flat plate; according to the homogeneous transformation matrix of the vision coordinate system of the three-dimensional vision camera relative to the base coordinate system of the industrial robot in step 3) and the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable relative to the vision coordinate system of the three-dimensional vision camera, calculate to obtain the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable relative to the base coordinate system of the industrial robot, and finally complete the calibration of the homogeneous transformation matrix between the turntable coordinate system of the single-axis turntable, the vision coordinate system of the three-dimensional vision camera and the base coordinate system of the industrial robot.

[0013] 6) placing the workpiece to be processed on the single-axis turntable, rotating the workpiece to be processed through the single-axis turntable, and simultaneously capturing the workpiece to be processed from various angles through the three-dimensional vision camera in the rotating process, the single-axis turntable not rotating when the three-dimensional vision camera is capturing, thereby performing three-dimensional reconstruction on the workpiece to be processed to obtain a three-dimensional model of the workpiece to be processed and acquire processing characteristics of the workpiece to be processed, generating a processing path of the industrial robot according to the three-dimensional model of the workpiece to be processed and the processing characteristics, the coordinate transformation of the tool center point of the end effector relative to the flange coordinate system of the industrial robot, and the homogeneous transformation matrix between the turntable coordinate system of the single-axis turntable, the vision coordinate system of the three-dimensional vision camera, and the base coordinate system of the industrial robot, thereby realizing the processing operation of the workpiece to be processed by the industrial robot three-dimensional vision workstation.

[0014] In the step 1), the single-axis turntable, the three-dimensional vision camera and the industrial robot are all installed on the workstation base, and the area above the single-axis turntable is the mounting area of the workpiece to be processed, that is, the overlapping area of the field of view of the three-dimensional vision camera and the working space of the industrial robot.

[0015] In the step 1), the surface of the circle point calibration board is set as white dots on black background, the circle point calibration board is moved multiple times by the industrial robot, and each time the circle point calibration board is moved to the field of view of the three-dimensional vision camera and then stopped, the position and angle of the circle point calibration board are different each time, and each white dot on the circle point calibration board is captured by the three-dimensional vision camera, the two-dimensional image and the three-dimensional point cloud of the circle point calibration board, that is, the two-dimensional image and the three-dimensional point cloud of each white dot on the circle point calibration board, are acquired by the three-dimensional vision camera, and the joint angle values of the industrial robot at the time of stopping are also acquired.

[0016] In the step 2), the calibration board coordinate system of the circle point calibration board is specifically a Cartesian coordinate system established with the center coordinate of the outermost white dot on one of the top corners of the circle point calibration board as the origin, the length direction of the edge of the circle point calibration board on the two sides of the origin as the X-axis and Y-axis directions, and the Z-axis direction determined according to the right-hand rule.

[0017] In the step 2), the homogeneous transformation matrix of the calibration board coordinate system of the circle point calibration board relative to the vision coordinate system of the three-dimensional vision camera is acquired according to the two-dimensional image and the three-dimensional point cloud of each circle point calibration board, specifically, for the two-dimensional image and the three-dimensional point cloud of each white dot on the circle point calibration board acquired each time, the center pixel coordinates of each white dot on the circle point calibration board in the image coordinate system of the two-dimensional image are identified, then each center pixel coordinate is mapped to the three-dimensional point cloud to acquire the spatial coordinates of the center of each white dot on the circle point calibration board relative to the three-dimensional vision camera, and finally the homogeneous transformation matrix of the calibration board coordinate system of the circle point calibration board relative to the vision coordinate system of the three-dimensional vision camera is calculated and obtained.

[0018] In the step 3), the kinematics model of the industrial robot is identified, specifically, the nonlinear least squares algorithm is used for identification, and the specific process is as follows:

[0019]

[0020] Wherein, f Rob () represents the value of the loss function, MDH represents the kinematics model of the industrial robot, including the kinematics parameters of the industrial robot; And Respectively represent the homogeneous transformation matrix of the flange coordinate system of the industrial robot relative to the base coordinate system of the industrial robot obtained by the i and j times movement of the industrial robot, i≠j; Represents the homogeneous transformation of the calibration plate coordinates of the circle point calibration plate relative to the flange coordinate system of the industrial robot; And Respectively represent the homogeneous transformation matrix of the calibration plate coordinate system of the circle point calibration plate relative to the vision coordinate system of the three-dimensional vision camera obtained by the i and j times movement of the industrial robot; ‖·‖ represents the Euclidean distance.

[0021] In the step 3), the joint angle values of the industrial robot in the step 1) are input into the identified kinematics model of the industrial robot in turn, and then the pose parameters of the industrial robot are calculated by using the forward kinematics formula, and the eye-in-hand hand-eye calibration is carried out according to the pose parameters of the industrial robot and the homogeneous transformation matrix of the calibration plate coordinate system of the circle point calibration plate relative to the vision coordinate system of the three-dimensional vision camera in the step 2), to obtain the homogeneous transformation matrix of the vision coordinate system of the three-dimensional vision camera relative to the base coordinate system of the industrial robot.

[0022] In the step 4), the end effector is moved to the same preset position by the industrial robot, specifically, the tool center point of the end effector is moved to the same space point in different attitudes, the joint angle values of the industrial robot are obtained, and then the nonlinear least squares algorithm is used to calculate the coordinate transformation of the tool center point of the end effector relative to the flange coordinate system of the industrial robot.

[0023] The nonlinear least squares algorithm is specifically as follows:

[0024]

[0025] Wherein, f TCP () represents the value of the loss function, x tcp , y tcp And z tcp Respectively represent the relative coordinate values of the x axis, y axis and z axis of the tool center point of the end effector relative to the flange coordinate system of the industrial robot; Let i represent the homogeneous transformation matrix of the flange coordinate system of the industrial robot relative to the base coordinate system of the industrial robot, where i ≠ j; and Let represent the homogeneous transformation matrices of the tool center point of the end effector relative to the robot flange coordinate system obtained during the i-th and j-th movements of the industrial robot, respectively, i≠j; ‖·‖ represents the Euclidean distance.

[0026] In step 5), a flat plate is placed at an angle on a single-axis turntable. First, the tilt angle of the plate is set as the first tilt angle. The plate is rotated multiple times to various rotation angles using the single-axis turntable. A 3D vision camera is used to acquire point cloud data of the plate at each rotation angle. Then, the Random Sample Consensus Algorithm (RANSAC) is used to identify the plane equations of the plate in the point cloud data. A nonlinear least squares algorithm is then used to fit each plane equation, thereby obtaining the first intersection point coordinates P1 of multiple plane equations. The tilt angle of the plate is increased to a second tilt angle, and the same operation is performed on the plate at the first tilt angle to obtain the second intersection point coordinates P2. The vector between the first intersection point coordinates P1 and the second intersection point coordinates P2 is defined as follows: Using the Z-axis as an example, we define the X and Y axes of the Cartesian coordinate system according to the right-hand rule, establish the turntable coordinate system of the single-axis turntable, and thus establish the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable relative to the visual coordinate system of the 3D vision camera.

[0027] The beneficial effects of this invention are:

[0028] 1. This invention proposes a method for calibrating and operating the coordinate system of a 3D vision workstation for industrial robots. It utilizes a 3D vision camera to assist in identifying the kinematic parameters of the industrial robot and also introduces a novel calibration method for the 3D vision camera and single-axis turntable. This invention is simple and easy to use while ensuring calibration accuracy, and can be widely applied to various robot workstations, possessing a very broad market application prospect. It has extremely important practical significance for improving the digitalization and intelligentization level of my country's manufacturing industry.

[0029] 2. This invention establishes a kinematic error model for the industrial robot by calculating the difference between the actual Euclidean distance of a spatial reference point and the theoretical Euclidean distance calibrated by the industrial robot. Finally, a nonlinear least squares algorithm is used to identify the kinematic model of the industrial robot. In this invention, the process of identifying the kinematic parameters of the industrial robot does not require expensive measuring equipment, and the entire calibration process is also very simple.

[0030] 3. The application proposes a novel three-dimensional vision camera and single-axis turntable calibration method, which solves the rotation axis of the single-axis turntable through the multi-plane fitting intersection point method, thereby realizing the coordinate system calibration between the three-dimensional vision camera and the single-axis turntable. This method greatly reduces the calibration difficulty, improves the calibration accuracy, and can also be applied to three-dimensional vision camera and multi-axis turntable applications. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 The flowchart of the implementation of the application is shown in the figure.

[0032] Figure 2 The figure is a schematic diagram of an industrial robot three-dimensional vision workstation.

[0033] In the figure: 1, single-axis turntable, 2, three-dimensional vision camera, 3, industrial robot, 4, workstation base, 5, robot teach pendant. DETAILED DESCRIPTION

[0034] The application will be further described in detail below in combination with the drawings and specific embodiments.

[0035] The industrial robot three-dimensional vision workstation coordinate system calibration and operation method of the application comprises:

[0036] 1) Install a circle point calibration plate on the industrial robot three-dimensional vision workstation composed of a single-axis turntable 1, a three-dimensional vision camera 2, an industrial robot 3 and a workstation base 4, and install the circle point calibration plate at the end of the industrial robot 3; drive the circle point calibration plate to move multiple times by the industrial robot 3, and acquire the two-dimensional image and three-dimensional point cloud of the circle point calibration plate by the three-dimensional vision camera 2, and simultaneously acquire the joint angle values of the industrial robot 3.

[0037] In step 1), the single-axis turntable 1, the three-dimensional vision camera 2 and the industrial robot 3 are all installed on the workstation base 4, and the area above the single-axis turntable 1 is the installation area of the workpiece to be processed, i.e. the overlapping area of the field of view of the three-dimensional vision camera 2 and the working space of the industrial robot 3.

[0038] In step 1), the surface of the circle point calibration plate is set as white dots on black background, the circle point calibration plate is driven to move multiple times by the industrial robot 3, and each time the circle point calibration plate is moved to the field of view of the three-dimensional vision camera 2 and then stopped moving, the position and angle of the circle point calibration plate are different each time, and each white dot on the circle point calibration plate is captured by the three-dimensional vision camera 2, the two-dimensional image and three-dimensional point cloud of each white dot on the circle point calibration plate are acquired by the three-dimensional vision camera 2, and the joint angle values of the industrial robot 3 at the time of stopping moving are simultaneously acquired.

[0039] 2) establish the calibration board coordinate system of the circle dot calibration board, and obtain the homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board relative to the vision coordinate system of the three-dimensional vision camera 2 according to the two-dimensional image and the three-dimensional point cloud of each circle dot calibration board.

[0040] In step 2), the calibration board coordinate system of the circle dot calibration board is specifically a Cartesian coordinate system established with the center coordinate of the outermost white dot at one of the top corners of the circle dot calibration board as the origin, the length direction of the side of the circle dot calibration board on the two sides of the origin as the X-axis and Y-axis directions, and the Z-axis direction determined according to the right-hand rule.

[0041] In step 2), the homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board relative to the vision coordinate system of the three-dimensional vision camera 2 is obtained according to the two-dimensional image and the three-dimensional point cloud of each circle dot calibration board. Specifically, for each two-dimensional image and three-dimensional point cloud of each white dot on the circle dot calibration board obtained each time, the center pixel coordinates of each white dot on the circle dot calibration board in the image coordinate system of the two-dimensional image are identified, and then the center pixel coordinates are mapped to the three-dimensional point cloud to obtain the spatial coordinates of the center of each white dot on the circle dot calibration board relative to the three-dimensional vision camera 2. Finally, the homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board relative to the vision coordinate system of the three-dimensional vision camera 2 is calculated and obtained.

[0042] 3) establish the industrial robot kinematics model of the industrial robot 3, identify the industrial robot kinematics model to obtain the identified industrial robot kinematics model, input the joint angle values of the industrial robot 3 in step 1) into the identified industrial robot kinematics model, calculate the pose parameters of the industrial robot 3, and obtain the homogeneous transformation matrix of the vision coordinate system of the three-dimensional vision camera 2 relative to the base coordinate system of the industrial robot 3 according to the pose parameters of the industrial robot 3 and the homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board relative to the vision coordinate system of the three-dimensional vision camera 2 in step 2).

[0043] In step 3), the industrial robot kinematics model is identified, specifically by using a nonlinear least squares algorithm for identification, as follows:

[0044]

[0045] wherein f Rob represents the value of the loss function, MDH represents the industrial robot kinematics model of the industrial robot 3, including the kinematics parameters of the industrial robot 3; and respectively represent the homogeneous transformation matrix of the flange coordinate system of the industrial robot 3 relative to the base coordinate system of the industrial robot 3 obtained when the industrial robot 3 moves the i-th and j-th times, i≠j; a homogeneous transformation of a calibration board coordinate of a circle dot calibration board relative to a flange coordinate system of the industrial robot 3; and respectively represent a homogeneous transformation matrix of a calibration board coordinate system of a circle dot calibration board obtained when the industrial robot 3 is moved for the i-th and j-th time relative to a vision coordinate system of the three-dimensional vision camera 2; ‖·‖ represents the Euclidean distance.

[0046] In step 3), the joint angle values of the industrial robot 3 in step 1) are sequentially input into the identified industrial robot kinematics model, and then the pose parameters of the industrial robot 3 are calculated using the forward kinematics formula. According to the pose parameters of the industrial robot 3 and the homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board relative to the vision coordinate system of the three-dimensional vision camera 2 in step 2), hand-eye calibration is performed, and a homogeneous transformation matrix of the vision coordinate system of the three-dimensional vision camera 2 relative to the base coordinate system of the industrial robot 3 is obtained.

[0047] 4) Remove the circle dot calibration board, install the end effector on the end of the industrial robot 3, and move the end effector to the same preset position multiple times with different poses by the industrial robot 3. Each time the end effector moves to the preset position, the joint angle values of the industrial robot 3 are obtained, so as to calculate the coordinate transformation of the tool center point of the end effector relative to the flange coordinate system of the industrial robot 3.

[0048] In step 4), the industrial robot 3 drives the end effector to move to the same preset position multiple times, which specifically means that the tool center point of the end effector is moved to the same spatial point with multiple different poses, and the joint angle values of the industrial robot 3 are obtained. Then, a nonlinear least squares algorithm is used to calculate the coordinate transformation of the tool center point of the end effector relative to the flange coordinate system of the industrial robot 3.

[0049] The nonlinear least squares algorithm is specifically as follows:

[0050]

[0051] wherein f TCP () represents the value of the loss function, x tcp , y tcp and z tcp respectively represent the relative coordinate values of the tool center point of the end effector relative to the x-axis, y-axis and z-axis of the flange coordinate system of the industrial robot 3; represents a homogeneous transformation matrix of the flange coordinate system of the industrial robot 3 relative to the base coordinate system of the industrial robot 3, i≠j; and respectively represent the homogeneous transformation matrix of the tool center point of the end effector relative to the robot flange coordinate system obtained by the i-th and j-th movement of the industrial robot 3, i≠j; ‖·‖ represents the Euclidean distance.

[0052] 5) A flat plate is placed on the single-axis turntable 1 at an inclination, the flat plate is rotated multiple times by the single-axis turntable 1, and the point cloud data of the flat plate is acquired by using the three-dimensional vision camera 2, so as to establish the turntable coordinate system of the single-axis turntable 1, and the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable 1 relative to the vision coordinate system of the three-dimensional vision camera 2 is established according to the point cloud data of the flat plate; the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable 1 relative to the base coordinate system of the industrial robot 3 is calculated according to the homogeneous transformation matrix of the vision coordinate system of the three-dimensional vision camera 2 relative to the base coordinate system of the industrial robot 3 in step 3) and the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable 1 relative to the vision coordinate system of the three-dimensional vision camera 2, and finally the calibration of the homogeneous transformation matrix between the turntable coordinate system of the single-axis turntable 1, the vision coordinate system of the three-dimensional vision camera 2 and the base coordinate system of the industrial robot 3 is completed.

[0053] In step 5), a flat plate is placed on the single-axis turntable 1 at an inclination, the inclination angle of the flat plate is first set to a first inclination angle, the flat plate is rotated multiple times to each rotation angle by the single-axis turntable 1, the point cloud data of the flat plate at each rotation angle is acquired by using the three-dimensional vision camera 2, then the random sample consensus algorithm RANSAC is used to identify each plane equation of the flat plate in the point cloud data, and then a nonlinear least squares algorithm is used to fit each plane equation, so as to obtain a first intersection coordinate P1 of the multiple plane equations; the inclination angle of the flat plate is increased to a second inclination angle, and then the same operation of the flat plate at the first inclination angle is performed to obtain a second intersection coordinate P2; the first intersection coordinate P1 is taken as an origin, and a vector between the first intersection coordinate P1 and the second intersection coordinate P2 is taken as a Z axis, the X axis and the Y axis of the Cartesian coordinate system are defined according to the right-hand rule, the turntable coordinate system of the single-axis turntable 1 is established, and the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable 1 relative to the vision coordinate system of the three-dimensional vision camera 2 is established. The X axis and the Y axis of the Cartesian coordinate system are defined according to the right-hand rule, the turntable coordinate system of the single-axis turntable 1 is established, and the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable 1 relative to the vision coordinate system of the three-dimensional vision camera 2 is established.

[0054] 6) Place the workpiece to be processed on the single-axis turntable 1, rotate the workpiece to be processed through the single-axis turntable 1, and simultaneously shoot the workpiece to be processed from various angles through the three-dimensional vision camera 2 during the rotation, the single-axis turntable 1 does not rotate when the three-dimensional vision camera 2 shoots, thereby performing three-dimensional reconstruction on the workpiece to be processed to obtain a three-dimensional model of the workpiece to be processed and acquire the processing characteristics of the workpiece to be processed, generating a processing path of the industrial robot 3 according to the three-dimensional model of the workpiece to be processed and the processing characteristics, the coordinate transformation of the tool center point of the end effector relative to the flange coordinate system of the industrial robot 3, and the homogeneous transformation matrix between the turntable coordinate system of the single-axis turntable 1, the vision coordinate system of the three-dimensional vision camera 2, and the base coordinate system of the industrial robot 3, thereby realizing the processing operation of the workpiece to be processed by the industrial robot three-dimensional vision workstation.

[0055] The specific embodiments of the present application are as follows:

[0056] As shown in the drawings, the industrial robot three-dimensional vision workstation used in the embodiments of the present application is shown in the drawings, and the specific steps of the coordinate system calibration and operation method are as follows: Figure 2 Figure 1 As shown in the drawings, the specific steps of the coordinate system calibration and operation method are as follows:

[0057] 1) A white dot black background circle point calibration board is installed at the end of the industrial robot 3; the circle point calibration board is made of metal (aluminum alloy), and the surface pattern is 4x5 asymmetric white dot black background circle points with a circle center distance of 35mm.

[0058] 2) Teach the industrial robot 3 to move the circle point calibration board into the field of view of the three-dimensional vision camera 2, record the joint angle values of the industrial robot 3 at this time, and operate the three-dimensional vision camera 2 to obtain two-dimensional images and three-dimensional point clouds; the three-dimensional camera has been aligned with the two-dimensional image and three-dimensional point cloud data, and each pixel in the two-dimensional image and each point in the three-dimensional point cloud are one-to-one corresponding; repeat the above operation a total of 35 times to obtain 35 groups of two-dimensional images and three-dimensional point clouds and corresponding joint angle values of the industrial robot 3.

[0059] 3) For each two-dimensional image and three-dimensional point cloud, first identify the circle center pixel coordinates of each circle point on the circle point calibration board in the two-dimensional image, then map the circle center pixel coordinates to the three-dimensional point cloud to obtain the spatial coordinates of each circle center relative to the three-dimensional vision camera 2; establish a calibration board coordinate system of the circle point calibration board, and calculate the homogeneous transformation matrix of the calibration board coordinate system of the circle point calibration board relative to the vision coordinate system of the three-dimensional vision camera 2.

[0060] 4) According to the configuration of the industrial robot 3, a five-parameter industrial robot kinematics model is established, denoted as MDH, as shown in Table 1:

[0061] Table 1 Five-parameter industrial robot kinematics parameter table

[0062] ​

[0063] In the table, i represents the serial number of the robot link, a i-1 represents the length of the common perpendicular line between axis i-1 and axis i, a i-1 represents the angle of axis i-1 around a i-1 turning to axis i, d i represents the angle between a i-1 and the intersection point of axis i, pointing to a i represents the length of the intersection point of axis i, θ i represents the angle between a i-1 and axis i, turning to a i , β i represents the offset between two parallel axes.

[0064] High-precision industrial robot kinematics parameters are obtained by optimization using a nonlinear least squares algorithm. The high-precision five-parameter industrial robot kinematics parameters obtained by identification are shown in Table 2.

[0065] Table 2 High-precision five-parameter industrial robot kinematics parameter table

[0066]

[0067] 5) The joint angle values of each group of industrial robot 3 recorded in step 2) are sequentially brought into the high-precision industrial robot kinematics parameters obtained by identification in step 4), and the forward kinematics formula is used to calculate the pose parameters of industrial robot 3; according to the pose parameters of industrial robot 3 and the homogeneous transformation matrix of the circular dot calibration board relative to the three-dimensional vision camera 2, eye-in-hand hand-eye calibration is implemented to obtain the homogeneous transformation matrix of the three-dimensional vision camera relative to the industrial robot base coordinate system, denoted as Specifically as follows:

[0068]

[0069] 6) Remove the circular dot calibration board installed at the end of the industrial robot 3, install the end effector of the industrial robot 3, and set the tool center point of the end effector; teach the industrial robot 3 to move the tool center point to the same spatial point in 5 poses, and record the joint angle values of the industrial robot 3 when the tool center point moves to the same spatial point in different poses.

[0070] 7) Obtain the tool center point position parameters of the industrial robot 3 by optimization using a nonlinear least squares algorithm. The tool center point position parameters x tcp , y tcp , and z tcp of the industrial robot 3 obtained by solving are -20.84, 0.16, and -113.89, respectively.

[0071] 8) A flat plate is placed on the single-axis turntable 1 at an angle, the single-axis turntable 1 is rotated to 5 different rotation angles, and the point cloud data of the flat plate at these angles is collected by the three-dimensional vision camera 2; the plane equation of the flat plate in each point cloud data is identified by the random sample consensus algorithm RANSAC, and then the 5 plane equations are fitted by the nonlinear least squares algorithm to obtain the intersection coordinates of multiple planes, denoted as P1; the inclination angle of the flat plate is increased, and the above operation is repeated to obtain the second intersection coordinates, denoted as P2; the P1 is taken as the origin, and the vector is the Z axis, the X axis and the Y axis of the Cartesian coordinate system are defined according to the right-hand rule, the homogeneous transformation matrix of the single-axis turntable 1 relative to the three-dimensional vision camera 2 is established, denoted as Specifically as follows:

[0072]

[0073] 9) According to the homogeneous transformation matrix of the three-dimensional vision camera 2 relative to the base coordinate system of the industrial robot 3 and the homogeneous transformation matrix of the single-axis turntable 1 relative to the three-dimensional vision camera 2 The homogeneous transformation matrix of the single-axis turntable 1 relative to the base coordinate system of the industrial robot 3 is calculated, denoted as Specifically as follows:

[0074]

[0075] After the coordinate system calibration is completed, the precision verification experiment is carried out on the industrial robot three-dimensional vision work station as shown in Figure 2 The verification accuracy is about 1.0mm, which meets the industrial application requirements of robot welding and gluing.

Claims

1. A method for calibration and operation of a coordinate system of a three-dimensional vision station of an industrial robot, characterized in that, The method comprises the following steps: 1) installing a circle dot calibration board on an industrial robot three-dimensional vision work station composed of a single-axis turntable (1), a three-dimensional vision camera (2), an industrial robot (3) and a work station base (4), the circle dot calibration board being installed at the end of the industrial robot (3); driving the circle dot calibration board to move multiple times by the industrial robot (3), and acquiring two-dimensional images and three-dimensional point clouds of the circle dot calibration board by the three-dimensional vision camera (2), and simultaneously acquiring joint angle values of the industrial robot (3); 2) establishing a calibration board coordinate system of the circle dot calibration board, and acquiring a homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board relative to a vision coordinate system of the three-dimensional vision camera (2) according to the two-dimensional images and the three-dimensional point clouds of each circle dot calibration board; 3) establishing an industrial robot kinematics model of the industrial robot (3), identifying the industrial robot kinematics model to obtain an identified industrial robot kinematics model, inputting the joint angle values of the industrial robot (3) in step 1) into the identified industrial robot kinematics model, calculating to obtain pose parameters of the industrial robot (3), and obtaining a homogeneous transformation matrix of the vision coordinate system of the three-dimensional vision camera (2) relative to a base coordinate system of the industrial robot (3) according to the pose parameters of the industrial robot (3) and the homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board relative to the vision coordinate system of the three-dimensional vision camera (2) in step 2); 4) removing the circle dot calibration board, installing an end effector at the end of the industrial robot (3), and driving the end effector to move to a same preset position multiple times in different postures by the industrial robot (3), and acquiring joint angle values of the industrial robot (3) each time when the end effector moves to the preset position, so as to calculate a coordinate transformation of a tool center point of the end effector relative to a flange coordinate system of the industrial robot (3); 5) placing a flat plate on the single-axis turntable (1) in an inclined manner, driving the flat plate to rotate multiple times by the single-axis turntable (1) while changing the inclination angle of the flat plate, and acquiring point cloud data of the flat plate by the three-dimensional vision camera (2), so as to establish a turntable coordinate system of the single-axis turntable (1), and according to the point cloud data of the flat plate, to establish a homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable (1) relative to the vision coordinate system of the three-dimensional vision camera (2); according to the homogeneous transformation matrix of the vision coordinate system of the three-dimensional vision camera (2) relative to the base coordinate system of the industrial robot (3) in step 3) and the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable (1) relative to the vision coordinate system of the three-dimensional vision camera (2), a homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable (1) relative to the base coordinate system of the industrial robot (3) is calculated, and finally the calibration of the homogeneous transformation matrix among the turntable coordinate system of the single-axis turntable (1), the vision coordinate system of the three-dimensional vision camera (2) and the base coordinate system of the industrial robot (3) is completed. 6) placing the workpiece to be processed on the single-axis turntable (1), rotating the workpiece to be processed through the single-axis turntable (1), and simultaneously shooting the workpiece to be processed from various angles through the three-dimensional vision camera (2) during the rotation, the single-axis turntable (1) not rotating when the three-dimensional vision camera (2) is shooting, thereby performing three-dimensional reconstruction on the workpiece to be processed to obtain a three-dimensional model of the workpiece to be processed and to obtain processing characteristics of the workpiece to be processed, generating a processing path of the industrial robot (3) according to the three-dimensional model and the processing characteristics of the workpiece to be processed, a coordinate transformation of a tool center point of an end effector relative to a flange coordinate system of the industrial robot (3), and a homogeneous transformation matrix between a turntable coordinate system of the single-axis turntable (1), a vision coordinate system of the three-dimensional vision camera (2), and a base coordinate system of the industrial robot (3), and thereby realizing the processing operation of the workpiece to be processed by the industrial robot three-dimensional vision workstation.

2. The industrial robot 3D vision station coordinate system calibration and operation method of claim 1, wherein: In the step 1), the single-axis turntable (1), the three-dimensional vision camera (2), and the industrial robot (3) are all installed on the workstation base (4), and the upper area of the single-axis turntable (1) is the mounting area of the workpiece to be processed, that is, the overlapping area of the field of view of the three-dimensional vision camera (2) and the working space of the industrial robot (3).

3. The industrial robot 3D vision workcell coordinate system calibration and operation method of claim 1, wherein: In the step 1), the surface of the circular dot calibration board is set as white dots on a black background, the circular dot calibration board is moved multiple times by the industrial robot (3), and each time the circular dot calibration board is moved to the field of view of the three-dimensional vision camera (2) and then stopped, the position and angle of the circular dot calibration board are different each time, and each white dot on the circular dot calibration board is shot by the three-dimensional vision camera (2), the two-dimensional image and the three-dimensional point cloud of the circular dot calibration board, that is, the two-dimensional image and the three-dimensional point cloud of each white dot on the circular dot calibration board, are obtained by the three-dimensional vision camera (2), and the joint angle values of the industrial robot (3) at the time of stopping are simultaneously obtained.

4. The industrial robot 3D vision station coordinate system calibration and operation method of claim 3, wherein: In the step 2), the calibration board coordinate system of the circular dot calibration board is specifically a Cartesian coordinate system established by taking the center coordinate of the outermost white dot at one of the top corners of the circular dot calibration board as the origin, taking the length direction of the edge of the circular dot calibration board on the two sides of the origin as the X-axis and Y-axis directions, and determining the Z-axis direction according to the right-hand rule.

5. The industrial robot 3D vision workcell coordinate system calibration and operation method of claim 3, wherein: In the step 2), the homogeneous transformation matrix of the calibration board coordinate system of the circular dot calibration board relative to the vision coordinate system of the three-dimensional vision camera (2) is obtained according to the two-dimensional image and the three-dimensional point cloud of each circular dot calibration board, specifically, for the two-dimensional image and the three-dimensional point cloud of each white dot on the circular dot calibration board obtained each time, the center pixel coordinates of each white dot on the circular dot calibration board in the image coordinate system of the two-dimensional image are identified, then each center pixel coordinate is mapped to the three-dimensional point cloud to obtain the center space coordinates of each white dot on the circular dot calibration board relative to the three-dimensional vision camera (2), and finally the homogeneous transformation matrix of the calibration board coordinate system of the circular dot calibration board relative to the vision coordinate system of the three-dimensional vision camera (2) is calculated and obtained.

6. The industrial robot 3D vision workcell coordinate system calibration and operation method of claim 1, wherein: In the step 3), the industrial robot kinematic model is identified, specifically by using a nonlinear least squares algorithm, and specifically as follows: wherein f Rob () represents the value of the loss function, MDH represents an industrial robot kinematic model of the industrial robot (3) comprising kinematic parameters of the industrial robot (3); and respectively represent the homogeneous transformation matrix of the flange coordinate system of the industrial robot (3) with respect to the base coordinate system of the industrial robot (3) obtained at the i-th and j-th movement of the industrial robot (3), respectively, i≠j; represents the homogeneous transformation of the calibration plate coordinates of the circle dot calibration plate with respect to the flange coordinate system of the industrial robot (3); and respectively represent the homogeneous transformation matrix of the calibration plate coordinate system of the circle dot calibration plate with respect to the vision coordinate system of the three-dimensional vision camera (2) obtained at the i-th and j-th movement of the industrial robot (3), respectively; ||·|| represents the Euclidean distance.

7. The industrial robot 3D vision workcell coordinate system calibration and operation method of claim 1, wherein: In the step 3), the joint angle values of the industrial robot (3) in the step 1) are sequentially input into the identified industrial robot kinematics model, then the pose parameters of the industrial robot (3) are calculated by using the forward kinematics formula, and according to the pose parameters of the industrial robot (3) and the homogeneous transformation matrix of the calibration board coordinate system of the circle dot calibration board in the step 2) relative to the vision coordinate system of the three-dimensional vision camera (2), the hand-eye calibration is implemented, and the homogeneous transformation matrix of the vision coordinate system of the three-dimensional vision camera (2) relative to the base coordinate system of the industrial robot (3) is obtained.

8. The industrial robot 3D vision workcell coordinate system calibration and operation method of claim 1, wherein: In the step 4), the end effector is moved to the same preset position by the industrial robot (3) for multiple times, specifically, the tool center point of the end effector is moved to the same space point in multiple different poses, the joint angle values of the industrial robot (3) are obtained, and then the coordinate transformation of the tool center point of the end effector relative to the flange coordinate system of the industrial robot (3) is calculated by using the nonlinear least square algorithm.

9. The industrial robot 3D vision workcell coordinate system calibration and operation method of claim 8, wherein: The nonlinear least square algorithm is specifically as follows: wherein f TCP represents the value of the loss function, x tcp , y tcp and z tcp represent the relative coordinate values of the tool center point of the end effector with respect to the x-axis, y-axis and z-axis of the flange coordinate system of the industrial robot (3), respectively; represents the homogeneous transformation matrix of the flange coordinate system of the industrial robot (3) with respect to the base coordinate system of the industrial robot (3), i≠j; and represent the homogeneous transformation matrix of the tool center point of the end effector with respect to the robot flange coordinate system obtained at the i-th and j-th motion of the industrial robot (3), i≠j, respectively; ||·|| represents the Euclidean distance.

10. The industrial robot 3D vision workcell coordinate system calibration and operation method of claim 1, wherein: In the step 5), a flat plate is obliquely placed on the single-axis turntable (1), the oblique angle of the flat plate is first set as a first oblique angle, the flat plate is rotated to various rotation angles for multiple times by the single-axis turntable (1), the point cloud data of the flat plate at various rotation angles are acquired by using the three-dimensional vision camera (2), then the random sampling consensus algorithm RANSAC is used to identify various plane equations of the flat plate in the point cloud data, then the nonlinear least square algorithm is used to fit various plane equations, so that the first intersection coordinates P1 of the multiple plane equations are obtained; the oblique angle of the flat plate is increased to a second oblique angle, then the same operation of the flat plate at the first oblique angle is performed, the second intersection coordinates P2 are obtained; the first intersection coordinates P1 is taken as an origin, the vector between the first intersection coordinates P1 and the second intersection coordinates P2 is taken as a Z axis, the X axis and the Y axis of the Cartesian coordinate system are defined according to the right-hand rule, the turntable coordinate system of the single-axis turntable (1) is established, so that the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable (1) relative to the vision coordinate system of the three-dimensional vision camera (2) is established. The X axis and the Y axis of the Cartesian coordinate system are defined according to the right-hand rule, the turntable coordinate system of the single-axis turntable (1) is established, so that the homogeneous transformation matrix of the turntable coordinate system of the single-axis turntable (1) relative to the vision coordinate system of the three-dimensional vision camera (2) is established.

Images