Robot disordered grasping method based on point cloud normal vector and coordinate system simulation

Abstract

The present invention relates to a method for the robot to perform unordered grasping based on the simulation of the normal vector and coordinate system of the target object point cloud. A coordinate system is simulated at the grasping point position of the target object point cloud. The coordinate system takes the grasping point of the target object point cloud as the origin. During this process, the conversion relationship from the point cloud coordinate system to the 3D camera coordinate system can be known, that is, a pose at the grasping point of the target point cloud can be provided for the robot. At the same time, the present invention uses the method of obstacle avoidance with the normal vector, so that the end of the robot first moves in a straight line in a posture to the normal direction of the fitting plane where the grasping point of the target object is located, and then the end moves vertically towards the grasping point in a straight line in a posture until it reaches the grasping point position. That is to say, the present invention first determines the coordinate system of the grasping point of the target object, and then makes the end of the robot reach this position in two sections, and can quickly complete an unordered grasping once.

Description

Technical Field

[0001] The present invention relates to the technical field of robots, and particularly to a method for a robot not to grasp based on the point cloud vector of an object and coordinate system simulation, which is applicable to a six-degree-of-freedom robot equipped with a binocular 3D camera. Background Art

[0002] Point cloud is the main information product when a 3D vision camera works. Point cloud is a set of points after the three-dimensional reconstruction of the surface of an object in the field of view. The normal vector is an important attribute of the point cloud, and its function is to indicate the vertical direction of the plane fitted with each point on the point cloud as the center.

[0003] An industrial robot equipped with a 3D camera is an important way to achieve unordered grasping. The current common matching scheme is to place the 3D camera on the robot axis body or place the 3D camera separately from the robot. Both schemes require hand-eye calibration to obtain the hand-eye matrix, and the function of the hand-eye matrix is to realize the transformation between the 3D camera coordinate system and the robot base coordinate system. Usually, the transformation relationship between the robot end coordinate system and the robot base coordinate system is determined during the factory calibration of the robot. Therefore, the conversion relationship between the robot end coordinate system and the 3D camera coordinate system can be known. The three-dimensional coordinate information of each point in the point cloud is relative to the 3D camera coordinate system. The coordinate of a certain point on the point cloud can be transformed to the robot base coordinate system through the hand-eye matrix, and then transformed from the robot base coordinate system to the robot end coordinate system, and the robot end can move to this position to achieve the effect of visual positioning.

[0004] In the unordered grasping of a robot, it is necessary to locate the position of the object and determine the posture at the end of grasping. Therefore, there are two problems in the current unordered grasping of robots. One is that it is necessary to specify the grasping pose of the robot end with respect to the object, and the other is how to avoid obstacles so as to prevent the robot from colliding with the objects around the object during the grasping process. Summary of the Invention

[0005] Aiming at the problems existing in the prior art, the purpose of the present invention is to provide a method for a robot not to grasp based on the normal vector of the object point cloud and coordinate system simulation, so as to realize the fast unordered grasping of the robot and effectively avoid obstacles during the unordered grasping process.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is:

[0007] A method for unordered grasping of a robot based on the normal vector of the object point cloud and coordinate system simulation, the method comprising the following steps:

[0008] Step 1, perform hand-eye calibration on the 3D camera and the robot to obtain the transformation relationship from the 3D camera coordinate system to the robot base coordinate system, that is, the hand-eye matrix H;

[0009] Step 2: Place a target object under the 3D camera. The 3D camera takes a picture to obtain the point cloud of the target object, which is used as the template point cloud.

[0010] Step 3: Determine a point on the template point cloud as the grasping point and obtain the normal vector of this point.

[0011] Step 4: Manually move the end of the robot to this point to grasp the target object, record the pose of the end of the robot at this time, and record this pose as the initial grasping pose.

[0012] Step 5: Invert the hand-eye matrix H obtained after hand-eye calibration to get H -1 , and transform the initial grasping pose through H -1 to obtain the pose in the 3D camera coordinate system. This pose represents transforming the 3D camera coordinate system to the template point cloud, that is, simulating a coordinate system on the template point cloud with the grasping point as the origin.

[0013] Step 6: The 3D camera takes pictures of the next object of the same type, and uses the template point cloud to perform point cloud matching on the obtained point cloud to obtain the pose transformation relationship between the two point clouds.

[0014] Step 7: Transform the simulated coordinate system on the template point cloud through the pose transformation relationship between the two point clouds to the next target point cloud, then transform it to the robot base coordinate system through the hand-eye matrix, and finally transform it to the robot end coordinate system to obtain the new target grasping pose.

[0015] Step 8: Determine an obstacle avoidance height h, add h along the normal vector to the coordinates of the new target grasping pose, e2 = h×e normal + e1; where, O camera -x-y is the 3D camera coordinate system, e1 is the vector coordinate corresponding to the grasping pose of the new target point cloud, e normal is the normal vector, and e2 is the vector coordinate after adding the normal direction h; when grasping, first send the pose corresponding to e2, and then send the target grasping pose corresponding to e1. The robot will first reach the vertical direction of the target grasping point and then fall along the vertical direction to reach the target grasping point to grasp the object, completing one grasping.

[0016] Step 9: Repeat steps 6 - 8 to complete unordered grasping.

[0017] After adopting the above solution, the present invention simulates a coordinate system at the position of the grasping point of the target object point cloud. This coordinate system takes the grasping point of the target object point cloud as the origin. During this process, the conversion relationship between this point cloud coordinate system and the 3D camera coordinate system can be known, that is, a pose at the grasping point of the target point cloud can be provided for the robot. At the same time, the present invention uses the method of obstacle avoidance with the normal vector, so that the end of the robot first moves in a straight line in the pose to the normal direction of the fitting plane where the grasping point of the target object is located, and then the end moves vertically towards the grasping point in a straight line in the pose until it reaches the grasping point position. That is to say, the present invention first determines the coordinate system of the grasping point of the target object, and then makes the end of the robot reach this position in two sections, and can quickly complete an unordered grasping. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flowchart of the method of the present invention;

[0019] Figure 2 is a schematic diagram of the coordinates when the present invention grasps. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] As Figure 1 shown, the present invention discloses a method for unordered grasping of a robot based on the normal vector and coordinate system simulation of a target object point cloud, which includes the following steps:

[0021] Step 1: Perform hand-eye calibration on the 3D camera and the robot to obtain the transformation relationship from the 3D camera coordinate system to the robot base coordinate system - the hand-eye matrix H. Hand-eye calibration is a mature existing technology, and in this embodiment, the hand-eye calibration can be implemented by using existing methods.

[0022] Step 2: Place a target object under the 3D camera, and the 3D camera takes a picture to obtain the point cloud of the target object as the template point cloud;

[0023] Step 3: Determine a point on the template point cloud as the grasping point, and obtain the normal vector of this point;

[0024] Step 4: Manually move the end of the robot to this point to grasp the target object, record the pose of the end of the robot at this time, and record this pose as the initial grasping pose;

[0025] Step 5: Invert the hand-eye matrix H obtained after hand-eye calibration to obtain H -1 , and transform the initial grasping pose through H -1 to obtain the pose in the 3D camera coordinate system. This pose represents converting the 3D camera coordinate system to the template point cloud, that is, simulating a coordinate system with the grasping point as the origin on the template point cloud.

[0026] Step 6: The 3D camera captures the next object of the same type, and uses the template point cloud to perform point cloud matching on the obtained point cloud to obtain the pose transformation relationship between the two point clouds. This step belongs to the prior art and will not be elaborated here.

[0027] Step 7: Transform the coordinate system simulated on the template point cloud to the next target point cloud through the pose transformation relationship between the two point clouds, then convert it to the robot base coordinate system through the hand-eye matrix, and finally transform it to the robot end coordinate system to obtain the new target grasping pose.

[0028] Step 8: Determine an obstacle avoidance height h, and add h along the normal vector to the coordinates of the new target grasping pose, as Figure 2 shown, e2 = h×e normal +e1, ( Figure 2 In camera -x-y is the 3D camera coordinate system, e1 is the vector coordinate corresponding to the new target point cloud grasping pose, e normal is the normal vector, and e2 is the vector coordinate after adding the normal direction h). When grasping, first send the pose corresponding to e2, and then send the target grasping pose corresponding to e1. The robot will first reach the vertical direction of the target grasping point, and then fall along the vertical direction to reach the target grasping point to grasp the object, completing one grasping. The obstacle avoidance height h is a set value and can be set according to specific usage requirements. In this embodiment, the obstacle avoidance height h is set to 50mm.

[0029] Step 9: Repeat steps 6-8 to complete unordered grasping.

[0030] In the method of the present invention, hand-eye calibration, point cloud template matching, etc. belong to the prior art.

[0031] In the present invention, a coordinate system is simulated at the position of the target object point cloud grasping point. This coordinate system takes the target object point cloud grasping point as the origin. During this process, the conversion relationship from this point cloud coordinate system to the 3D camera coordinate system can be known, that is, a pose at the target point cloud grasping point can be provided for the robot. At the same time, the method of using the normal vector for obstacle avoidance in the present invention enables the robot end to first move in a straight line in the pose to the normal direction of the fitting plane where the target object grasping point is located, and then the end moves vertically in a straight line pose towards the grasping point until reaching the grasping point position. That is to say, the present invention first determines the coordinate system of the target object grasping point, and then divides it into two segments to make the robot end reach this position, and can quickly complete an unordered grasping.

[0032] The above is only an embodiment of the present invention, and does not impose any limitation on the technical scope of the present invention. Therefore, any minor modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention still belong to the scope of the technical solution of the present invention.

Claims

1. A robot disordered grasping method based on the simulation of the normal vector of the target object point cloud and the coordinate system, characterized in that: The method includes the following steps: Step 1: Perform hand-eye calibration on the 3D camera and the robot to obtain the transformation relationship from the 3D camera coordinate system to the robot base coordinate system, that is, the hand-eye matrix H; Step 2: Place a target object under the 3D camera, and the 3D camera takes a picture to obtain the point cloud of the target object as the template point cloud; Step 3: Determine a point on the template point cloud as the grasping point and obtain the normal vector of this point; Step 4: Manually move the robot end to this point to grasp the target object, record the pose of the robot end at this time, and record this pose as the initial grasping pose; Step 5: Invert the hand-eye matrix H obtained after hand-eye calibration to get H -1 , and transform the initial grasping pose through H -1 to obtain the pose in the 3D camera coordinate system. This pose represents the conversion of the 3D camera coordinate system to the template point cloud, that is, simulating a coordinate system with the grasping point as the origin on the template point cloud; Step 6: The 3D camera takes pictures of the next object of the same type, and uses the template point cloud to perform point cloud matching on the obtained point cloud to obtain the pose transformation relationship between the two point clouds; Step 7: Transform the simulated coordinate system on the template point cloud to the next target point cloud through the pose transformation relationship between the two point clouds, then convert it to the robot base coordinate system through the hand-eye matrix, and finally transform it to the robot end coordinate system to obtain the new target point cloud grasping pose; Step 8: Determine an obstacle avoidance height h, add h to the coordinates of the new target point cloud grasping pose along the normal vector, e2 = h × e normal + e1; where e1 is the vector coordinate corresponding to the new target point cloud grasping pose, e normal is the normal vector, and e2 is the vector coordinate obtained by adding h to e1 along the normal direction; during grasping, first send the pose corresponding to e2, and then send the target grasping pose corresponding to e1. The robot will first reach the vertical direction of the target grasping point and then fall along the vertical direction to reach the target grasping point to grasp the object, completing one grasping; Step 9: Repeat steps 6-8 to complete unordered grasping.

Images