Two-dimensional target detection-based point cloud recognition and segmentation method, device and system for workpiece most suitable for grabbing

Abstract

The invention discloses a two-dimensional target detection-based point cloud recognition and segmentation method, device and system for the workpiece most suitable for grabbing. The method comprises the steps of: carrying out two-dimensional target detection of a color image, and outputting the confidence coefficients of all detected workpieces and the positions of bounding boxes to which the workpieces belong based on a target detection result; selecting the workpiece with the maximum confidence coefficient as the workpiece most suitable for grabbing; according to the position parameter of the bounding box to which the workpiece with the maximum confidence coefficient belongs, determining and segmenting an area covered by the bounding box on a disparity map to realize two-dimensional positioning and segmentation of the workpiece; and performing three-dimensional reconstruction on the area where the upper bounding box of the disparity map is located, and generating point cloud of the workpiece most suitable for grabbing. According to the invention, the three-dimensional reconstruction technology and the two-dimensional target detection technology are fused, the workpiece most suitable for grabbing in the out-of-order workpieces can be quickly recognized, and the point cloud of the workpiece can be independently reconstructed, so that the situation that the workpiece most suitable for grabbing is recognized through complex global feature matching in the pose estimation of three-dimensional point cloud is avoided, the calculation amount is remarkably reduced, and the speed of recognizing and segmenting the workpiece most suitable for grabbing is accelerated.

Description

technical field [0001] The invention relates to the fields of two-dimensional target detection and point cloud recognition and segmentation, in particular to a method, device and system for point cloud recognition and segmentation most suitable for grasping workpieces based on two-dimensional target detection. Background technique [0002] In recent years, with the development of 3D vision technology, out-of-sequence workpiece grasping technology based on 3D vision has begun to be applied and promoted in the industrial field. In practical applications, out-of-order workpieces are stacked and placed, and the occluded workpieces are not suitable for being grasped by robots. Therefore, the out-of-order workpiece grasping system based on 3D vision is used for pose estimation and grasping Before that, it is necessary to identify the workpiece with the lowest occlusion rate among the out-of-sequence workpieces—the most suitable for grabbing workpieces. [0003] The general recogn...

AI Technical Summary

Problems solved by technology

The 3D point cloud recognition and segmentation technology is a common workpiece point cloud recognition and segmentation technology in the out-of-sequence workpiece grasping system. This technology allows the template point cloud to be registered with all workpieces in the out-of-order workpiece point cloud one by one according to the feature point pairs. The workpiece with the smallest registration error with the template point cloud is identified as the most suitable workpiece to be captured and segmented separately, but the recognition and segmentation method based on the 3D point cloud has a large amount of calculation and slow calculation speed, which is not suitable for rapid identification and segmentation of out-of-sequence workpieces requirements

With the rapid development of deep learning, two-dimensional image recognition and segmentation technologies based on deep learning emerge in endlessly. Among them, the two-dimensional target detection algorithm based on deep convolutional neural network has the performance of independently completing the learning of target features and extracting target feature information. The rapid and high-precision recognition of 2D images of out-of-sequence workpieces, but because the workpiece pose estimation part of the out-of-order workpiece grasping system needs to input the 3D point cloud that is most suitable for grasping the workpiece to calculate the 6 degrees of freedom of the workpiece, therefore, The recognition results of two-dimensional image recognition and segmentation technology cannot be directly applied to the pose estimation of out-of-sequence workpieces

Images