The invention discloses a transparent object mechanical arm grabbing method and
system based on semantic geometry fusion
perception, and the method comprises the steps: obtaining an
RGB image and an original depth image of a to-be-grabbed object, carrying out the fusion prediction based on a pre-trained
monocular perception network model, and outputting a segmentation
mask and a restoration depth value of the to-be-grabbed transparent object; performing restoration
processing on the original depth image through the restoration depth value, and outputting a complete three-dimensional
point cloud of the to-be-grabbed object; object grabbing detection is carried out based on the grabbing detection
network model, and a plurality of candidate 6-DoF grabbing postures and confidence degree scores and disturbance tolerance degrees corresponding to the grabbing postures are obtained; and non-maximum suppression is conducted through a
collision test, selection is conducted, and the optimal grabbing posture is determined to achieve grabbing of the transparent object. According to the method, the object grabbing accuracy in a real scene can be improved. The transparent object mechanical arm grabbing method and
system based on semantic and geometric fusion
perception can be widely applied to the technical field of
robot grabbing.