Multimodal perception system of digital control-based collaborative robot

Abstract

The present disclosure discloses a multimodal perception system of digital control-based collaborative robots that belongs to the technical field of collaborative robot control. The present disclosure can implement the perception of multimodal information, including RGB images, depth images, pressure sensors, and use the maximum distance between any two points on the outer edge of the top surface of the workpiece as the dimension characteristic parameter of the workpiece that can effectively ensure the pneumatic gripper at the robot end effector for grasping workpieces to complete the grasping task at the time. Through combining the contour detection technology and the depth camera theory, the multimodal perception system of the present disclosure can cleverly and precisely calculate the distance data of the workpiece, thus to control the corresponding distance for lowering the pneumatic gripper of the robot for grasping workpieces and to ensure a steady operation of the subsequent grasping task. Through the combination of the pressure sensor set installed in the corresponding areas of two gripping plates of the pneumatic gripper and the pre-clamp operation, the multimodal perception system of the present disclosure can precisely identify the deflection angle of the workpiece to be used as an accurate base for subsequent angle adjustments of the pneumatic gripper.

Images