Man, machine and object interaction mechanical arm teaching system based on RGB-D image

Abstract

The invention belongs to the field of robot technology and application and relates to a man, machine and object interaction mechanical arm teaching system based on an RGB-D image. The teaching systemrecognizes an objected on the basis of the RGB-D image. By means of tf tree and Kinect V2 point cloud information in an ROS system, the object and a mechanical arm are unified to the same coordinate to locate the object. According to the human behavior habit, the high level actions of the mechanical arm are planned on the basis of MoveIt. In the teaching process, by selecting the object in an operation interface, the type and pose of the object can be obtained, then one action is selected from the high level actions in a centralized manner, the mechanical arm is controlled to operate the corresponding object in the actual space, and multi-step interaction forms the teaching track. The teaching system can achieve man, machine and object intelligent interaction facing the task level, can replace an actual robot system to carry out teaching study, and the system has the characteristics of being high in efficiency, convenient, safe and the like.

Description

technical field [0001] The invention belongs to the field of robot technology and application, and relates to a human-machine-object interactive manipulator teaching system based on RGB-D images. Background technique [0002] In the process of machine learning, human-computer interaction is usually an effective way. The work related to interaction in robot acquisition skills mainly includes virtual reality (Virtual Reality, VR) teaching and offline programming. Both of the above two methods need to establish a 3D scene model in advance, and the performance of real-time perception and adaptation to changing environments is poor. The RGB-D camera can solve this problem very well. It can directly obtain the depth information of the three-dimensional space and present it in the form of point cloud. It is often used to observe and capture the behavior sequence of the trainer, and then adapt to the Robot joints are the main way for robots to obtain environmental information. [...

AI Technical Summary

Problems solved by technology

High workload when teaching everyday life skills

In addition, the data dimension of the joint action is high, and the training time of the learning model is long, which is easy to cause the phenomenon of "dimension disaster"

At present, there are few studies on directly using RGB-D images as teaching platforms.

Images