A method for image recognition and capture of manipulator parts based on kinect sensor

Abstract

The invention discloses a recognizing and grabbing method of an image of a mechanical arm part based on a Kinect sensor. The recognizing and grabbing method comprises the steps of calibrating and registering a color camera and an infrared camera of the Kinect sensor by adopting a positive-friend calibration method; carrying out hand-eye calibration on a mechanical arm and the Kinect sensor, and solving by adopting a Tsai two-step method to obtain a hand-eye transformation matrix; preprocessing a point cloud image of the part to be grabbed obtained by the Kinect sensor to obtain an RGD image, pre-establishing a grid grabbing position detection model based on the convolution neural network method, and inputting the RGD image of the part to be grabbed to obtain the grabbing position of the part to be grabbed in the image space; and according to the hand-eye transformation matrix and the mechanical arm inverse solution algorithm, mapping the grabbing position in the image space into the grabbing position and joint angle of the mechanical arm, and controlling the mechanical arm to move to the target position and the target joint angle according to tracks to execute the grabbing task. According to the recognizing and grabbing method, the feasible grabbing position is detected from the RGB-D image of the part, the speed is high, the result is reliable, the generalization performance of the detection model is good, and the stability is high.

Description

technical field [0001] The invention belongs to the field of industrial robots, relates to a part grasping position image detection model based on a convolutional neural network, and develops a grasping method for manipulator image processing based on a Kinect sensor. Background technique [0002] With the development of digitalization and intelligence in machinery manufacturing, industrial robots have become the mainstream equipment for automated production lines in industries such as construction machinery, electronic appliances, and the automobile industry. As the most widely used form of equipment for industrial robots, the mechanical arm is a mechanical structure formed by a series of connecting rods connected through joints. It can perform various functions based on its own power and control capabilities according to instructions. The robotic arm usually has 3 or more degrees of freedom and can be divided into three parts: arm, wrist, and end effector. The end effector...

AI Technical Summary

Problems solved by technology

For vision-based robotic arm grasping tasks, at present, manual design and image feature extraction methods are mostly used, and the target objects and operating environments are relatively single. Influenced by uncertain factors such as shooting angle and external lighting changes, the algorithm has poor robustness and is not easy to reuse

Images