Object six-degree-of-freedom pose estimation method based on color and depth information fusion

Abstract

The invention relates to an object six-degree-of-freedom pose estimation method based on color and depth information fusion. The object six-degree-of-freedom pose estimation method comprises the following steps of: acquiring a color image and a depth image of a target object, and carrying out instance segmentation on the color image; cutting a color image block containing a target object from thecolor image, and acquiring a target object point cloud from the depth image; extracting color features from the color image block, and combining the color features to the target object point cloud atthe pixel level; carrying out point cloud processing on the target object point cloud to obtain a plurality of point cloud local region features fusing the color information and the depth informationand a global feature, and combining the global feature into the point cloud local region features; and predicting the pose and confidence of one target object by means of each local feature, and taking the pose corresponding to the highest confidence as a final estimation result. Compared with the prior art, color information and depth information are combined, the object pose is predicted by combining the local features and the global features, and the object six-degree-of-freedom pose estimation method has the advantages of being high in robustness, high in accuracy rate and the like.

Description

technical field [0001] The invention relates to the field of robot vision, in particular to a six-degree-of-freedom pose estimation method for an object based on fusion of color and depth information. Background technique [0002] Computer vision-based object six-degree-of-freedom pose (a total of six degrees of freedom for the three-dimensional translation and rotation transformation parameters of the object relative to the camera coordinate system) estimation technology enables the robot to perceive the surrounding environment from the three-dimensional level, which is the key to realizing robot grasping and dexterity. The key technology of operation is of great significance to promote the application of service robots and industrial robots. In addition, this technology also has broad application prospects in the fields of autonomous driving, augmented reality, and virtual reality technology. [0003] The existing object pose estimation techniques mainly include the follo...

AI Technical Summary

Problems solved by technology

[0004] The first is the method based on template matching: this method performs feature matching on the image collected online and the template made offline, and determines the pose of the object according to the matched template; the existing methods of this type have the following problems: 1. When objects are mixed and stacked, it is difficult to accurately match the target object, and the robustness is not high; the template matching time will increase sharply with the increase of the number of templates, and it is difficult to meet the real-time requirements

[0005] The second is the method based on key point correspondence: this method first predicts the two-dimensional feature points in the color image, and then uses the corresponding relationship between the two-dimensional feature points and the points on the model, and uses the PnP algorithm to solve the six-degree-of-freedom pose of the object; now Some such methods mainly have the following problems: traditional key point detection methods are difficult to detect two-dimensional feature points on non-textured or weakly textured objects; methods based on deep learning are also susceptible to the mutual occlusion of objects for the detection of key points , resulting in less robustness in this case

[0006] The third is the method of direct regression: this method uses the color image as the input of the deep network, and directly returns the six-degree-of-freedom pose of the object; the existing methods of this type mainly have the following problems: when the object whose pose is estimated is located When the background is cluttered and stacked with each other, the information used to estimate the pose will inevitably include information about the background and other stacked objects besides the object itself, which has a great impact on feature extraction, thereby reducing the object position. Therefore, this type of method finally performs pose refinement on the preliminary estimation result to correct the predicted pose, but the pose refinement process takes a long time, which reduces the efficiency of the algorithm

Images