Mature pomegranate positioning method based on Mask R-CNN and three-dimensional sphere fitting

Abstract

The invention discloses a mature pomegranate positioning method based on Mask R-CNN and three-dimensional sphere fitting, and relates to the field of fruit picking and positioning of agricultural robots. Rapid and accurate positioning of fruits is the premise of accurate picking of the picking robot. According to the method, a pomegranate color image RGB and a depth image D are acquired by adopting Kinect V2.0, the acquired color image RGB is rapidly positioned to the picture position of a mature pomegranate by adopting Mask R-CNN, then a corresponding depth picture is segmented according to the RGB picture of the mature pomegranate, and three-dimensional point cloud reconstruction is performed by adopting camera calibration parameters and a rotation matrix; finally, the mature pomegranatepoint cloud is complemented for the single mature pomegranate point cloud data by adopting a mode of fitting a sphere based on a least square algorithm to obtain complete mature pomegranate point cloud data, and a target pomegranate is positioned. According to the method, the mature pomegranate is positioned by integrating the color image RGB and the depth image D, the defect that the actual sizeof the fruit is difficult to obtain only by using an RGB image positioning method is overcome, the complex matching process of multi-azimuth point cloud data fusion is also overcome, and the real-time performance and the precision of fruit positioning are greatly improved.

Description

technical field [0001] The invention relates to fruit picking positioning of agricultural robots, in particular to a ripe pomegranate positioning method based on Mask R-CNN and 3D sphere fitting. Background technique [0002] At present, there are many researches on fruit positioning methods, mainly using color images to segment fruits, RGB and D images to segment fruits, or laser point cloud data to segment fruits. These methods have obvious shortcomings: [0003] (1) Only 2D RGB images are used to segment fruits. Although high segmentation accuracy can be achieved in color images, it is difficult to use spatial 3D information for picking; [0004] (2) The method of combining RGB and D is used to segment the fruit, and the segmentation efficiency is improved, but there are very few studies on the completion of the fruit to find the appropriate center coordinate parameters and geometric dimensions; [0005] (3) Only three-dimensional laser point cloud is used to segment, id...

AI Technical Summary

Problems solved by technology

[0003] (1) Only 2D RGB images are used to segment fruits. Although high segmentation accuracy can be achieved in color images, it is difficult to use spatial 3D information for picking;

[0004] (2) The method of combining RGB and D is used to segment the fruit, and the segmentation efficiency is improved, but there are very few studies on the completion of the fruit to find the appropriate center coordinate parameters and geometric dimensions;

[0005] (3) Only use 3D laser point cloud to segment, identify and locate fruit, but laser point cloud equipment is expensive and not convenient to be used as a sensor for a robotic arm

Images