Method for recognizing sweet peppers and determining picking sequence based on multi-view three-dimensional reconstruction

Abstract

The invention discloses a sweet pepper recognition and picking sequence determination method based on multi-view three-dimensional reconstruction. Firstly, point cloud data of a target fruit is obtained through multi-view three-dimensional reconstruction in combination with a plurality of views, so that point cloud noise can be effectively reduced, and the influence of leaf shielding can be reduced; recognizing the obtained RGBD point cloud information through an HSV recognition algorithm, and filtering out non-red sweet pepper fruit point cloud; solving the number of point clouds of the sweet pepper fruits and the number of point clouds in each point cloud of the sweet pepper fruits through Euclidean clustering; and finally, the recognized fruit point cloud clusters, namely the sweet pepper fruit point cloud, are processed, the size of each fruit point cloud cluster, the grabbing difficulty degree and the distance from the optimal grabbing point in each fruit point cloud cluster to the tail end of the mechanical arm are obtained, and the optimal grabbing sequence of the fruits is obtained through a multi-target optimization algorithm. According to the method, information in multiple aspects is combined to determine how the picking robot grabs the recognized fruits, the picking sequence of the fruits is determined, the grabbing success rate of the picking robot can be increased. Meanwhile, energy consumption of the mechanical arm is reduced, the picking success rate is increased, and the fruit damage rate is reduced.

Description

technical field [0001] The invention belongs to the field of picking by a mobile mechanical arm, and relates to a method for identifying sweet peppers and determining a picking order based on multi-view three-dimensional reconstruction. Background technique [0002] In the field of mobile picking, the vision module carried by the mobile robotic arm is like the eyes of human beings. Only when they can see clearly and identify accurately can the picking efficiency be effectively improved. The environment of the mobile manipulator in the greenhouse operation is very complicated. There are leaves, stems, and devices for fixing plants in the environment, which often block the fruit. The vision module usually cannot observe the whole picture of the fruit through a single viewing angle, which will give The accurate recognition of the fruit by the follow-up vision module brings great difficulties, and serious cases may lead to picking failure. [0003] During the picking process of...

AI Technical Summary

Problems solved by technology

[0004] Compared with single-view and multi-view, there are mainly the following three disadvantages (1) A single view usually acquires the picking environment through RGB (color image) and point cloud. The point cloud information cannot fully describe the three-dimensional information of the fruit and is accompanied by a lot of noise, which makes it difficult to accurately identify the subsequent fruit

(2) In terms of fruit recognition, the traditional color recognition algorithm is highly sensitive to light, and the recognition effect largely depends on the light intensity. In terms of grasping, RGB images and point cloud data obtained from a single viewing angle cannot be accurately obtained. grab point

Images