Deep learning instance segmentation method and device fusing point cloud information, and equipment

Abstract

The invention discloses a deep learning instance segmentation method and device fusing point cloud information and computer equipment. The method comprises the following steps: obtaining an instance object image and corresponding point cloud data by adopting a 3D camera; carrying out instance segmentation on the instance object image by utilizing an improved deep learning recognition framework; preprocessing the point cloud data to obtain effective point cloud; performing information fusion on the instance segmentation object and the effective point cloud to obtain a corrected instance segmentation object; and externally outputting and applying the corrected instance segmentation object to the project. Image data and measurement data can be provided for an identification system, the systemcan be directly matched with an application system responsible for completing the whole working process in project implementation of a destacking and stacking system and the like, the system integration degree is high, and deployment is convenient; point cloud information is fused into instance segmentation, the fusion mode is flexible, and risks and uncertainty possibly existing when only one recognition mode is used are avoided.

Description

technical field [0001] The invention relates to a deep learning instance segmentation method, device and computer equipment for merging point cloud information, and belongs to the technical field of image processing. Background technique [0002] In projects based on vision guidance, image-based deep learning instance segmentation is a very effective technical solution. Commonly used instance segmentation methods include SDS, CFM, FCIS, Mask R-CNN, etc. Because of their high accuracy, the application scenarios are becoming more and more extensive. In industrial projects, the accuracy and precision of segmentation are required to be higher. If only relying on deep learning recognition results, it will bring greater risks and uncontrollable factors to industrial applications. [0003] On the other hand, with the development of electronic technology in recent years, the 3D point cloud of 3D cameras has been greatly improved in terms of imaging quality, speed, and accuracy, and...

AI Technical Summary

Problems solved by technology

Therefore, deep learning still has problems such as identification risk in instance segmentation, immature application of 3D point cloud deep learning technology, and inaccurate single identification method.

Images