Two-stage multi-modal three-dimensional instance segmentation method

Abstract

The invention provides a two-stage multi-mode three-dimensional instance segmentation method. The method comprises two parts of two-dimensional prior information acquisition and three-dimensional instance segmentation. The two-dimensional prior information acquisition part adopts an RGBD multi-mode fusion two-dimensional instance segmentation network, a depth feature correction module and a data enhancement strategy for a depth map are designed for the problem of low quality of the depth map, and a feature alignment module is added in a feature pyramid module. And the three-dimensional instance segmentation part is used for generating a point cloud with instance segmentation information by using a coordinate transformation method in combination with a depth map according to the acquired two-dimensional prior information, and then removing noise points in the point cloud by using direct filtering and statistical filtering to realize three-dimensional instance segmentation of a target. The two-stage multi-modal three-dimensional instance segmentation method has the beneficial effects that the two-stage multi-modal three-dimensional instance segmentation method can perform accurate three-dimensional instance segmentation on a scene target in an application scene with a small amount of two-dimensional annotation data and without three-dimensional annotation data, and has the advantage of high real-time performance.

Description

technical field [0001] The invention belongs to the fields of computer vision and deep learning, and in particular relates to a two-stage multimodal three-dimensional instance segmentation method. Background technique [0002] With the rise of the industrial Internet, the manufacturing industry is gradually developing in the direction of intelligence. The smart workshop is an important part of the manufacturing process, and the degree of scene awareness is an important indicator of its intelligence level. As an effective scene-aware method, the 3D instance segmentation method can generate 3D object boundaries to obtain object category information and object spatial location information that cannot be obtained by 2D instance segmentation methods. Furthermore, 3D instance segmentation has been applied in areas such as autonomous driving, robot navigation, and virtual / augmented reality. However, most applications in smart workshops are only based on 2D scene perception. Ther...

AI Technical Summary

Problems solved by technology

[0003] (1) Due to the difficulty in collecting 3D information in the smart workshop scene and the high cost of 3D label labeling, it is an unavoidable problem to achieve accurate 3D instance segmentation of the smart workshop scene target in the absence of 3D instance segmentation labels;

[0004] (2) There are a large number of identical machine tools on the production line in the smart workshop scene. They have similar color features and shape features, which makes it difficult for existing recognition methods to distinguish their boundaries. In this case, accurately segmenting each instance is another problem

Images