Three-dimensional shape segmentation and semantic marking method based on projection convolutional network

Abstract

The invention proposes a three-dimensional shape segmentation and semantic marking method based on a projection convolutional network. An input is represented by adoption of a three-dimensional shape of a polygonal grid, information points cover a shape surface to a large extent, a rendering shape is a shadow image and a depth image, a dual-channel image is generated, through a fully-connected network (FCN) module of a same image, a confidence map is output for each function module of each input image, an image surface projection layer aggregates confidence maps of a plurality of views, combined with a boundary clue, surface condition random field (CRF) spreading is performed, modules of a task are trained, and finally a segmentation semantic marking result is obtained. The three-dimensional shape segmentation and semantic marking method based on a projection convolutional network does not need to use any manpower to adjust a geometric descriptor, reduces occlusion and covers the shape surface, does not lose a significant part label, effectively associates information, an occlusive part is also marked, integrity and coherence of segmentation are ensured, and the method is remarkably superior to previous methods.

Description

technical field [0001] The invention relates to the field of image segmentation, in particular to a three-dimensional shape segmentation and semantic labeling method based on a projection convolution network. Background technique [0002] The semantics of 3D shape segmentation into tokens has been widely used in robotics and virtual reality, such as 3D shape, cross-modal analysis, image object detection, image 3D reconstruction and style transfer of 3D objects, etc., according to the shape of the part and types, corresponding segmentation and semantic labeling, most of the existing semantic reasoning technologies for 3D geometric shape data rely on the heuristic processing stage and manual adjustment of geometric descriptors, and the clues of the segmentation boundary are very subtle, which requires a strong sense of noise Robustness, it is prone to serious occlusion, which affects the continuity and integrity of the segmentation boundary. [0003] The present invention pro...

AI Technical Summary

Problems solved by technology

[0002] The semantics of 3D shape segmentation into tokens has been widely used in robotics and virtual reality, such as 3D shape, cross-modal analysis, image object detection, image 3D reconstruction and style transfer of 3D objects, etc., according to the shape of the part and types, corresponding segmentation and semantic labeling, most of the existing semantic reasoning technologies for 3D geometric shape data rely on the heuristic processing stage and manual adjustment of geometric descriptors, and the clues of the segmentation boundary are very subtle, which requires a strong sense of noise Robustness, prone to serious occlusions, affecting the continuity and integrity of the segmentation boundary

Images