Semantic segmentation method and system for RGB-D image

Abstract

The invention discloses a semantic segmentation method and system for an RGB-D image. The semantic segmentation method comprises the steps: extracting RGB coding features and depth coding features ofan RGB-D image in multiple stages; inputting the RGB coding features and the depth coding features of each stage in the plurality of stages into an attention model to obtain each multi-mode fusion feature corresponding to each stage; extracting context semantic information of the multi-modal fusion features in the fifth stage by using a long short-term memory network; splicing the multi-modal fusion features and the context semantic information in the fifth stage to obtain context semantic features; and performing up-sampling on the context semantic features, and fusing the context semantic features with the multi-modal fusion features of the corresponding stage by using a jump connection mode to obtain a semantic segmentation map and a semantic segmentation model. By extracting RGB codingfeatures and depth coding features of the RGB-D image in multiple stages, the semantic segmentation method effectively utilizes color information and depth information of the RGB-D image, and effectively mines context semantic information of the image by using a long short-term memory network, so that the semantic segmentation accuracy of the RGB-D image is improved.

Description

technical field [0001] The present application relates to the field of deep learning technology, in particular to a method and system for semantic segmentation of RGB-D images. Background technique [0002] Semantic segmentation is particularly important in the application of computer intelligent image processing. The process of semantic segmentation is the process of identifying the category of each pixel in the image according to the visual content of the image. It can be understood that the pixel values ​​of the pixels belonging to the same category in an image are the same. As the basis of image scene understanding, semantic segmentation has important Research value and broad application prospects, such as UAV navigation and automatic driving, etc. [0003] With the rise of deep convolutional neural networks, deep convolutional networks have become the most effective method for extracting image features. In 2015, the full convolutional network opened up a new model of ...

AI Technical Summary

Problems solved by technology

[0005] However, the existing methods fail to make full use of color information and depth information, and at the same time fail to effectively mine the contextual semantic information of the image, resulting in a low accuracy rate for RGB-D image semantic segmentation.

Images