Semantic segmentation method based on double-flow feature fusion

Abstract

The invention discloses a semantic segmentation method based on double-flow feature fusion. The method comprises the steps of in a training stage, constructing a convolutional neural network comprising an input layer, a hidden layer and an output layer, and the hidden layer comprises an RGB image processing module, a depth image processing module, a fusion module and a first deconvolution layer; inputting the original image into a convolutional neural network for training to obtain a corresponding semantic segmentation prediction graph; calculating a loss function value between a set formed bysemantic segmentation prediction images corresponding to the original images and a set formed by one-hot coded images processed by corresponding real semantic segmentation images to obtain an optimalweight vector and a bias term of a convolutional neural network classification training model; and in a test stage, inputting an indoor scene image to be semantically segmented into the convolutionalneural network classification training model to obtain a predicted semantic segmentation image. According to the method, the semantic segmentation efficiency and accuracy of the indoor scene image are improved.

Description

technical field [0001] The invention is a semantic segmentation method based on a fully convolutional neural network, in particular a semantic segmentation method based on dual-stream feature fusion. Background technique [0002] Semantic segmentation is a fundamental technique for many computer vision applications, such as scene understanding, autonomous driving. With the development of Convolutional Neural Networks, especially Fully Convolutional Neural Networks (FCNs), many promising results have been achieved on benchmarks. FCN has a typical encoder-decoder structure—semantic information is first embedded into the feature map through the encoder, and the decoder is responsible for generating segmentation results. Typically, the encoder is a pre-trained convolutional model to extract image features, and the decoder contains multiple upsampling components to recover resolution. Although the encoder's most important feature map may be highly semantic, it has limited abili...

AI Technical Summary

Problems solved by technology

Although the encoder's most important feature map may be highly semantic, it has limited ability to reconstruct precise details in the segmentation map due to insufficient resolution, which is very common in modern backbone models.

Images