Streetscape image semantic segmentation system and segmentation method, electronic equipment and computer readable medium

Abstract

The invention discloses a streetscape image semantic segmentation system and segmentation method, electronic equipment and a computer readable medium. The segmentation method comprises the following steps: step 1, acquiring a streetscape image and carrying out preprocessing and data enhancement on the streetscape image; step 2, encoding the streetscape image into an output feature map by using an encoder; step 3, collecting features of the last three output feature maps by using a multi-level feature combined up-sampling module, and fusing the features to obtain a second output feature map; 4, converting the second output feature map into a third output feature map; 5, inputting the third output feature map into a convolution classifier to obtain a semantic segmentation feature value; step 6, performing end-to-end training by using a back propagation algorithm to obtain a streetscape image semantic segmentation model; and 7, performing semantic segmentation on the streetscape image by using the streetscape image semantic segmentation model. According to the method, under the condition that semantic segmentation precision is not reduced, the speed of network segmentation is increased, and the real-time response capability of the method in application is enhanced.

Description

technical field [0001] The invention belongs to the technical field of image semantic segmentation, and in particular relates to a street view image semantic segmentation system and segmentation method, electronic equipment and a computer-readable medium. Background technique [0002] Semantic segmentation is one of the basic tasks of computer vision. Its purpose is to assign a semantic label to each pixel in the image, so as to obtain pixel-level segmentation results. Nowadays, most driverless visual perception systems use semantic segmentation technology to process the perceived objects, such as roads, pedestrians, cars, buildings, etc., so it plays an extremely important role in the field of unmanned driving. Due to the particularity of unmanned driving, it has high requirements for the accuracy of the semantic segmentation network. There is also an urgent need for real-time segmentation. [0003] As the most primitive fully convolutional neural network, FCN is transform...

AI Technical Summary

Problems solved by technology

[0003] As the most primitive fully convolutional neural network, FCN is transformed from a convolutional neural network dedicated to image classification. Following FCN, thanks to deep learning technology, semantic segmentation has made great progress in recent years; Semantic segmentation algorithms applied to unmanned driving are generally divided into two categories: the first category is a network based on an encoder-decoder structure, such as Unet and SegNet, etc., using an encoder-decoder structure for few categories. When segmenting tasks, the classification speed is fast and the accuracy is high, but when the classification categories increase, the speed and accuracy of semantic segmentation will be greatly reduced; the second type is a network based on context information, such as PSPNet and DeepLab v3+, etc., this type of network Improve the scene analysis ability of the network by introducing more context information, and keep the receptive field unchanged by introducing hole convolution, and use hole pyramid pooling on the top of the final feature map to avoid downsampling operations and obtain a lot of feelings However, since the introduction of hole convolution will increase the computational complexity and memory usage of the network, the network has a serious shortage in terms of segmentation speed.

[0004] The existing semantic segmentation network often generates a large number of parameters during operation, which consumes a lot of running time, and only considers the segmentation accuracy without considering the real-time performance of the network. In the field of unmanned driving, not only the accuracy of the semantic segmentation network is required, It is also very sensitive to the real-time nature of the algorithm, requiring the semantic segmentation algorithm to have real-time processing speed and fast interaction and response capabilities, so the above-mentioned network is not suitable for unmanned driving

Images