Panoramic segmentation method with bidirectional connection and shielding processing

Abstract

The invention discloses a panoramic segmentation method with bidirectional connection and shielding processing. The method comprises the following steps: 1) obtaining a data set for training panoramicsegmentation, and defining an algorithm target; 2) performing feature learning on intra-group images by using a full convolutional network; 3) extracting semantic features from a feature map throughsemantic feature extraction branches; 4) extracting instance features from the feature map through instance feature extraction branches; 5) establishing connection from instance segmentation to semantic segmentation, and aggregating the semantic features and the instance features to perform semantic segmentation; 6) establishing connection from semantic segmentation to instance segmentation, and aggregating the instance features and the semantic features to perform instance segmentation; and 7) using an occlusion processing algorithm, fusing the results of semantic segmentation and instance segmentation, and outputting the result of panoramic segmentation. According to the method, the complementarity between the semantic segmentation and the instance segmentation is fully utilized, meanwhile, the occlusion processing algorithm provided by the apparent information of the bottom-layer features is applied, and the panoramic segmentation of the image is efficiently completed.

Description

technical field [0001] The invention belongs to the field of computer vision, in particular to a panorama segmentation method with two-way connection and occlusion processing. Background technique [0002] The panoptic segmentation task is a collection of semantic segmentation tasks and instance segmentation tasks, which not only require predicting semantic classes at the pixel level, but also require distinguishing instances for foreground categories. This task is an important basic task for scene understanding, and has broad application value in fields such as autonomous driving. The current mainstream technology routes are divided into top-down and bottom-up methods. The top-down method first finds the bounding box of the instance, and then confirms whether the pixel in the box belongs to the instance. The bottom-up approach first predicts the pixel-by-pixel instance attribution, and then generates bounding boxes accordingly. In terms of empirical results, the performa...

AI Technical Summary

Problems solved by technology

First, in this scheme, there are two sub-networks for the two tasks of semantic segmentation and instance segmentation, but there is no way for information to propagate between these two sub-networks

Thus, the complementarity between the two tasks is not well exploited

Second, for the detected instances, there may be mutual occlusion

Past methods rely on the category scores of objects to handle occlusion relations, but this practice is obviously not optimal due to the correlation of category scores with other factors such as data distribution.

Images