A Panoramic Segmentation Method with Bidirectional Connection and Occlusion Handling

Abstract

The invention discloses a panorama segmentation method with two-way connection and occlusion processing. The method includes the following steps: 1) Obtain a data set for training panorama segmentation, and define the algorithm target; 2) Use a fully convolutional network to perform feature learning on the images in the group; 3) Extract semantic features from the feature map through the semantic feature extraction branch ; 4) Extract instance features from the feature map through the instance feature extraction branch; 5) Establish a connection from instance segmentation to semantic segmentation, and aggregate semantic features and instance features for semantic segmentation; 6) Establish a connection from semantic segmentation to instance segmentation, and instance Feature and semantic features are aggregated for instance segmentation; 7) Use occlusion processing algorithm to fuse the results of semantic segmentation and instance segmentation, and output panoramic segmentation results. This method makes full use of the complementarity between semantic segmentation and instance segmentation, and at the same time applies the occlusion processing algorithm proposed by the underlying feature appearance information to efficiently complete the panoramic image segmentation.

Description

technical field [0001] The invention belongs to the field of computer vision, in particular to a panorama segmentation method with two-way connection and occlusion processing. Background technique [0002] The panoptic segmentation task is a collection of semantic segmentation tasks and instance segmentation tasks, which not only require predicting semantic classes at the pixel level, but also require distinguishing instances for foreground categories. This task is an important basic task for scene understanding, and has broad application value in fields such as autonomous driving. The current mainstream technology routes are divided into top-down and bottom-up methods. The top-down method first finds the bounding box of the instance, and then confirms whether the pixel in the box belongs to the instance. The bottom-up approach first predicts the pixel-by-pixel instance attribution, and then generates bounding boxes accordingly. In terms of empirical results, the performa...

AI Technical Summary

Problems solved by technology

First, in this scheme, there are two sub-networks for the two tasks of semantic segmentation and instance segmentation, but there is no way for information to propagate between these two sub-networks

Thus, the complementarity between the two tasks is not well exploited

Second, for the detected instances, there may be mutual occlusion

Past methods rely on the category scores of objects to handle occlusion relations, but this practice is obviously not optimal due to the correlation of category scores with other factors such as data distribution.

Images