Cross-domain semantic segmentation method based on three-branch cross training

Abstract

According to the cross-domain semantic segmentation method based on three-branch cross training, common knowledge and unique knowledge among different views are fully mined, the generalization ability and segmentation ability of each view are improved, and meanwhile diversity among the views is kept. The method comprises an asymmetric three-branch network and two indirect pseudo tag alignment techniques. The three-branch network comprises three segmentation networks with different network structures, the three segmentation networks share one feature extraction network, and any two networks generate high-credibility pseudo labels for the third network. The indirect pseudo-tag alignment technique comprises semantic feature alignment, feature center points of a segmentation network are extracted through pseudo-tags, a simple category only needs to pull close the center points of the same category, and a complex category also needs to pull away the center points of different categories; comprising the following steps: estimating the adaptive capacity, evaluating the difference between two views for generating pseudo labels, and aligning the output of each pixel point according to the intensity of the adaptive capacity.

Description

technical field [0001] The invention relates to the technical field of unsupervised field adaptation, in particular to a cross-domain semantic segmentation method based on three-branch cross-training. Background technique [0002] Semantic segmentation aims to assign a semantic class label to each pixel of an image, and it is a key approach to provide comprehensive scene understanding for various real-world applications (e.g. autonomous driving, robotics). However, manually labeling every pixel in an image is very tedious. The more complex the scene in the image, the harder it is for a human to accurately label each pixel. This makes the available training data for the semantic segmentation task extremely limited, so the model can be trained with data generated from the virtual world (source domain) that can automatically label scenes, allowing the model to automatically label real life (target domain) without labels The data. However, due to the huge difference in data d...

AI Technical Summary

Problems solved by technology

[0004] The present invention provides a cross-domain semantic segmentation method based on three-branch cross-training, which is used to solve or at least partially solve the technical problem of poor cross-domain segmentation in the prior art

Images