Joint Depth estimation and semantic segmentation from a single image

Abstract

Joint depth estimation and semantic labeling techniques usable for processing of a single image are described. In one or more implementations, global semantic and depth layouts are estimated of a scene of the image through machine learning by the one or more computing devices. Local semantic and depth layouts are also estimated for respective ones of a plurality of segments of the scene of the image through machine learning by the one or more computing devices. The estimated global semantic and depth layouts are merged with the local semantic and depth layouts by the one or more computing devices to semantically label and assign a depth value to individual pixels in the image.

Description

technical field [0001] This application generally relates to joint depth estimation and semantic annotation for a single image. Background technique [0002] Depth estimation in images is generally used to estimate the distance between objects in the image scene and the camera used to capture the image. This is conventionally performed using stereo images or dedicated depth sensors (eg, time-of-flight or structured light cameras) to identify objects, support gestures, etc. Thus, this reliance on dedicated hardware, such as stereo cameras or dedicated depth sensors, limits the usability of these conventional techniques. [0003] Semantic annotation in images is used to assign labels to pixels in the image to eg describe objects at least partially represented by the pixels, such as sky, ground, buildings, etc. This can be used to support functions such as object removal and replacement in images, masking, segmentation techniques, etc. However, conventional ways to perform s...

AI Technical Summary

Problems solved by technology

However, conventional ways to perform semantic annotation are usually addressed separately or sequentially from depth estimation using different and unrelated techniques, lack accuracy and may cause errors developed in early stages in the technique's execution to be propagated later stage

Images