Image matting using deep learning

Abstract

Embodiments in the invention generally relates to image matting using deep learning. Methods and systems are provided for generating mattes for input images. A neural network system can be trained where the training includes training a first neural network that generates mattes for input images where the input images are synthetic composite images. Such a neural network system can further be trained where the training includes training a second neural network that generates refined mattes from the mattes produced by the first neural network. Such a trained neural network system can be used toinput an image and trimap pair for which the trained system will output a matte. Such a matte can be used to extract an object from the input image. Upon extracting the object, a user can manipulate the object, for example, to composite the object onto a new background.

Description

technical field [0001] Embodiments of the present disclosure relate generally to image processing, and in particular to image masking using deep learning. Background technique [0002] It is often desirable to extract objects from images, e.g., to combine objects with different backgrounds. In order to remove foreground objects conveniently, an image mask (matte) or mask can be used to extract specific foreground objects in an image. Since the pixel colors in an image may be the color of the foreground object, the color of the background, or some combination of foreground and background colors, the image mask may include an alpha value indicating the percentage of the foreground color present at each pixel. For example, when a camera pixel sensor receives light from both foreground objects and the background, the pixels may have a combination of foreground and background colors. Typically, pixels around the edges of objects and in areas corresponding to hair, fur, and moti...

AI Technical Summary

Problems solved by technology

Because existing methods rely on color, such methods cannot accurately determine alpha values ​​when foreground and background colors and / or textures are similar to each other and have difficulty with edges, making images with fur or hair difficult to segment

Therefore, such methods cannot produce accurate masks of typical everyday scenes with similar foreground and background colors and / or textures

Images