Closed form method and system for matting a foreground object in an image having a background

Abstract

In a method and system for matting a foreground object F having an opacity α constrained by associating a characteristic with selected pixels in an image having a background B, weights are determined for all edges of neighboring pixels for the image and used to build a Laplacian matrix L. The equation α is solved where α=arg min α^T Lα s.t.α_i=s_i, ∀i ∈ S, S is the group of selected pixels, and s_i is the value indicated by the associated characteristic. The equation I_i=α_iF_i+(1−α_i)B_i is solved for F and B with additional smoothness assumptions on F and B; after which the foreground object F may be composited on a selected background B′ that may be the original background B or may be a different background, thus allowing foreground features to be extracted from the original image and copied to a different background.

Description

RELATED APPLICATONS [0001] This application claims benefit of provisional applications Ser. Nos. 60 / 699,503 filed Jul. 15, 2005 and 60 / 714,265 filed Sep. 7, 2005 whose contents are included herein by reference.FIELD OF THE INVENTION [0002] This invention relates to image matting. REFERENCES [0003] The description refers to the following prior art references, whose contents are incorporated herein by reference. [0004] [1] N. E. Apostoloff and A. W. Fitzgibbon. “Bayesian video matting using learnt image priors” In Proc. CVPR, 2004. [0005] [2] U.S. Pat. No. 6,134,345 A. Berman, P. Vlahos, and A. Dadourian. “Comprehensive method for removing from an image the background surrounding a selected object”, 2000 [0006] [3] Y. Boykov and M. P. Jolly “Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images” In Proc. ICCV, 2001. [0007] [4] Y. Chuang, A. Agarwala, B. Curless, D. Salesin, and R. Szeliski “Video matting of complex scenes”, ACM Trans. Graph., 21(3)...

AI Technical Summary

Benefits of technology

[0046] The invention provides a new closed-form solution for extracting the alpha matte from a natural image. We derive a cost function from local smoothness assumptions on foreground and background colors F and B, and show that in the resulting expression it is possible to analytically eliminate F and B, yielding a quadratic cost function in α. The alpha matte produced by our method is the global optimum of this cost function, which may be obtained by solving a sparse linear system. Since our approach computes α directly and without requiring reliable estimates for F and B, a surpri

Problems solved by technology

From a computer vision perspective, this task is extremely challenging because it is massively ill-posed—at each pixel we must estimate the foreground and the background colors, as well as the foreground opacity (“alpha matte”) from a single color measurement.

While mixed pixels may represent a small fraction of the image, human observers are remarkably sensitive to their appearance, and even small artifacts could cause the composite to look fake.

Obviously, this is a severely under-constrained problem, and user interaction is required to extract a good matte.

As a consequence, trimap-based approaches typically experience difficulty handling images with a significant portion of mixed pixels or when the foreground object has many holes [15].

Another problem with the trimap interface is that the user cannot directly influence the matte in the most important part of the image: the mixed pixels.

The requirement of a hand-drawn segmentation becomes far more limiting whe

Images