Digital processing method and system for determination of object occlusion in an image sequence

Abstract

A method and system for occlusion region detection and measurement between a pair of images are disclosed. A processing device receives a first image and a second image. The processing device estimates a field of motion vectors between the first image and the second image. The processing device motion compensates the first image toward the second image to obtain a motion-compensated image. The processing device compares a plurality of pixel values of the motion-compensated image to a plurality of pixels of the first image to estimate an error field. The processing device inputs the error field to a weighted error cost function to obtain an initial occlusion map. The processing device regularizes the initial occlusion map to obtain a regularized occlusion map.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application is a continuation of and claims the benefit of U.S. utility patent application Ser. No. 14 / 065,704 filed Oct. 29, 2013, which claims the benefit of U.S. provisional patent application No. 61 / 769,311 filed Feb. 26, 2013, the disclosures of which are incorporated herein by reference in their entirety.TECHNICAL FIELD[0002]The present invention relates generally to digital image processing, and more particularly, to a method and system for automatic determination of the presence, location, and type of occlusion regions between a pair of images.BACKGROUND[0003]Determining an optical flow or motion vector field between two images, particularly for sequences of video frames and / or fields, is frequently encountered in many high-value video processing tasks such as coding, frame rate conversion, noise reduction, etc. Conventional methods for calculating optical flow encounter several stumbling blocks—many solutions of which are de...

AI Technical Summary

Benefits of technology

[0015]Examples of the present disclosure provide a method and system for detecting and characterizing occlusion regions without any assumptions that depend on scene types, motion types, or supervised learning datasets. Examples of the present disclosure provide an accurate and precise occlusion region map. The occlusion detection is decoupled from the motion estimation itself, providing for flexible addition to any suitable optical flow or motion estimation system or method.

Problems solved by technology

As taught in the '534 patent, object occlusion presents a challenge for any motion estimation system, such as an optical flow estimation system.

These two types of areas within a pair of images (collectively known herein as occlusion regions) are very problematic for motion estimation in general, and for many optical flow systems in particular, because erroneous motion vector values in these regions tend to propagate into non-occlusion regions, adversely affecting the overall accuracy of the optical flow estimation.

With no motion information, occlusion boundary detection is an inherently ambiguous problem.

2226-2234 (2010), explicitly models occlusion and the results obtained are relatively accurate, but the method possesses a huge computational load.

Finding occlusion regions represents a common problem in multi-view 3D projection and display methods.

The most recent researched methods in this area are still prone to gross errors when the background or foreground underlying pixel data in these regions is homogeneous or have no texture information.

In Alvarez, et al., “Symmetrical dense optical flow estimation with occlusions detection,” International Journal of Computer Vision 75(3), 371-385 (2007), (hereinafter, Alvarez), passing interest is focused on the role of the diffusion tensor and subsequent eigenvalue analysis, but this is only used to analyze the forward and backward symmetry of the optical flow solution, and not used to directly improve the accuracy of either the optical flow computation nor the occlusion computation.

Image Processing 17(8), 1443-1451 (2008), (hereinafter, “Ince”), discloses a method and systems for joint determination of optical flow and occlusion, but the systems are coupled and this method is not applicable for coupling to a non-optical-flow motion estimation system, such as block matching.

These methods encounter problems within highly textured imagery areas and do not succeed with large displacements or occlusion regions.

Further, this method requires multiple iterations for convergence of the disclosed regularization function and is therefore not suitable for real-time computation for contemporaneous video resolutions such as 1080 and 4K.

Even the best conventional motion estimation methods with coupled occlusion detection systems suffer from two primary disadvantages.

First, these methods are too computationally complex for real-time processing.

Second, the occlusion region maps they produce are inherently noisy.

Pixels marked as occlusions may frequently be false-positives or false-negatives, rendering their usage in subsequent video processing and analysis tasks challenging or impossible.

Images