Automatic object extraction

Abstract

A method for automatic, stable and robust object extraction of moving objects in color video frames, achieved without any prior knowledge of the video content. For high rate video, the method includes providing at least a first and a second high frame rate video frames, performing a reciprocal illumination correction of the first and second video frames to yield respective first and second smoothed frames, performing a change detection operation between the first and second smoothed frames to obtain a difference image, and performing a local adaptive thresholding operation on the difference image to generate a binary image containing extracted objects, the local thresholding operation using a weight test to determine a boundary of each of the extracted objects. For an extracted object with a fragmented boundary, the method further comprises re-unifying the boundary. For low rate video, additional steps include: an edge correction applied on the first image to yield a first edge-corrected image, a global thresholding applied to the first edge-corrected image to yield a first binary edge image, and an ANDing operation on the difference image and the first binary edge image to generate a second binary image which is fed to the local adaptive thresholding operation.

Description

FIELD AND BACKGROUND OF THE INVENTION[0001]The term image segmentation refers to the partition of an image into a set of non-overlapping regions that cover it. An object is composed of one or more segments, and the term image segmentation is thus closely associated with “object extraction”. The definition of the latter being well known. Image segmentation is probably one of the most important low-level techniques in vision, since virtually any computer vision algorithm incorporates some sort of segmentation. In general, a segmentation is classified as groups of pixels that have common similarities. The properties of a good image segmentation are defined as follows: regions of segments in the image segmentation should be uniform and homogeneous with respect to some characteristic such as gray tone or texture. Region interiors should be simple and without many small holes. Adjacent regions should have significantly different values with respect to the characteristic on which they are ...

AI Technical Summary

Benefits of technology

[0061]The present invention successfully addresses the shortcomings of the presently known configurations by providing a method of automatic object extraction for segmentation of video frames that is automatic, robust, and independent of the nature of the video images. The method of the present invention is based on algorithms that are fast, do not consume a lot of computer resources, do not depend on predefined parameters and data, and do not produce over-segmentation. The method and algorithms of the present invention thus enable and provide: adaptive bit allocation for video compression, interactive TV, efficient image representation, quality of service (QoS) and differentiated services (DifferServ) over diverse communication networks (narrow and broad band), video streaming, surveillance, gaming, web caching, video mail and unified messaging. In addition, working with objects enables application of transform codings that are not based on square blocks of pixels such as 8×8 or 16×16, but use different sizes and shapes of blocks to cover the image, thus reflecting the activities in the image through the locations and shapes of the objects. The main problem in the current compression methods (MPEG-1,2,4, H263, H26L) lies in the fact that blocks are chosen independently from their relations to the nature of the pixels. Thus, a single block can belong to both the border of the object and to the background. Therefo

Problems solved by technology

Decomposing a video sequence into VOPs is a very difficult task, and comparatively little research has been undertaken in this field.

An intrinsic problem of VOP generation is that objects of interest are not homogeneous with respect to low-level features such as color, intensity, or optical flow.

Thus, conventional segmentation algorithms will fail to obtain meaningful partitions.

(1) it can also be seen that apparent motion is highly sensitive to noise because of the derivatives, which can cause largely incorrect results.

Unfortunately, we can only observe apparent motion.

In addition to the difficulties mentioned above, motion estimation algorithms have to solve the so-called occlusion and aperture problems.

The occlusion problem refers to the fact that no correspondence vectors exist for covered and uncovered background.

The aperture problem states that the number of unknowns is larger than the number of observations.

1. Nonparametric representation, in which a dense field is estimated where each pixel is assigned a correspondence or flow vector. Block matching is then applied, where the current frame is subdivided into blocks of equal size, and for each block the best match in the next (or previous) frame is computed. All pixels of a block are assumed to undergo the same translation, and are assigned the same correspondence vector. The selection of the block size is crucial. Block matching is unable to cope with rotations and deformations. Nevertheless, their simplicity and relative robustness make it a popular technique. Nonparametric representations are not suitable for segmentation, because an object moving in the 3-D space generates a spatially varying 2-D motion field even within the same region, except for the simple case of pure translation. This is the reason why parametric models are commonly used in segmentation algorithms. However, dense field estimation is often the first step in calculating the model parameters.

2. Parametric models require a segmentation of the scene, which is our ultimate goal, and describe the motion of each region by a set of a few parameters. The motion vectors can then be synthesized from these model parameters. A parametric representation is more compact than a dense field description, and less sensitive to noise, because many pixels are treated jointly to estimate a few parameters.

Although parametric representations are less noise sensitive, they still suffer from the intrinsic problems of motion estimation.

The major drawbacks of this proposal are the computational complexity, and the need to specify the number of objects likely to be found.

The techniques of Adiv, Bouthemy and Francois, and Murray and Buxton, include only optical flow data into the segmentation decision, and hence, their performance is limited by the accuracy of the estimated flow field.

These results are not good since we get over-segmentation, and the method is computationally expensive.

These approaches suffer from high computational complexity, and many algorithms need the number of objects or regions in the scene as an input parameter.

On the other hand, these approaches suffer from high computational complexity, and many algorithms need the number of objects or regions in the scene as an input parameter.

The result is an over-segmentation.

A drawback of this technique is the lack of temporal correspondence to enforce continuity in time.

However, due to its nature, the watershed algorithm suffer

Images