Segment-based encoding system using segment hierarchies

Abstract

Video data is compressed using segmentation of reference frames where the segmentation results in a hierarchy of segments. The hierarchy can be generated bottom-up, where a first set of segments is generated and then a second set of segments generated by grouping segments of the first set, based on pixel value and / or segment boundaries. The hierarchy can be generated top-down, where a first set is generated and then a second set is generated by dividing up segments of the first set, based on pixel value and / or segment boundaries. The hierarchy can be included in the compressed video data or omitted such that a decoder would have to independently generate it. The hierarchy can be two or more levels. The hierarchy can be used for editing, formatting and / or compressing frames, as well as associating metadata with elements of the frame and coding motion, residue or other kinetic information.

Description

CROSS-REFERENCES TO RELATED APPLICATIONS [0001] The present application is a continuation of U.S. application Ser. No. 10 / 338,794, entitled “Segment-Based Encoding System Using Segment Hierarchies,” filed on Jan. 7, 2003 with named inventors Adityo Prakash, Eniko Fodor, and Edward R. Ratner. U.S. application Ser. No. 10 / 338,794 is a continuation-in-part of U.S. application Ser. No. 09 / 550,705, filed on Apr. 17, 2000, which is a non-provisional of and claims the benefit of U.S. Application No. 60 / 129,854, filed on Apr. 17, 1999, and U.S. Application No. 60 / 129,853, filed on Apr. 17, 1999. The complete disclosures of the aforementioned applications are hereby incorporated by reference.FIELD OF THE INVENTION [0002] The present invention relates generally to the compression of video data, and more particularly to a synchronized encoder / decoder system wherein the decoder performs a segmentation that is performed by the encoder for encoding video data so that the encoder need not convey t...

AI Technical Summary

Problems solved by technology

However, JPEG and similar compression schemes do not address the crucial issue of temporal redundancy.

Unfortunately, fractal compression is not a viable method of general compression.

In addition, fractal compression is very computationally intensive.

Although state of the art, block matching is highly inefficient and fails to take advantage of the known general physical characteristics or other information inherent in the images.

However, since blocks do not correspond to real objects, block-based systems cannot use this information to further reduce the bitstream.

Yet another major limitation of block-based matches arises because the residue created by block-based matching is generally noisy and patchy.

Thus, block-based residues do not lend themselves to good compression via standard image compression schemes such as DCT, wavelets, or fractals.

It is well recognized that the state of the art needs improvement, specifically in that the block-based method is extremely inefficient and does not produce an optimally compressed bitstream for motion video information.

While some compression gains are achieved, the associated overhead information is substantially increased because, in addition to the motion and residue information, these schemes require that structural or shape information for each object in a frame must also be sent to the receiver.

Images