Modeling for enumerative encoding

Abstract

Entropy encoders and decoders employ an entropy-pump operation. The entropy-pump operation includes classifying blocks of the input sequence's symbols in accordance with the probabilities of occurrence of those blocks' values. Entropy coding an output component that represents the sequence of classes to which the blocks belong tends to be profitable, because that component has an entropy density less than that of the input sequence. In contrast, little opportunity for compression tends to be lost by not entropy encoding an output component that represents the values of the blocks that belong to single class, because those values' probabilities of occurrence are similar.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]The present application claims the benefit of U.S. Provisional Patent Application Ser. No. 60 / 693,963, which was filed on Jun. 23, 2005, by Ratko V. Tomic for Modeling for Quantized-Index Type Enumerative Encoding, and U.S. Provisional Patent Application Ser. No. 60 / 737,073, which was filed on Nov. 15, 2005, by Ratko V. Tomic for Encoder Employing Quantized Indexing, both of which are hereby incorporated by reference.BACKGROUND OF THE INVENTION[0002]1. Field of the Invention[0003]The present invention is directed to entropy encoding and decoding, and it particularly concerns modeling employed for such encoding and decoding.[0004]2. Background Information[0005]Data compression usually includes multiple phases, where initial phases are more dependent on the specific data source. An encoder to be employed on sequences of symbols that represent values of pixels in an image, for example, may take the form of FIG. 1's decoder 10. The symbols ma...

AI Technical Summary

Benefits of technology

"The patent describes new methods for low-level modeling that are useful for entropy encoding, specifically for enumerative encoding. These methods involve dividing an input sequence into blocks, classifying them, and generating an output that represents the sequence of classes to which those blocks belong and an output of indexes that uniquely identify the values of each block within its class. These methods take advantage of the fact that there is often less redundancy in the index-based component of the model, resulting in higher compression values with lower computational costs. The technical effect of these methods is improved efficiency and effectiveness in entropy encoding, particularly for enumerative encoding."

Problems solved by technology

The principal weakness of the Huffman code is its sub-optimality in the case of more-general probabilities (those not of the form ½k).

Huffman coding is especially inefficient when one symbol has a probability very close to unity and would therefore need only a tiny fraction of one bit; since no symbol can be shorter than a single bit, the code length can exceed the entropy by a potentially very large ratio.

While there are workarounds for the worst cases (such as run-length codes and the construction of multi-character symbols in accordance with, e.g., Tunstall coding), such workarounds either fall short of optimality or otherwise require too much computation or memory as they approach the theoretical entropy.

A second important weakness of the Huffman code is that its coding overhead increases, both in speed and memory usage, when the adaptive version of the algorithm is used to track varying symbol probabilities.

For sufficiently variable sources, moreover, even adaptive Huffman algorithm cannot build up statistics accurate enough to reach coding optimality over short input-symbol spans.

Despite minor improvements over the decades, its fatal drawback was the requirement that its arithmetic precision be of the size of output data, i.e., divisions and multiplications could have to handle numbers thousands of bits long.

But arithmetic coding is like Huffman coding in that its performance suffers when source statistics change rapidly.

Since the number of patterns for a given population can be quite large, it would not be practical to arrive at a significant-length sequence's pattern index by storing associations between indexes and patterns in a look-up table.

The combinatorial values used as “add-on” terms in the index calculation can be expensive to compute, of course, but in practice they would usually be pre-computed once and then simply retrieved from a look-up table.

To the extent that source statistics vary, the look-up table's contents have to be recomputed if near-optimal compression is to be achieved, and this imposes a heavy computational burden if the source statistics vary rapidly.

Enumerative coding has nonetheless enjoyed little use as a practical tool.

The calculation's addition steps therefore tend to involve expensive multiple-word-resolution additions.

Still, the best-compression settings of modern implementations require multiplications on the encoder and divisions on the decoder for each processed symbol, so they are slower than a static Huffman coder, especially on the decoder side.

By some evaluations, moreover, the arithmetic coder compresses even less effectively than the Huffman coder when its probability tables fail to keep up with the source probabilities or otherwise do not match them.

For a long time, though, this did not appear possible; it turns out that the indexes that result from simply rounding the conventional path-count values to such low-resolution values are not always unique.

Images