Parse Tree Compression Dictionary With Embedded Entropy Ranks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Combining Ziv-Lempel and Huffman Coding for data compression and decompression introduces inefficiencies due to additional memory accesses and increased memory requirements, particularly when translating symbols to ranks, which can tax system resources and reduce compression and decompression performance.
Innovation Solution
A method that generates a value conversion dictionary using a parse tree based compression algorithm, estimates likelihoods for each value, assigns ranks, and reorders dictionary entries to create an architected dictionary, allowing for compression and decompression without additional memory lookups by walking from dictionary entries to ranks or vice versa.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If Ziv-Lempel and Huffman Coding are combined for data compression, then compression efficiency is improved, but additional memory accesses are required which increases system overhead
Solution Approach 1:
The patent combines Ziv-Lempel parse tree structure with Huffman coding by integrating the rank assignment directly into the dictionary entry structure. The dictionary now contains both the symbol mapping and the rank information in a unified data structure, eliminating the need for separate memory accesses to retrieve rank information after symbol lookup.
Solution Approach 2:
The patent pre-calculates and stores the rank information within the dictionary entry itself during the dictionary construction phase. This preliminary action ensures that when compression occurs, both the symbol mapping and rank are immediately available without requiring additional memory accesses during the actual compression process.
2Productivity
If Ziv-Lempel and Huffman Coding are combined for data compression, then compression efficiency is improved, but memory space requirements increase
Solution Approach 1:
The patent merges the symbol dictionary and rank storage into a single integrated data structure. Instead of storing symbols and ranks separately in different memory locations, each dictionary entry contains both the symbol mapping information and the associated rank, reducing overall memory space requirements.
Solution Approach 2:
The dictionary structure serves multiple functions simultaneously: it stores the symbol-to-value mappings required by Ziv-Lempel compression and the rank information required by Huffman coding. This multi-functional design eliminates the need for separate storage structures, optimizing memory utilization.
3Ease of operation
If symbol to rank translation is performed during compression, then compression functionality is achieved, but additional unpredictable memory accesses occur
Solution Approach 1:
The rank information is pre-computed and embedded within each dictionary entry during the initialization phase. This eliminates the need for additional memory accesses during compression operations, as both the symbol mapping and rank are immediately available in the same memory location.
Solution Approach 2:
The integrated dictionary entry acts as an intermediary structure that combines both the symbol mapping and rank information. This mediator structure allows the compression algorithm to retrieve both pieces of information in a single memory access, eliminating the need for sequential accesses to separate data structures.
Data Source
AI summary
A method, computer program product, and system includes a processor obtaining data including values and generating a value conversion dictionary by applying a parse tree based compression algorithm to the data, where the value conversion dictionary includes dictionary entries that represent the values. The processor obtains a distribution of the values and estimates a likelihood for each based on the distribution. The processor generates a code word to represent each value, a size of each code word is inversely proportional to the likelihood of the word. The processor assigns a rank to each code word, the rank for each represents the likelihood of the value represented by the code word; and based on the rank associated with each code word, the processor reorders each dictionary entry in the value conversion dictionary to associate each dictionary entry with an equivalent rank, the reordered value conversion dictionary comprises an architected dictionary.