Parse Tree Compression Dictionary Reordering for Fewer Memory Lookups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Combining Ziv-Lempel and Huffman Coding for data compression and decompression introduces inefficiencies due to additional memory accesses and increased memory requirements, particularly when translating symbols to ranks, which can tax system resources and reduce compression and decompression performance.
Innovation Solution
A method that generates a value conversion dictionary using a parse tree based compression algorithm, estimates likelihoods for each value, assigns ranks, and reorders dictionary entries to create an architected dictionary, allowing for compression and decompression without additional memory lookups by walking from dictionary entries to ranks or vice versa.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If Ziv-Lempel and Huffman Coding are combined for data compression, then compression efficiency is improved, but memory access requirements increase and system performance deteriorates
Solution Approach 1:
The patent merges the symbol table from Ziv-Lempel compression with the code word assignments from Huffman coding into a unified data structure. This integration eliminates the need for separate memory lookups for symbol-to-rank translation, as the symbol table directly contains the compressed representation. The merging reduces memory access overhead while maintaining the compression efficiency benefits of combining both algorithms.
2Productivity
If symbol to rank translation is performed in combined compression systems, then data can be compressed, but additional memory accesses are required which increases overhead
Solution Approach 1:
The patent performs preliminary organization of the symbol table during the compression setup phase, arranging symbols in an order that corresponds to their rank assignments. This preliminary ordering eliminates the need for runtime memory searches to translate symbols to ranks, as the positional relationship is already established. The time complexity of symbol-to-rank translation is reduced from O(log n) or O(n) to O(1) by this preliminary arrangement.
3Productivity
If additional memory space is allocated for storing code words and ranks, then compression can be performed, but system memory requirements increase
Solution Approach 1:
The patent creates a unified data structure that serves multiple functions: it acts as both the symbol table for Ziv-Lempel compression and the code word assignment table for Huffman coding. This multi-functional structure eliminates the need for separate memory allocations for symbols, ranks, and code words. The same data structure is used for both compression and decompression operations, reducing overall memory requirements while maintaining full compression functionality.
Data Source
AI summary
A method, computer program product, and system includes a processor obtaining data including values and generating a value conversion dictionary by applying a parse tree based compression algorithm to the data, where the value conversion dictionary includes dictionary entries that represent the values. The processor obtains a distribution of the values and estimates a likelihood for each based on the distribution. The processor generates a code word to represent each value, a size of each code word is inversely proportional to the likelihood of the word. The processor assigns a rank to each code word, the rank for each represents the likelihood of the value represented by the code word; and based on the rank associated with each code word, the processor reorders each dictionary entry in the value conversion dictionary to associate each dictionary entry with an equivalent rank, the reordered value conversion dictionary comprises an architected dictionary.