Compression Dictionary Retention for Higher Lempel-Ziv Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques, such as the V.42 BIS modem communication standard, face inefficiencies due to dictionary overflow and underutilization, leading to suboptimal compression ratios and performance degradation, particularly in Lempel_Ziv compression where dictionary entries become invalid and are not effectively managed.
Innovation Solution
A method and system that delays the deletion of fixed-size code words from the dictionary when a match is found, using a dictionary with nodes and a history index to store and update entries, and employing strategies like cyclic indexing, doubly linked lists, or priority queues to efficiently manage dictionary space and maintain high compression ratios by eliminating unused entries and keeping frequently used ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If dictionary entries are deleted when the dictionary is full, then dictionary space is managed, but compression ratio deteriorates due to loss of frequently used entries
Solution Approach 1:
The patent changes the parameter of dictionary entry management from simple FIFO deletion to frequency-based retention. By tracking usage frequency and selectively retaining high-frequency entries while deleting low-frequency ones, the system maintains dictionary fullness while preserving compression efficiency. This is achieved through modifying the deletion criterion from positional (oldest entry) to attribute-based (lowest usage frequency).
Solution Approach 2:
The patent implements a feedback mechanism by continuously monitoring the usage frequency of dictionary entries. This feedback information is used to dynamically adjust which entries are retained or deleted, ensuring that the dictionary adapts to the actual data patterns. The system uses this feedback to prevent deletion of frequently used entries that would harm compression ratio while still managing dictionary space effectively.
2Quantity of substance
If dictionary size is increased to avoid overflow, then more entries can be stored, but memory usage increases and processing complexity increases
Solution Approach 1:
The patent applies dynamics by making the dictionary content dynamic rather than static. Instead of increasing fixed dictionary size, the system dynamically adjusts which entries are present in the dictionary based on their usage frequency. This allows the effective dictionary capacity to adapt to actual data patterns without permanently allocating additional memory resources, thus avoiding increased processing complexity associated with larger fixed-size dictionaries.
3Quantity of substance
If old entries are deleted to make space for new entries, then dictionary space is maintained, but previously useful compression patterns are lost
Solution Approach 1:
The patent changes the deletion parameter from entry age (FIFO) to entry utility (usage frequency). This ensures that entries are deleted based on their actual contribution to compression performance rather than their position in the dictionary. High-frequency entries that provide reliable compression patterns are retained even if they are 'old' in terms of when they were added, while low-frequency entries are deleted to make space, thus maintaining both dictionary space utilization and compression performance stability.
Data Source
AI summary
An improved data compression system and method is disclosed. The data compression method uses smaller fixed bit words to represent streams of digital data. The fixed bit words are sent instead of the streams of digital data. A table relating the fixed bit words and streams of digital data is provided as a dictionary. Each entry has a node which is indexed in a stored index. As additional fixed bit words are established for new streams of digital data, older fixed bit words are deleted from the dictionary. The older fixed bit words are prevented from being deleted if they are used for subsequent streams of data. There are a number of methods to determine how the older fixed bit words are retained which are disclosed.