Lossless Data Compression Using Variable-Length Dictionary Codes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods are inefficient in identifying and representing repeated sequences within data streams, leading to suboptimal compression ratios and increased storage requirements.
Innovation Solution
The method employs dictionary coding using the Lempel-Ziv 77 algorithm, followed by variable length coding of control codes with Elias Delta codes, and speculative string matching to generate control codes comprising literals and indices with variable length coded length values, allowing for efficient representation of repeated data sequences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If dictionary coding is applied to compress data, then compression ratio is improved, but device complexity increases due to the need for modelers and encoders
Solution Approach 1:
The compression system is segmented into distinct functional modules: a modeler that identifies repeated sequences and generates control codes, and an encoder that variable-length encodes these control codes. This segmentation allows each module to specialize in a specific task, improving overall compression efficiency while maintaining manageable complexity through modular design.
Solution Approach 2:
The modeler performs preliminary analysis of the input data stream to identify repeated sequences before the encoding stage. By pre-processing the data to generate control codes that mark repetitions, the system prepares the data in advance for efficient compression, separating the pattern recognition function from the actual compression function.
2Loss of substance
If variable length coding is applied to control codes, then compression ratio is improved, but processing time increases
Solution Approach 1:
Variable length encoding is applied selectively only to specific portions of the control codes (the length fields indicating repetition lengths), rather than encoding the entire control code structure. This local application of variable length encoding compresses the most variable parts of the data while keeping the overall processing time manageable by avoiding full-variable-length encoding of all control code elements.
3Loss of substance
If speculative string matching is used to identify repeated sequences, then compression ratio is improved, but computational complexity increases
Solution Approach 1:
The modeler performs preliminary scanning of the input data stream to identify potential repeated sequences before committing to full matching. By speculatively identifying patterns and then verifying them, the system efficiently handles the complexity of string matching by breaking it into preliminary detection and confirmation stages, reducing the overall computational burden.
Data Source
AI summary
An embodiment of a method for compressing data includes variable length coding one or more values of control codes generated from dictionary coding the data, wherein the control codes comprise literals and indices, wherein each index comprises a length value and a pointer to previously read in data. An embodiment of a system for losslessly compressing input data includes a modeler operable to determine whether a set of input data matches prior input data, an encoder operable to generate control codes including one or more literal control codes and one or more index control codes, wherein each index control code includes a pointer to a location in the input data and a length value indicating a number of bytes to copy from the location, and wherein the length value is variable length coded.