LZ77 Run-Aware Hash Chaining for Faster Data Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
LZ77-based compression algorithms face inefficiencies due to high search efforts in large files with long runs of literals, leading to variance in compression performance across different file types, with current implementations spending excessive resources on searching for matches in files with long runs of literals.
Innovation Solution
The proposed solution involves detecting runs of literals and using previous search results to calculate results for remaining bytes, reducing the number of searches needed and skipping updates to hash chains within runs, thereby optimizing the compression process by focusing efforts on the initial positions of runs and adjusting match distances for improved compression ratios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If LZ77 compression searches all locations in the history buffer to find the longest match, then the compression ratio is improved, but the processing time and computational effort increase significantly
Solution Approach 1:
The patent segments the history buffer into multiple zones (e.g., recent history, older history) and applies different search strategies to each zone. For recent history where matches are more likely, exhaustive search is performed. For older history, limited or heuristic search is applied, reducing overall processing time while maintaining acceptable compression ratios.
Solution Approach 2:
The patent applies different search depths and strategies to different locations in the history buffer based on their likelihood of containing matches. Locations closer to the current position receive more thorough search, while distant locations receive lighter search, optimizing the balance between compression ratio and processing time.
2Manufacturing precision
If LZ77 compression performs exhaustive searches at high compression levels, then the compression ratio is maximized, but the number of searches per byte position increases to up to 4K
Solution Approach 1:
The patent dynamically adjusts the search effort and compression level based on the characteristics of the input data. For data with long runs of literals, the search effort is reduced since matches are unlikely. For highly repetitive data, full search effort is applied. This dynamic adaptation maintains high compression ratios when needed while improving speed for other cases.
Solution Approach 2:
The patent changes search parameters such as hash table size, search depth, and number of hash chains explored based on the detected characteristics of the input stream. When long runs of literals are detected, parameters are adjusted to reduce search effort, thereby improving compression speed without significantly impacting compression ratio.
3Manufacturing precision
If LZ77 compression searches for matches in every byte position, then the compression ratio is improved, but the effort is excessive for files with long runs of literals
Solution Approach 1:
The patent performs preliminary analysis of the input data to detect patterns such as long runs of literals before committing to exhaustive search. When such patterns are detected, the search strategy is adjusted in advance to avoid unnecessary computational effort, while still maintaining acceptable compression ratios through alternative encoding methods.
Data Source
AI summary
Methods and apparatus are described by for compressing data using LZ77 compression. Embodiments determine an initial run from input data. The initial run includes repeating data at a first location and has a first length. A hash chain is updated with a proper set of hashes from prefixes from the initial run. A first search engine determines a second run that includes the repeating data at a second location. The second run has a second length less than the first length. A first matching location is determined within the input data having the repeating data using the hash chain and the second run. The first matching location is the first location. The first matching location, the second location, and the second length are written to an output buffer. The output buffer includes a compressed version of the input data.