Compression Engine Match Limiting for Quality-Bandwidth Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data compression engines face a trade-off between throughput speed and compression quality, as thorough searching for repeated data patterns increases processing time, reducing bandwidth, and existing methods speculate potential matches but lack efficiency, especially for poor compressibility datasets.
Innovation Solution
The system employs a pipeline operation with a control input to limit the number of potential string matches processed, using parallel memory and redundant operations to improve throughput, and speculates on match absence to initiate searches before full results are known, allowing for concurrent processing of multiple matches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the compression engine performs thorough searching for repeated data patterns, then compression quality is improved, but throughput speed decreases
Solution Approach 1:
The patent implements dynamic control of search thoroughness by adjusting the number of candidate matches processed based on data characteristics. The system transitions from static predetermined search parameters to dynamic adaptation, processing more candidates when data shows high compressibility and fewer candidates when compressibility is low, thereby optimizing the trade-off between compression quality and throughput speed for different data types
Solution Approach 2:
The system changes the parameter of search thoroughness (number of candidate matches processed) based on detected data compressibility characteristics. By monitoring data patterns and adjusting the search parameter dynamically, the system achieves high compression quality for compressible data while maintaining high throughput for incompressible data, resolving the contradiction between search thoroughness and processing speed
2Measurement precision
If the compression engine processes more potential string matches, then compression quality is improved, but processing time increases
Solution Approach 1:
The patent applies partial action by processing only a limited number of candidate matches (e.g., top 1-3 candidates) rather than exhaustively searching all potential matches. This partial processing approach achieves sufficient compression quality for most practical applications while dramatically reducing processing time, as the majority of compression gains come from finding the best few matches rather than all possible matches
3Measurement precision
If the search engine performs comprehensive matching operations, then compression quality is improved, but bandwidth is reduced
Solution Approach 1:
The system performs preliminary analysis of data compressibility characteristics before initiating full search operations. By pre-assessing whether the data is likely to be compressible based on initial patterns, the system can decide whether to perform thorough searching or use faster approximate methods, thereby maintaining high bandwidth while achieving good compression quality when it matters most
Data Source
AI summary
The disclosure provides a system and method to vary bandwidth/speed of a compression engine to tradeoff with compression quality. The system comprises an input port receiving a data stream having a current byte and a stream of preceding and subsequent bytes thereof; a history memory storing the stream of preceding bytes in blocks of history data an index engine having a table of keys associated to memory addresses of the blocks, the index engine accessing the table to output a plurality of potential string matches upon entry of the current byte; a search engine processing the potential string matches by fetching the blocks of history data associated with each of the plurality of potential string matches and comparing a portion of each of the blocks to the current byte to output a string upon matching consecutive bytes in the block to the current byte and subsequent bytes thereof; and a control input limiting the number of potential string matches processed by the search engine.