Reconfigurable Compression Engine for Window Size and Search Depth Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression technologies face challenges in supporting multiple lossless compression algorithms while maintaining silicon die area and power consumption constraints, particularly in achieving high compression ratios and throughput with varying window sizes and search depths.
Innovation Solution
A reconfigurable compression engine that supports multiple lossless algorithms like zstd and zlib, with adaptive history buffer sizes and search depths, reusing search engines and reconfiguring memory to optimize for different compression schemes, allowing operation in high-throughput and high-compression-ratio modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If larger window sizes and deeper search depths are used to achieve higher compression ratios, then compression ratio is improved, but silicon die area and power consumption increase
Solution Approach 1:
The compression engine is designed to be reconfigurable, allowing dynamic adjustment of window size and search depth parameters. The history buffer size can be changed between 64KB and 256KB, and the number of parallel search engines can be adjusted between 1 and 8, enabling the system to adapt its resource usage based on compression requirements without requiring multiple fixed-configuration engines
Solution Approach 2:
A single reconfigurable compression engine can perform multiple compression algorithms (zlib, zstd, lz4) by changing its configuration parameters. The same hardware resources (history buffer, search engines, hash tables) can be allocated differently to support various algorithms with different window sizes and search depths, eliminating the need for separate dedicated engines for each algorithm
2Adaptability or versatility
If multiple compression algorithms are supported with different window sizes and search depths, then adaptability is improved, but device complexity increases
Solution Approach 1:
Different compression algorithms are supported by changing key parameters such as history buffer size (64KB vs 256KB), number of parallel search engines (1-8), and hash table configuration. For example, zlib uses 64KB window with 16 searches while zstd uses 256KB window with 64 searches. These parameter changes allow a single engine to implement multiple algorithms without requiring separate hardware for each
3Manufacturing precision
If more search engines are allocated to support larger window sizes, then compression ratio is improved, but power consumption increases
Solution Approach 1:
The number of parallel search engines is dynamically configurable based on the required compression level and algorithm. The system can operate with 1 search engine for low-power modes or scale up to 8 parallel search engines for high-compression-ratio modes. This dynamic allocation allows the system to optimize power consumption according to real-time compression requirements rather than always operating at maximum power
Data Source
AI summary
Examples described herein relate to an encoder circuitry to apply one of multiple lossless data compression schemes on input data. In some examples, to compress input data, the encoder circuitry is to utilize a search window size and number of searches based on an applied compression scheme. In some examples, content of a memory is reconfigured to store data corresponding to a search window size of the applied compression scheme. In some examples, an applicable hash function is configured based on the applied compression scheme. In some examples, a number of searches are made for a byte position. In some examples, the encoder circuitry includes a hash table look-up and a bank decoder. In some examples, the hash table look-up is to generate a hash index to identify an address of an entry in the search window. In some examples, the bank decoder is to select a bank based on the hash index.