Hardware Decompression Using Predefined Dictionaries for Small Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital computer systems face inefficiencies in data compression and decompression, particularly with small data sizes, as current algorithms like LZ77-type technology struggle to find significant patterns, leading to suboptimal compression ratios and increased latency.
Innovation Solution
A computer system with a hardware accelerator that utilizes pre-defined compression dictionaries stored in an internal millicode storage area, allowing the operating system to select and deliver the appropriate dictionary for decompression, reducing memory allocation overhead and improving performance for small data records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If LZ77-type compression algorithms are used to compress small data sizes, then the compression process can be performed, but the compression ratio is suboptimal and latency increases
Solution Approach 1:
The patent pre-calculates and stores compression ratios for different data sizes and compression algorithms in a lookup table during system initialization. When compression is needed, the system quickly queries this pre-computed table to determine the optimal algorithm and parameters, avoiding the time-consuming process of real-time pattern matching and compression ratio calculation that plagues traditional LZ77 algorithms with small data sizes.
Solution Approach 2:
The patent segments the compression problem by creating separate optimization paths for different data size ranges. Instead of using a single LZ77 algorithm for all data sizes, the system divides data into categories (e.g., small, medium, large) and applies different compression strategies to each segment, allowing small data to use more efficient methods while large data benefits from traditional compression algorithms.
2Productivity
If traditional compression algorithms perform extensive pattern matching, then compression can be achieved, but memory allocation overhead increases and performance decreases for small data records
Solution Approach 1:
The system performs memory allocation and compression parameter setup in advance by pre-computing compression ratios and storing them in lookup tables. This eliminates the need for dynamic memory allocation during the actual compression of small data records, reducing overhead and improving performance.
Solution Approach 2:
The patent uses lightweight, pre-allocated compression structures that can be quickly instantiated and discarded for small data records. Instead of allocating complex data structures and performing extensive pattern matching for each small record, the system uses simplified compression paths with pre-computed parameters that minimize memory usage and allocation overhead.
Data Source
AI summary
A computer system includes a hardware controller and an internal millicode storage area. The controller includes an accelerator that decompresses a data stream requested by an application. The internal millicode storage area can store a compression dictionary library including a plurality of different pre-defined compression dictionaries. A host system includes a dictionary manager that determines a compression dictionary from the plurality of different pre-defined compression dictionaries included in the dictionary library to decompress the data stream. The accelerator can access the internal millicode storage area to obtain the determined compression dictionary, and to decompress the data stream according to the determined compression dictionary.