Dictionary Compression Accelerator for Parallel Decompression Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer processors face limitations in decompression throughput due to the serial nature of lossless compression algorithms like Deflate, which restricts decompression to a single symbol per cycle, despite having multiple cores and specialized hardware.
Innovation Solution
A dictionary compression accelerator is introduced, which includes a compression/decompression accelerator that performs page-level compression and decompression, utilizing a preset dictionary to enhance compression efficiency and a history buffer with a hash table to match data patterns, allowing for parallel processing and improved throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If lossless compression algorithms like Deflate are used, then compression ratio is improved, but decompression throughput deteriorates due to serial processing limitations
Solution Approach 1:
The decompression process is divided into multiple independent parallel pipelines, each capable of processing different symbols simultaneously. The single serial Deflate algorithm is segmented into multiple parallel execution paths that can operate concurrently on different data streams, breaking the throughput bottleneck while maintaining compression efficiency.
Solution Approach 2:
The patent transitions from single-core serial processing to multi-core parallel processing by adding a spatial dimension to the decompression operation. Multiple cores process multiple symbols in parallel, transforming the throughput limitation from a single-dimensional serial constraint to a multi-dimensional parallel architecture that achieves both high compression ratio and high throughput.
2Power
If multiple cores and specialized hardware are available, then processing capacity is improved, but serial algorithm limitations cause throughput to deteriorate
Solution Approach 1:
The accelerator is designed with universal multi-functional units that can handle both compression and decompression operations across multiple data streams simultaneously. Each processing unit is configured to perform Deflate operations on multiple symbols in parallel, making full use of available processing capacity and transforming idle hardware resources into productive throughput.
Solution Approach 2:
The system performs preliminary configuration of multiple parallel decompression pipelines before processing begins, pre-initializing state machines and buffers for each parallel stream. This preliminary setup enables immediate parallel processing upon data arrival, fully utilizing processing capacity without serialization delays.
3Device complexity
If serial processing is used to maintain algorithm simplicity, then device complexity is reduced, but decompression throughput deteriorates
Solution Approach 1:
The patent creates multiple identical copies of the Deflate decompression state machine, each copy processing a different symbol in parallel. These copied state machines maintain the simplicity of the original serial algorithm while achieving parallel throughput by replicating the proven simple design across multiple independent processing units.
Data Source
AI summary
Apparatus and method for dictionary accelerator compression. For example, one embodiment of an apparatus comprises: a plurality of cores; a compression/decompression accelerator coupled to or integral to one or more of the plurality of cores, the compression/decompression accelerator to perform decompression and compression operations in response to read and write operations, respectively, wherein responsive to notification of a compression job to compress a memory page or a portion thereof, a history buffer associated with the compression/decompression accelerator to is to be initialized with pre-configured dictionary data, the compression/decompression accelerator to match portions of the pre-configured dictionary data with portions of the memory page to generate compressed output data.