Parallel Huffman Decompression Using Block Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression technologies face challenges in efficiently reconstructing original data without loss, particularly in lossless compression methods, and there is a need for improved methods to process compressed data in parallel to enhance efficiency.
Innovation Solution
A processor-implemented method involving Huffman coding and metadata-based processing is used to divide data into blocks, generate sub-compressed data streams, and utilize decoding engines to reconstruct the data efficiently, allowing for parallel processing of compressed data streams.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is divided into blocks and processed sequentially, then data integrity is maintained, but processing speed decreases
Solution Approach 1:
The patent divides the compressed data stream into multiple blocks, where each block contains a segment of the data and metadata for error detection. This segmentation allows parallel processing of multiple blocks simultaneously while maintaining data integrity through individual block validation, thus resolving the contradiction between sequential processing reliability and parallel processing speed.
Solution Approach 2:
The patent performs preliminary error detection by appending metadata (such as CRC checksums) to each block during the compression phase. This preliminary action enables rapid validation of each block during parallel decompression without requiring sequential verification, thereby maintaining data integrity while enhancing processing speed.
2Productivity
If parallel processing is implemented, then processing speed increases, but computational overhead increases
Solution Approach 1:
By segmenting the data stream into independent blocks with self-contained metadata, the patent enables parallel processing units to work simultaneously on different blocks without requiring complex inter-processor communication or synchronization mechanisms, thus increasing processing speed while minimizing additional computational overhead.
Solution Approach 2:
The patent introduces metadata (such as block length indicators and error detection codes) as an intermediary element that facilitates parallel processing by providing each processing unit with the necessary information to independently validate and process its assigned block, reducing the need for complex coordination overhead between parallel processors.
3Quantity of substance
If compression ratio is increased, then storage efficiency improves, but decompression complexity increases
Solution Approach 1:
The patent divides the compressed data into blocks with embedded metadata that records essential decoding information for each block. This segmentation allows the decompression process to work on smaller, manageable units with self-contained information, reducing the overall decompression complexity while maintaining high compression ratios through efficient block-level encoding.
Solution Approach 2:
The patent performs preliminary encoding operations during compression, including the calculation and attachment of metadata (such as block lengths and error detection codes) to each block. This preliminary action simplifies the decompression process by providing all necessary decoding information in advance, reducing decompression complexity while preserving storage efficiency.
Data Source
AI summary
A processor-implemented method including generating k sub-compressed data streams based on a compressed data stream for a plurality of symbols divided into a plurality of k blocks and count information for each of the plurality of k blocks, generating k sub-symbols by processing each of the k sub-compressed data streams using k decoding engines, metadata about the compressed data stream, and generating an output data stream corresponding to the plurality of symbols based on the k sub-symbols.