Low-Complexity Entropy Encoder Without Sorting Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional entropy coding techniques, such as Huffman coding, require serial processing due to sorting operations, leading to higher implementation complexity and slower compression/decompression throughput in storage systems.
Innovation Solution
A low complexity encoder architecture that avoids sorting by using a frequency gathering system, preprocessing system, post processing system, and code table generation system to assign code lengths and swap them based on occurrence frequencies, enabling parallel processing for faster data compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional entropy coding techniques (e.g., Huffman coding) are used, then data compression is achieved, but implementation complexity increases and processing speed decreases due to required sorting operations
Solution Approach 1:
The patent extracts and removes the sorting operation from the traditional Huffman encoding process. By separating the code length assignment from the sorting requirement, the invention eliminates the serial processing bottleneck while maintaining compression effectiveness through alternative frequency-based code length determination methods
Solution Approach 2:
The invention changes the parameter of code length assignment methodology. Instead of using sorting-based methods that require serial processing, the patent employs direct frequency-to-code-length mapping and iterative optimization techniques that enable parallel processing, thereby improving throughput while managing complexity
2Productivity
If sorting operations are performed in entropy coding, then optimal code lengths are assigned, but serial processing is required which slows down compression throughput
Solution Approach 1:
The patent performs preliminary frequency gathering and code length assignment before the actual encoding process. By pre-calculating code lengths using frequency information without requiring sorting during the main encoding loop, the invention enables faster parallel processing while still achieving optimal compression ratios
Solution Approach 2:
The invention introduces dynamic code length adjustment through iterative optimization. Instead of static sorting-based assignment, the system dynamically refines code lengths based on frequency distribution, allowing parallel processing while adapting to the actual data characteristics for optimal compression
Data Source
AI summary
A storage infrastructure, method and encoder device for implementing low complexity encoding, The described encoder includes: a preprocessing system that assigns a code length to each unique symbol based on the frequency without performing a sort operation and determines maximum and minimum occurrence frequencies of symbols of each given code length, and the maximum and minimum code length among all the symbols; and a post processing system that cycles through each code length, determines if a maximum occurrence frequency of a current code length, associated with a first symbol, is greater than a minimum occurrence frequency of an adjacent code length, associated with a second symbol, and if greater, swaps code lengths of the first and second symbols.