Parallel Data Compression Pipelines Beyond Feedback Loop Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional lossless data compression algorithms, such as Lempel-Ziv techniques, often suffer from low throughput due to critical feedback loops, limiting data compression and decompression rates to one symbol or byte per clock cycle, which restricts their application in high-speed data processing.
Innovation Solution
Implementing pipelined processing with parallel processing techniques that allow for the simultaneous comparison and encoding of multiple symbols, discarding invalid results, and bundling codewords to achieve a defined rate interface, thereby breaking the critical feedback loop and ensuring a minimum throughput of more than one symbol or byte per clock cycle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional lossless data compression algorithms are used, then data compression is achieved, but throughput is limited to one symbol or byte per clock cycle due to critical feedback loops
Solution Approach 1:
The patent segments the data processing into multiple independent parallel pipelines, where each pipeline processes a portion of the input data simultaneously. This segmentation eliminates the single-critical-feedback-loop bottleneck by creating multiple independent processing paths that can operate concurrently, thereby increasing throughput beyond one symbol or byte per clock cycle.
Solution Approach 2:
The patent transitions from sequential single-dimensional processing to parallel multi-dimensional processing by implementing multiple processing pipelines that operate simultaneously. This dimensional change from time-sequential to space-parallel architecture allows multiple data elements to be processed in the same clock cycle, breaking the throughput limitation imposed by the traditional critical feedback loop.
2Productivity
If parallel processing is implemented to increase throughput, then data processing speed is enhanced, but validity determination of parallel results becomes complex
Solution Approach 1:
The patent applies preliminary action by pre-computing validity indicators alongside parallel processing results. Each parallel processing pipeline generates not only its computational result but also a validity flag that indicates whether the result is valid for use. This preliminary validation approach eliminates the need for complex post-processing validity checks, as the validity information is already determined and attached to each result before the parallel processing completes.
3Productivity
If pipelined processing with parallel operations is used, then throughput exceeds one symbol per clock cycle, but processing of previous results must be completed before new processing begins
Solution Approach 1:
The patent implements continuity of useful action through overlapping parallel pipelines operating in a coordinated manner. While one pipeline is completing its processing, another pipeline has already begun processing the next set of data. This continuous overlapping execution eliminates idle waiting time between processing stages, ensuring that the system continuously performs useful work at maximum throughput without interruptions for result collection or validation.
Data Source
AI summary
The present disclosure includes apparatus, systems and techniques relating to pipelined processing. In some implementations, a method performed by a data processing device includes storing data in a memory module. The method includes processing the stored data in accordance with a compression algorithm to produce processed data. Processing the stored data includes pipelined processing of a defined number of symbols of the stored data in parallel, and discarding results of the pipelined processing that are rendered invalid by other results of the pipelined processing. Additionally, the method includes outputting the processed data.