Parallel History Search Encoding for Dictionary Compression Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression techniques face challenges in achieving high compression speed and ratio simultaneously, particularly in dictionary-based methods where history search is a bottleneck, limiting parallelism and efficiency.
Innovation Solution
A circuit and method for parallel history search using a dictionary-based approach with a parallel processor that implements a sorting technique to generate tokens from an input data stream, followed by an encoder that coalesces copy and literal tokens to produce a compressed output stream, utilizing graphic processing units or parallel central processing units for enhanced processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If dictionary-based compression methods are used to achieve high compression ratio, then compression speed deteriorates due to history search bottleneck
Solution Approach 1:
The patent divides the history search process into multiple independent parallel search operations, where different segments of the input data stream are searched simultaneously against different segments of the history buffer. This segmentation eliminates the sequential bottleneck and enables high-speed parallel processing while maintaining comprehensive history search coverage for high compression ratios.
Solution Approach 2:
The patent transitions from sequential single-threaded history search to parallel multi-threaded search by adding the dimension of concurrency. Multiple processor cores or GPU threads perform history search operations simultaneously, effectively transforming the time dimension into a parallel processing architecture that achieves both speed and compression quality.
2Productivity
If parallel processing is increased to enhance compression speed, then device complexity worsens
Solution Approach 1:
The patent designs a universal parallel processing architecture that can handle both history search and token encoding operations. The same parallel processor or GPU core performs multiple functions including generating tags, sorting, comparing, and encoding, reducing the need for separate dedicated hardware for each function and thereby managing complexity while maintaining high speed.
Solution Approach 2:
The patent implements self-service mechanisms where the parallel processing system automatically manages its own task distribution, memory access, and result aggregation. The architecture includes built-in synchronization and coordination protocols that enable the system to organize its own parallel operations without external control, reducing the complexity of managing parallelism.
3Quantity of substance
If history search scope is expanded to improve compression ratio, then processing time increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing the input data stream to generate tags that represent potential match patterns before the main history search. These pre-generated tags are organized and sorted in advance, allowing the parallel history search to quickly locate relevant history segments without examining the entire history buffer sequentially, thus expanding search scope without proportionally increasing processing time.
Data Source
AI summary
Systems, devices and methods for data compression using history search for dictionary based compression. Systems, devices and methods may use parallel processing techniques for data compression and encoding. Systems, devices and methods may provide memory search techniques for hardware.