Dictionary Compression Matching Beyond Parallel Search Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional parallel search methods in dictionary encoding have an upper limit for match length, limiting the achievable compression ratio and throughput in data compression.
Innovation Solution
A data compression device that includes a dictionary match determination unit, an extended matching generator, a match selector, and a match connector, which searches for matches across multiple sequences of varying lengths, allowing for extended matching and combining consecutive matches with the same offset to enhance compression ratio beyond the traditional substring-by-substring limits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional parallel search method is used, then throughput is improved, but compression ratio deteriorates due to upper limit on match length
Solution Approach 1:
The patent divides the matching process into two distinct stages: a parallel search stage that processes multiple sequences simultaneously to ensure high throughput, and a sequential extension stage that extends matches beyond the parallel search limit. This segmentation allows each stage to optimize for its specific function without compromising the other.
Solution Approach 2:
The patent implements continuous matching by extending matches that exceed the parallel search unit length. The extension process continues sequentially until no further extension is possible, ensuring that the full potential of each match is realized without interruption, thereby improving compression ratio while maintaining the throughput benefits of parallel processing.
2Loss of information
If match length is extended beyond substring length, then compression ratio is improved, but device complexity increases
Solution Approach 1:
The patent performs preliminary parallel search to identify candidate matches within the search unit length before extending them sequentially. This preliminary action filters out non-matching candidates early, reducing the workload for the extension stage and preventing exponential complexity growth.
Solution Approach 2:
The patent employs dynamic control of the matching process by conditionally extending matches only when they exceed the parallel search unit length. The extension process adapts to each match's specific characteristics, extending only as far as necessary, which optimizes resource usage and prevents unnecessary complexity.
3Productivity
If multiple sequences are searched in parallel, then productivity is improved, but measurement precision of match length deteriorates due to upper limit
Solution Approach 1:
The patent segments the match length determination into two parts: the parallel search phase that identifies matches up to the search unit length, and the sequential extension phase that determines the additional match length beyond that limit. This segmentation allows each phase to operate optimally within its constraints while collectively achieving precise full match length measurement.
Solution Approach 2:
The patent uses the parallel search results as an intermediary step that provides initial match information, which then serves as the basis for the sequential extension process. This intermediary approach allows the system to leverage the speed of parallel processing while ultimately achieving precise match length measurement through the extension phase.
Data Source
AI summary
According to one embodiment, a data compression device includes a dictionary match determination unit, an extended matching generator, a match selector and a match connector. The dictionary match determination unit searches for first past input data matching first new input data. The extended matching generator compares second past input data subsequent to the first past input data with second new input data subsequent to the first new input data. The match selector generates compressed data by replacing a part of the input data with match information output from the dictionary match determination unit or the extended matching generator. The match connector replaces a plurality of match information in the compressed data with single match information.