String Search CAM Lazy Match Evaluation for Better Deflate Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The deflate compression algorithm, used in ZLIB and GZIP, faces inefficiencies in searching for iterative character sequences due to the large amount of character sequence comparison processing, which can lead to long processing times and incomplete searches, especially when many sequences share the same hash value.
Innovation Solution
An apparatus utilizing content addressable memory (CAM) to determine a search start point by generating row presence information, allowing for efficient compression by replacing partial data sequences with information on the position and length of matching sequences, thereby improving compression ratios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If software uses the hash method to reduce search time, then processing speed is improved, but search completeness deteriorates because some character sequences are discarded
Solution Approach 1:
The patent segments the search process into two distinct phases: a hashing phase for quick filtering and a CAM verification phase for complete validation. The hashing phase divides the search space into buckets, while the CAM phase independently verifies each candidate without discarding any potential matches, thus resolving the contradiction between speed and completeness.
Solution Approach 2:
The patent introduces CAM as an intermediary component between the hashing mechanism and the final match confirmation. The CAM acts as a verification layer that takes hash output addresses and confirms whether actual matches exist at those locations, preventing premature discarding of sequences while maintaining the speed benefits of hashing.
2Reliability
If the whole character sequence is searched to ensure completeness, then search accuracy is improved, but processing time increases significantly
Solution Approach 1:
The patent segments the search space using hashing to create discrete buckets, allowing the system to focus verification efforts only on specific locations indicated by hash values. This segmentation prevents the need to examine every possible position in the search window, reducing processing time while maintaining accuracy through CAM verification of the segmented candidates.
Solution Approach 2:
The patent performs preliminary hashing to pre-identify candidate match locations before actual verification. This preliminary action filters out non-matching positions in advance, so that the time-consuming verification process only needs to check a small subset of promising candidates, thereby reducing overall processing time while ensuring no matches are missed.
3Loss of information
If up to 32 kilobytes of preceding data is searched for iterative sequences, then compression ratio is improved, but the amount of comparison processing increases
Solution Approach 1:
The patent segments the 32KB search window into multiple 1KB segments, each with its own dedicated CAM. This segmentation divides the complex comparison task into smaller, parallelizable units, reducing the complexity of individual comparison operations while maintaining the ability to search the entire 32KB window for better compression ratios.
Solution Approach 2:
The patent transitions from a single-dimensional sequential search to a multi-dimensional parallel search structure. Multiple CAMs operate simultaneously on different segments of the search window, adding a parallelism dimension that reduces the effective comparison complexity while preserving the comprehensive search capability needed for high compression ratios.
Data Source
AI summary
A plurality of stored data sequences that match one or more search data sequences are determined. Each of the stored data sequences of the plurality of stored data sequences comprise a plurality of data elements and the stored data sequences are stored in a content addressable memory array. A longest stored data sequence of the plurality of stored data sequences is determined using a plurality of tracing circuits. An address associated with the longest stored data sequence of the plurality of stored data sequences is determined. A count of data elements of the longest stored data sequence of the plurality of stored data sequences is determined.