Hash Table Address Segmentation in Sliding Window Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing hash-based sliding window compression technology requires significant storage space for historical hash keys and their addresses, which is inefficient and costly.
Innovation Solution
A simplified data structure for the hash table is introduced, using a hash line storage format that reduces storage space by employing a prefix address column and offset columns, where each entry stores a hash key and an offset, and a valid flag column to indicate valid or invalid data, allowing for efficient storage and compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a conventional hash table stores complete addresses for all historical hash keys, then address lookup accuracy is maintained, but storage space consumption increases significantly
Solution Approach 1:
The address is segmented into two parts: a prefix address (common to all entries in a hash line) and a per-entry offset. This segmentation allows the hash table to store only the prefix address once per hash line rather than repeating it in every entry, significantly reducing storage space while maintaining the ability to reconstruct complete addresses for accurate lookup.
Solution Approach 2:
The invention introduces a hierarchical address representation by adding the dimension of address decomposition. Instead of storing addresses in a single flat dimension, the system uses a two-level structure (prefix + offset) that separates common and unique address components, enabling space-efficient storage without sacrificing lookup precision.
2Manufacturing precision
If the hash table stores all historical hash key information in detail, then compression accuracy is improved, but device complexity increases
Solution Approach 1:
The invention extracts the common prefix address component from each individual entry and stores it separately at the hash line level. This extraction eliminates redundant storage of identical address prefixes in every entry, simplifying the data structure while preserving all necessary information for accurate compression operations.
Solution Approach 2:
The invention merges the storage of common address information at the hash line level rather than repeating it in each entry. By combining the prefix address storage with the hash line structure and integrating offsets into existing entry fields, the system reduces overall data structure complexity while maintaining compression accuracy.
Data Source
AI summary
A data compressor with a hash computing hardware configured to evaluate the hash value for the current hash key extracted from a source data string, obtain a hash line corresponding to the hash value from a hash table, and perform hash key comparison to find at least one matching hash key. The hash line includes a prefix address column that stores a prefix address. Each entry of the hash line is provided to store a hash key and an offset. The hash computing hardware evaluates an address of the at least one matching hash key by combining the prefix address and an offset of the at least one matching hash key, and the offset of the at least one matching hash key is obtained from an entry storing the at least one matching hash key.