Compressed Block Map for Dense Data Structures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face inefficiencies due to high write amplification in solid state disk drives, which increases write overhead and reduces SSD lifespan, and metadata consumption leads to increased I/O operations and slower system performance.
Innovation Solution
Implementing delta encoding for key-value entries in a distributed resources system, where each key is encoded as a delta from the previous key with leading zero bits removed, and storing these entries in memory to minimize write operations and reduce metadata storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If delta encoding is applied to key-value entries, then metadata storage space is reduced, but encoding complexity increases
Solution Approach 1:
The patent applies parameter changes by transforming key-value entries through delta encoding, where each key is encoded as the difference from the previous key. This changes the representation parameter from absolute values to relative differences, significantly reducing the number of bits required to store metadata while managing encoding complexity through systematic bit manipulation operations.
2Duration of action of stationary object
If write operations are minimized in SSDs, then SSD lifespan is extended, but system performance may be reduced
Solution Approach 1:
The patent implements preliminary action by performing delta encoding on key-value entries before writing them to the datastore. This preprocessing step compresses the metadata size, which subsequently reduces the number of write operations required to maintain the same level of system functionality. By preparing data in advance with reduced size, the system extends SSD lifespan through fewer write cycles while maintaining performance through efficient memory utilization.
3Loss of time
If metadata size is reduced, then I/O operations are reduced, but data compression overhead increases
Solution Approach 1:
The patent applies parameter changes by transforming metadata representation from absolute key values to delta-encoded relative differences. This parameter transformation reduces metadata size, which directly decreases the amount of data that needs to be read and written during I/O operations. The compression overhead is managed through efficient bit manipulation operations that encode each key as a delta from the previous key, removing leading zero bits to achieve compact representation.
Data Source
AI summary
Embodiments of the disclosure provide techniques for creating a compressed mapping structure in a system of resources. For example, a distributed resources system may use delta encoding to store, in memory, numerous entries of dense data structures in the system. In a compressed block of such entries, the distributed resources system encodes the key of each entry as the delta from the key of the previous entry. The content of each entry is encoded similarly. The distributed resources system suppresses the leading zero bits of each resulting field.