Short Hash Handle Generation for Content-Based Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems, particularly primary storage solutions, face challenges in supporting content-based storage with its advantages due to intensive computational load, block fragmentation, inability to maintain sequential data location, and algorithmic difficulties in distributing tasks across processing and storage elements, leading to performance degradation.
Innovation Solution
A method and apparatus utilizing hash signatures and short hash handles to efficiently manage data storage through content addressing, separating control and data planes, and distributing computational tasks across multiple nodes to achieve scalable and high-performance storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If content-based storage is implemented, then storage capacity efficiency is improved, but computational load increases
Solution Approach 1:
The patent segments the computational task of content-based mapping into two parts: a hash function computation that can be distributed and cached, and a final mapping operation. The hash function results are pre-computed and stored in cache, reducing the computational load during actual storage operations while maintaining content-based deduplication efficiency.
Solution Approach 2:
The patent performs preliminary computation of hash functions and stores the results in a cache structure before they are needed for actual mapping operations. This preliminary action reduces the computational burden during runtime when data needs to be stored or retrieved, resolving the contradiction between computational load and storage efficiency.
2Measurement precision
If large blocks are broken into smaller blocks for content addressing, then content addressing granularity is improved, but performance degrades
Solution Approach 1:
The patent introduces a new dimension by using a hash function that maps content to a compact representation space. Instead of directly comparing large blocks, the system computes hash values that serve as proxies, enabling fine-grained content addressing without the performance penalty of processing entire large blocks.
3Speed
If address-based mapping is used, then sequential data location is maintained, but storage capacity efficiency decreases
Solution Approach 1:
The patent introduces hash function results as an intermediary between content and physical storage location. The hash value serves as a mediator that enables content-based identification while the patent's caching mechanism maintains sequential access performance by pre-computing and storing the mapping relationships.
4Adaptability or versatility
If content-based mapping is distributed across multiple processing elements, then scalability is improved, but algorithmic complexity increases
Solution Approach 1:
The patent segments the content-based mapping algorithm into distributable components: hash function computation that can be performed independently on different processing elements, and a coordination layer that manages the distributed hash table. This segmentation enables scalability while keeping individual algorithmic components relatively simple.
Data Source
AI summary
In one aspect, a method includes receiving hash signature, determining if there is an existing hash signature that matches the hash signature received, determining if there is an existing short hash handle that matches the hash signature received, designating n-1 least significant bits of the hash signature to be n-1 least significant bits of a n-bit short hash handle if there is not an existing short hash handle that matches the hash signature received and adding a hash handle bit to the n-1 bits to form the n-bit short hash handle.