Multi-Tier Metadata Hashing for All-Flash Array Capacity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage solutions, particularly primary storage systems, face challenges in achieving content-based storage with ultra-high performance due to intensive computational load, block fragmentation, loss of sequential data location, and algorithmic difficulties in distributing tasks across multiple processing and storage elements.
Innovation Solution
A method and system implementing content addressable storage with separate control and data planes, using hashing for content addressing, and distributing computational and data operations across multiple nodes to achieve scalable and efficient data management, including inline deduplication and optimal resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If content-based storage is implemented, then storage capacity efficiency is improved, but computational load increases
Solution Approach 1:
The patent divides the storage system into multiple independent nodes, each capable of performing content-based storage operations autonomously. This segmentation distributes the computational load across multiple processors while maintaining content-based storage benefits at each node, preventing any single processor from becoming a bottleneck.
Solution Approach 2:
The patent introduces a hierarchical addressing scheme that adds a new dimension to content-based storage by implementing multi-level indirection. Instead of direct content-to-location mapping, the system uses intermediate indexing structures that reduce the computational complexity of content verification and enable more efficient data retrieval operations.
2Productivity
If data is mapped based on content instead of address, then duplicate block writes are eliminated, but sequential location of data blocks is lost
Solution Approach 1:
The patent pre-computes and stores content-based hashes or fingerprints of data blocks when they are first written to the system. These pre-computed values are stored in indexing structures that enable rapid lookup, eliminating the need to compute content hashes during every read operation and maintaining efficient data retrieval despite content-based mapping.
3Adaptability or versatility
If content-based mapping is distributed across multiple processing elements, then system scalability is improved, but maintaining single content-addressing space becomes difficult
Solution Approach 1:
The patent merges the content-addressing spaces of multiple distributed nodes by implementing a unified hashing scheme that maps content to a global address space. Each node maintains a portion of the overall addressing space, and the system provides transparent routing between nodes, allowing the distributed system to present a unified content-addressing interface to users while maintaining scalability.
Data Source
AI summary
Example embodiments of the present invention relate to a method, a system, and a computer program product for tiering metadata. The method includes selecting a consecutive range of addresses of a logical device having a parent data structure associated therewith maintaining a first set of hash values at a first granularity of the logical device. A second hash value then may be calculated over the consecutive range of addresses of the logical device at a second granularity of the logical device and inserted into a child data structure associated with the parent data structure. Entries in the parent data structure at the first granularity for the consecutive range of addresses then may be freed in favor of the second hash value at the second granularity for the consecutive range of addresses inserted into the child data structure, for storing hash values for other addresses of the logical device.