Short Hash Handle Management for Content Addressable Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage solutions, particularly primary storage systems, face challenges in supporting content-based storage with ultra-high performance due to intensive computational load, block fragmentation, inability to maintain sequential data location, and algorithmic difficulties in distributing tasks across processing and storage elements.
Innovation Solution
A method and apparatus for managing a short hash handle in a Content Addressable Storage (CAS) system, utilizing separate control and data planes, content-based mapping, and cryptographic hashing to optimize storage efficiency, scalability, and performance by distributing computational tasks and data flow across multiple nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If content-based storage mapping is implemented, then storage capacity efficiency is improved through deduplication, but computational load increases and performance degrades
Solution Approach 1:
The patent divides the storage system into separate control plane and data plane, with control plane handling content-based mapping and deduplication decisions, while data plane handles high-speed data operations. This segmentation allows computational tasks to be distributed and parallelized, resolving the contradiction between deduplication capability and performance.
Solution Approach 2:
The patent introduces hash signatures as intermediary objects that represent data blocks. Instead of directly comparing large data blocks for deduplication, the system computes and compares compact hash signatures, which dramatically reduces computational load while maintaining deduplication effectiveness.
2Measurement precision
If data blocks are broken into smaller sizes for content addressing, then fine-grained content addressing is achieved, but block fragmentation degrades performance
Solution Approach 1:
The patent applies different qualities to different parts of the storage system: fine-grained content addressing is applied locally at the data block level for deduplication precision, while sequential access patterns are maintained at the larger data plane level for performance. This local differentiation resolves the contradiction between granularity and performance.
3Speed
If address-based mapping is used, then sequential data location is maintained for fast access, but storage capacity efficiency decreases due to inability to deduplicate
Solution Approach 1:
The patent adds a content-based dimension to the traditional address-based mapping by introducing hash signatures as a second indexing mechanism. Data blocks can be accessed via original addresses for sequential performance or via hash signatures for deduplication, creating a multi-dimensional mapping system that resolves the contradiction.
4Stability of the object's composition
If computational tasks are centralized for content-based mapping, then mapping consistency is maintained, but task distribution across multiple processing elements becomes difficult
Solution Approach 1:
The patent merges the functions of content-based mapping, deduplication decision-making, and hash signature management into a unified control plane. This consolidation maintains mapping consistency while allowing the control plane to distribute tasks across multiple processing elements through standardized interfaces, resolving the contradiction between consistency and distribution.
Data Source
AI summary
Example embodiments of the present invention relate and a method and an apparatus for managing a short hash handle. The method including receiving an I/O including a first identifier for a data block and examining the first identifier in comparison with a second identifier. The data block identified in the I/O then may be managed according to the first identifier and the second identifier.