Flash CAS Encryption with Distributed Hash Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems, particularly primary storage solutions, face challenges in supporting content-based storage schemes due to intensive computational loads, block fragmentation, loss of sequential data location, and architectural difficulties in distributing tasks across multiple processing and storage elements, leading to performance degradation and scalability issues.
Innovation Solution
A method involving content addressable storage (CAS) with separate control and data planes, using cryptographic hashing for content-based mapping, and distributing computational tasks among multiple nodes to achieve scalable and efficient data storage and retrieval, while maintaining optimal load balancing and deduplication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If content-based storage mapping is implemented, then storage capacity efficiency is improved, but computational load increases significantly
Solution Approach 1:
The patent divides the storage system into multiple independent nodes, each maintaining its own content-based mapping. This segmentation distributes the computational load across nodes while preserving content-addressing capabilities locally, avoiding the need for centralized computation that would handle all mapping operations.
Solution Approach 2:
The patent introduces a hash function as an intermediary between data content and physical storage location. Instead of directly computing and storing full content-based mappings, the system uses hash functions to generate compact keys that enable efficient content-addressing with reduced computational overhead.
2Manufacturing precision
If large blocks are broken into smaller blocks for content addressing, then content addressing granularity is improved, but performance degrades due to block fragmentation
Solution Approach 1:
The patent segments storage into hierarchical levels: volumes are divided into extents, which are further divided into blocks. This multi-level segmentation allows content addressing at fine granularity while maintaining larger logical units for efficient management and reducing fragmentation overhead.
Solution Approach 2:
The patent introduces a new dimension of organization by mapping content hashes to physical locations through multiple layers (volume -> extent -> block). This dimensional approach allows the system to achieve fine content addressing granularity without the performance penalty of direct fine-grained fragmentation, as each layer manages a different aspect of the data hierarchy.
3Productivity
If content-based mapping is used, then duplicate block writes are eliminated, but sequential location of data blocks is lost
Solution Approach 1:
The patent performs preliminary actions by maintaining content-based mapping information in advance through address-to-hash tables and hash-to-location mappings. This pre-computed mapping structure enables the system to quickly resolve data locations during retrieval operations, eliminating the time loss that would otherwise occur when searching for sequentially located blocks.
Solution Approach 2:
The patent creates and maintains copy structures (address-to-hash tables, hash-to-location mappings) that parallel the actual data storage. These copied mapping structures enable fast lookup of data locations without requiring sequential search, thus preserving retrieval performance while eliminating duplicate writes through content-based deduplication.
4Adaptability or versatility
If content-based storage is implemented across multiple nodes, then scalability is improved, but maintaining single content-addressing space becomes difficult
Solution Approach 1:
The patent implements a universal content-addressing namespace that works across multiple nodes. Each node runs the same content-based storage logic and maintains consistent mapping structures, allowing the system to scale while presenting a unified content-addressing interface. This multi-functionality enables nodes to independently handle content addressing while contributing to the global namespace.
Solution Approach 2:
The patent uses feedback mechanisms where nodes report their content mapping state and hash-to-location mappings to a coordination mechanism. This feedback allows the system to maintain consistency across distributed nodes, ensuring that the single content-addressing space is preserved even as the system scales to multiple nodes with different data partitions.
Data Source
AI summary
In one aspect, a method includes assigning a base key to a storage array, assigning a volume auxiliary key to each volume in the storage array, for each volume in the storage array that is encrypted, encrypting every block of a backend using the base key and one of a full hash auxiliary key or a short hash handle auxiliary key and for each volume in the storage array that is encrypted, encrypting an address-to-hash table of a volume using the base key and the volume auxiliary key.