Cloud Storage Data Placement via Distributed Hash Table
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Decentralized data protection systems face challenges in maintaining data availability and flexibility in Internet-scale environments due to high churn rates and the inability to efficiently direct fine-grained data placement in cloud storage nodes without centralized servers, leading to issues like racing conditions and unpredictable behavior.
Innovation Solution
A system that directs the placement of data replicas in cloud storage nodes by identifying a redundancy number and using a consistent hash function to select appropriate nodes, ensuring predictable behavior and maintaining data availability even with high churn rates, through a decentralized policy-driven approach with a distributed hash table and Inter-Planetary File System protocol.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a decentralized data protection system is used without centralized servers, then system flexibility and data mobility are improved, but data placement becomes unpredictable and racing conditions occur
Solution Approach 1:
The patent introduces a distributed hash table as an intermediary structure that mediates between decentralized storage nodes and data placement decisions. Each node maintains a local DHT that maps data identifiers to node identifiers, enabling predictable data placement without centralized coordination. This intermediary layer resolves the contradiction by providing structured guidance in a decentralized environment.
Solution Approach 2:
The system changes the parameter of node identification from arbitrary decentralized identifiers to hashes of node attributes (IP address, port, etc.). This parameter transformation enables consistent and predictable data placement by ensuring that the same data always maps to the same node based on reproducible hash values, while maintaining decentralized operation.
2Reliability
If data is replicated across multiple cloud storage nodes, then data availability is improved, but system complexity increases due to placement management
Solution Approach 1:
Each storage node autonomously determines its own role in data storage by computing hashes of its attributes and comparing them against the distributed hash table. Nodes self-organize into the replication structure without external coordination, reducing placement management complexity while maintaining multiple replicas for high availability.
Solution Approach 2:
The patent segments the data placement management into independent, identical node behaviors. Each node follows the same algorithm to determine which data it should store, based on its own attribute hash. This segmentation of responsibilities eliminates complex centralized placement logic while ensuring consistent replication across nodes.
3Adaptability or versatility
If fine-grained data placement is implemented in decentralized systems, then data mobility is improved, but racing conditions and unpredictable behavior occur
Solution Approach 1:
The patent replaces mechanical coordination mechanisms (centralized servers, explicit communication protocols for placement decisions) with a mathematical hash-based system. Data placement is determined by deterministic hash functions rather than dynamic coordination, eliminating racing conditions while enabling fine-grained control over which nodes store which data.
Data Source
AI summary
A system receives a request to store a replica of a data object into any cloud storage node associated with an attribute, and then identifies a redundancy number associated with the data object. The system identifies a number of cloud storage nodes that are associated with the attribute. If the identified number of cloud storage nodes is greater than the redundancy number, the system identifies the redundancy number of cloud storage nodes as a subset of the number of cloud storage nodes, based on the data object and a unique identifier associated with each of the number of cloud storage nodes. The system stores the redundancy number of replicas into the corresponding redundancy number of cloud storage nodes.