Distributed Hash Table Storage System for Data Reliability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional block storage systems face limitations in reliability, availability, extensibility, and cost-effectiveness due to reliance on RAID and controller-based architectures, which can lead to data loss, service interruptions, and high hardware costs.
Innovation Solution
A data processing method and apparatus that converts logical block addressing-based requests into key addressing-based requests, routing these through a library to a master storage node and backup nodes, utilizing distributed hash table technology for redundancy and scalability, thereby improving reliability, availability, and cost-effectiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If RAID and controller-based architecture is used, then data reliability is improved, but system complexity and cost increase
Solution Approach 1:
The patent segments the storage system into multiple independent storage nodes forming a distributed cluster, eliminating the need for complex centralized RAID controllers. Each node operates independently with its own storage capacity, and data is distributed across nodes through hash-based addressing, reducing overall system complexity while maintaining reliability through redundancy.
Solution Approach 2:
The patent replaces the mechanical/controller-based RAID system with a software-based distributed hash table (DHT) architecture. Instead of relying on hardware controllers to manage redundancy and failover, the system uses cryptographic hash functions to deterministically route data to appropriate nodes, substituting mechanical control with algorithmic distribution.
2Quantity of substance
If IP SAN or FC SAN is adopted, then storage capacity is improved, but processing capability becomes a bottleneck
Solution Approach 1:
The patent divides the storage system into multiple independent nodes that can be added or removed without affecting the entire system. Each node processes requests independently, distributing the processing load across the cluster. This segmentation allows storage capacity to scale independently from processing capability, as new nodes contribute both capacity and processing power.
Solution Approach 2:
The distributed hash table architecture enables storage nodes to autonomously route requests to the correct node based on hash calculations, eliminating the need for centralized controllers to manage data placement and retrieval. This self-service mechanism improves processing capability by distributing control and reducing bottlenecks.
3Reliability
If manual disk replacement and RAID rebuilding is performed, then data redundancy is maintained, but maintenance time and operational complexity increase
Solution Approach 1:
The patent implements automatic fault detection and data recovery through the distributed hash table architecture. When a node fails, the system automatically detects the failure through missing hash responses and initiates data replication to alternative nodes without human intervention. This self-service recovery mechanism maintains data redundancy while eliminating manual maintenance time.
Solution Approach 2:
The system pre-determines data placement locations through hash calculations before failures occur. Each data block is associated with specific nodes based on hash values, so when failures happen, the system already knows which nodes should hold replica copies, enabling immediate recovery without complex rebuilding processes.
4Productivity
If FC SAN hardware is used, then storage performance is improved, but system cost increases
Solution Approach 1:
The patent creates a storage system where standard off-the-shelf hardware can be used for all nodes, eliminating the need for specialized FC SAN hardware. The distributed architecture provides universal functionality across different hardware platforms, allowing the system to achieve high performance through software optimization rather than expensive specialized components.
Solution Approach 2:
The system uses inexpensive standard storage nodes instead of expensive FC SAN hardware. Individual nodes can be added, removed, or replaced without affecting the entire system, allowing cost-effective scaling. The distributed architecture ensures that cheap individual components can be combined to create a high-performance system that rivals or exceeds expensive centralized solutions.
Data Source
AI summary
The present disclosure provides a storage system for data read and write. One embodiment of the storage system includes an origination device that is configured to receive a request for a logical block addressing-based operation on a volume, convert the logical block addressing-based operation request into a key addressing-based operation request carrying a key corresponding to data to be operated, and send the key addressing-based operation request to a routing library; the routing library is configured to receive the key addressing-based operation request, hash the key corresponding to the data to be operated, determine that a storage node taking charge of a hash region in which the hashed key is located is the master storage node, send the key addressing-based operation request to the master storage node of the data to be operated.


