Cluster Data Redundancy with Distributed Placement and Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing shared-nothing distributed storage systems face challenges in maintaining data integrity and redundancy during migration, recovery from node or storage failures, and policy changes, often resulting in increased network latency, coarse granularity of data management, and centralization issues that lead to complications and prolonged recovery times.
Innovation Solution
A distributed storage system employing a pseudo-random consistent hash (CRUSH) algorithm for data placement, aggregating data into consistency groups to reduce metadata processing loads, and using a distributed consensus algorithm like Paxos to ensure synchronized node operations and resilient data management across independent nodes, allowing for efficient recovery and redundancy adjustments without centralized coordination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If write marking is employed on a common/shared scoreboard to track data object state during migration, then data integrity is maintained, but network latency increases due to multiple steps required (logging request, sending message to each target, waiting for write and response)
Solution Approach 1:
The patent segments the centralized scoreboard approach into distributed per-node tracking. Each node independently tracks dirty chunks in its local bitmap rather than using a centralized scoreboard, eliminating network round trips for write tracking and reducing latency while maintaining data integrity through local state management.
Solution Approach 2:
Each node autonomously manages its own dirty chunk tracking without requiring coordination with other nodes or a centralized manager. The local bitmap allows each node to self-service the tracking of modified chunks, eliminating the need for network communication for write marking and reducing overall system latency.
2Ease of manufacture
If marking at the file system level is used to indicate data modification, then implementation is simple, but the granularity is too coarse for very large data files resulting in recoveries that require too long a period to complete
Solution Approach 1:
The patent divides files into smaller chunks and maintains a bitmap for each chunk rather than marking the entire file at the file system level. This fine-grained segmentation allows recovery operations to process only the modified chunks rather than entire large files, significantly reducing recovery time while maintaining implementation simplicity through the use of bitmaps.
3Ease of operation
If a central manager or coordinator is used to perform recovery operations in volume managers and multi-copy file systems, then recovery coordination is centralized, but the system stalls when the coordinator undergoes a failure leading to further complications
Solution Approach 1:
The patent extracts the recovery coordination function from a centralized manager and distributes it to individual nodes. Each node independently performs recovery operations for its local chunks using its own bitmap state, eliminating the single point of failure represented by the central coordinator and ensuring recovery availability even when individual nodes fail.
Solution Approach 2:
Each node autonomously performs recovery operations for its own dirty chunks without requiring a central coordinator. The local bitmap enables each node to self-service the recovery process, eliminating dependency on a central manager and ensuring that recovery can proceed even if other nodes or potential coordinators fail.
4Reliability
If data is replicated across multiple nodes in a shared-nothing architecture, then fault tolerance is improved, but metadata processing loads increase and coordination becomes more complex
Solution Approach 1:
The patent segments metadata management into distributed per-node bitmaps rather than centralized metadata structures. Each node maintains its own bitmap for tracking dirty chunks locally, eliminating the need for complex centralized metadata processing and coordination while maintaining fault tolerance through data replication across nodes.
Data Source
AI summary
A cluster system includes a plurality of computing nodes connected to a network, each node including one or more storage devices. The cluster system stores data and at least one of data replicas or erasure-coded segments across the plurality of nodes based on a redundancy policy. Further, configuration information, which may be indicative of a data placement of the data and the data replicas or erasure-coded segments on the plurality of nodes, is provided to each of the plurality of nodes. Additionally, each of the nodes may act as a first node which is configured to determine, upon a change of the redundancy policy, updated configuration information based on the change of the redundancy policy and to send a message including information indicating the change of the redundancy policy to the other nodes of the plurality of nodes.