Cluster Data Migration With Distributed Placement and Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing shared-nothing distributed storage systems face challenges in maintaining data integrity and redundancy during migration, recovery from node or storage failures, and policy changes, often resulting in increased network latency, coarse granularity of data management, and centralization issues that lead to complications and prolonged recovery times.
Innovation Solution
A distributed storage system employing a pseudo-random consistent hash (CRUSH) algorithm for data placement, aggregating data into consistency groups to reduce metadata processing loads, and using a distributed consensus algorithm like Paxos to ensure synchronized node operations and resilient data management across independent nodes, allowing for efficient recovery and redundancy adjustments without central coordination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If write marking is used on a common/shared scoreboard to track data modifications, then data integrity is maintained, but network latency increases due to multiple communication steps
Solution Approach 1:
The patent extracts the centralized scoreboard mechanism from the system architecture and replaces it with distributed tracking on each node. Each node independently maintains its own dirty bit map and data modification state, eliminating the need for centralized coordination and reducing network communication overhead while maintaining data integrity.
Solution Approach 2:
Each node in the distributed storage system autonomously tracks its own data modification state and manages its own recovery processes without requiring centralized coordination. Nodes independently identify dirty data, initiate recovery operations, and coordinate with other nodes through peer-to-peer communication, reducing network latency and improving system responsiveness.
2Ease of operation
If entire high level data storage areas are marked as dirty, then data modification tracking is simplified, but recovery time increases due to the large amount of data requiring recovery
Solution Approach 1:
The patent segments the data storage space into smaller units (data areas or regions) and maintains separate dirty bit maps for each segment. This allows the system to track and recover only the specific segments that contain modified data rather than recovering entire large storage areas, significantly reducing recovery time while maintaining operational simplicity.
3Ease of manufacture
If file system level marking is used to indicate data modification, then implementation is straightforward, but granularity is too coarse for very large data files resulting in prolonged recovery periods
Solution Approach 1:
The patent divides large data files into smaller data areas or regions and maintains individual dirty bit map entries for each region. This fine-grained segmentation allows the system to identify and recover only the specific regions that contain modified data within large files, rather than recovering entire files, thus reducing recovery time while keeping the implementation manageable through systematic organization.
4Ease of operation
If centralized database marking is used for data migration tracking, then coordination is simplified, but system scalability is limited and recovery stalls when the central coordinator fails
Solution Approach 1:
The patent extracts the centralized coordination function from the system and distributes it to individual nodes. Each node independently tracks its own data migration state and maintains local dirty bit maps, eliminating the single point of failure represented by the central coordinator. This distributed approach improves system reliability and recovery resilience while maintaining operational simplicity through standardized node behaviors.
Solution Approach 2:
The patent segments the centralized coordination functionality into independent node-level operations. Each node autonomously manages its own data tracking, migration, and recovery processes, with the ability to independently determine when recovery is needed based on its local dirty bit map state. This segmentation improves both reliability and scalability.
5Reliability
If large amounts of metadata are maintained in shared storage to account for coordinator failure, then recovery resilience is improved, but storage complexity and overhead increase
Solution Approach 1:
The patent extracts the metadata management function from centralized shared storage and distributes it to individual nodes. Each node maintains its own local dirty bit map and data state information, eliminating the need for large amounts of metadata in shared storage. This approach reduces storage complexity and overhead while maintaining recovery resilience through distributed state tracking.
Data Source
AI summary
A cluster system includes a plurality of computing nodes connected to a network. Each node is configured to access its own storage device, and to send and receive input/output (I/O) operations associated with its own storage device. Further, each node of the plurality of nodes may be configured to have a function of acting as a first node, which sends a first message to other nodes of the plurality of nodes. The first message may include configuration information indicative of a data placement of data on the plurality of nodes in the cluster system according to an event. Following receipt of the first message from the first node, each of the other nodes may be configured to determine, based at least in part on the configuration information, whether data stored on its own storage device is affected by the event.