Failure Domain Harmonization in Distributed Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed computing systems face challenges in maintaining reliability and efficiency during node failures and system upgrades due to the risk of unexpected failure conditions and prolonged upgrade times when scaling out work across multiple devices, leading to potential data unavailability and integrity issues.
Innovation Solution
Implementing failure domains in a distributed computing system by assigning computing nodes to specific domains, generating hashing information, and using a group update component to manage node operations, allowing for parallel updates and redundancy to ensure data availability even with node failures, thereby reducing the impact of upgrades and improving system adaptability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If nodes are upgraded sequentially in a distributed computing system, then system stability is maintained, but upgrade time and downtime increase
Solution Approach 1:
The system divides nodes into separate failure domains, allowing independent upgrading of each domain. This segmentation enables parallel upgrades across different domains while maintaining stability within each domain, resolving the contradiction between system stability and upgrade time.
Solution Approach 2:
The system performs preliminary actions by pre-assigning nodes to failure domains and pre-configuring failover mechanisms before upgrades begin. This allows the system to prepare for parallel upgrades in advance, reducing actual upgrade time while maintaining stability through pre-established redundancy.
2Productivity
If more nodes are added to increase computing efficiency, then processing capacity improves, but complexity of managing node failures and consistency increases
Solution Approach 1:
By segmenting nodes into failure domains, the system reduces the scope of failure management. Each domain can be managed independently, simplifying the complexity that would otherwise arise from managing all nodes uniformly. This allows more nodes to be added while keeping failure management tractable.
Solution Approach 2:
The system changes the parameter of node organization from flat individual management to hierarchical domain-based management. This parameter change transforms the complexity from O(n) individual node management to O(n/d) domain management where d is the number of domains, reducing overall complexity while maintaining scalability.
3Reliability
If hashing information is updated across all nodes simultaneously, then data consistency is maintained, but system downtime increases
Solution Approach 1:
The system segments hashing information updates by failure domain, allowing different domains to update their hashing information at different times. This maintains data consistency within each domain while enabling parallel updates across domains, reducing overall system downtime.
Solution Approach 2:
The system performs preliminary hashing information generation for each failure domain before updates are needed. This pre-computation allows domains to be updated independently and in parallel, maintaining consistency without requiring simultaneous system-wide updates, thereby reducing downtime.
4Adaptability or versatility
If nodes are removed or added frequently to scale the system, then adaptability improves, but unexpected failure conditions increase
Solution Approach 1:
By organizing nodes into failure domains, the system creates predictable failure boundaries. When nodes are added or removed, the impact is contained within specific domains rather than affecting the entire system unpredictably. This segmentation makes failure conditions more predictable while maintaining scalability.
Solution Approach 2:
The system changes the organizational parameter from individual node management to domain-based management. This allows frequent additions and removals of nodes within domains without propagating unexpected failures system-wide, improving both adaptability and failure predictability simultaneously.
Data Source
AI summary
Harmonization of failure domains in a distributed system is provided herein. A data storage system as described herein can include a memory that stores computer executable components and a processor that executes computer executable components stored in the memory. The computer executable components can include a node assignment component that assigns computing nodes in the data storage system to respective failure domains of a set of data failure domains, a hashing component that generates hashing information associated with respective computing nodes of the computing nodes assigned to a selected failure domain of the set of failure domains, and a group update component that designates at least one computing node of the computing nodes in the selected data failure domain to perform an operation associated with the hashing information.