Storage Cluster Configuration Modification for Fault Tolerance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The Raft consensus replication protocol fails to ensure data replication service continuity when half of the storage nodes become faulty, as it cannot satisfy the requirement of most nodes responding to a configuration change command, leading to a failure in cluster configuration change and service disruption.
Innovation Solution
A storage cluster configuration change method that includes an arbitration module and a configuration library, which sends a forced cluster configuration change instruction to fault-free nodes if half of the storage nodes are faulty and at least one node has a latest log index number greater than or equal to a committed log index, ensuring service continuity by re-electing a new leader and updating cluster configuration information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the Raft protocol requires most storage nodes to respond to a configuration change command, then data consistency is ensured, but when half of the storage nodes become faulty, the configuration change cannot be completed and service continuity is lost
Solution Approach 1:
The patent applies preliminary action by pre-dividing storage nodes into multiple groups before faults occur. When half the nodes fail, the system can immediately select a complete group from the remaining nodes to form a new quorum, avoiding the need to wait for responses from all original nodes. This pre-organization of node groups enables configuration changes to proceed even when the traditional 'most nodes' requirement cannot be met.
Solution Approach 2:
The patent segments the storage cluster into multiple groups of storage nodes. Instead of treating the cluster as a single entity requiring majority responses, it divides nodes into segments (groups) where each group can independently form a valid configuration. This segmentation allows the system to maintain service continuity by selecting a complete group from surviving nodes, even when half the total nodes are faulty.
2Reliability
If the storage cluster waits for responses from most storage nodes before allowing configuration change, then safety is maintained, but service availability drops to zero when half nodes are faulty
Solution Approach 1:
The system performs preliminary organization of storage nodes into groups before faults occur. When half the nodes fail, the arbitration module can immediately identify a valid group from remaining nodes and proceed with configuration changes without waiting for responses from all original nodes. This preliminary structuring maintains safety while enabling service continuity.
Solution Approach 2:
Instead of requiring responses from most original nodes (traditional approach), the patent inverts the logic by requiring responses only from nodes within a selected group. This inversion changes the quorum requirement from 'majority of total nodes' to 'all nodes in a selected group,' enabling configuration changes to proceed with fewer total responses while maintaining safety within the group context.
3Reliability
If the storage cluster uses traditional Raft protocol for configuration change, then consensus is achieved under normal conditions, but the cluster cannot adapt when half nodes become faulty
Solution Approach 1:
The patent introduces dynamics by making the node group configuration flexible and adaptable. Instead of a fixed quorum requirement based on total node count, the system dynamically selects groups from surviving nodes. The arbitration module can adaptively choose which group to use based on current node availability, allowing the cluster to maintain consensus under varying fault conditions.
Solution Approach 2:
The patent changes the parameter for determining quorum from 'number of responding nodes' to 'completeness of selected group.' Instead of requiring a fixed number of responses (e.g., majority of 5 nodes), the system requires that all nodes in a selected group respond. This parameter change allows the quorum size to adapt to different fault scenarios while maintaining consensus safety.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This application provides astorage cluster configurationmodification method, an apparatus, a computer system, and the like. In a storage cluster that uses a consensus replication protocol, when half of storage nodes become faulty, if it is determined that at least one storage node exists in the storage cluster after the fault, and a latest log index number of the storage node is greater than or equal to an index number of a committed log provided by the storage cluster for a client, an arbitration module sends a forced cluster configuration change instruction to a fault-free storage node. The forced cluster configuration change instruction is used to instruct the fault-free storage node to modify local cluster configuration information. In addition, the arbitration module updates cluster configuration information stored in a configuration library, so that updated cluster configuration information indicates a new cluster configuration. In this way, a problem of unavailability of the storage cluster after half of the storage nodes become faulty is resolved, and availability of the storage cluster is improved.