Primary Bias State Handling for Distributed Storage Race Conditions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-site distributed storage systems face disruptions due to temporary network failures and mediator unavailability, leading to connectivity issues and potential split-brain situations where multiple sites attempt to serve I/O operations simultaneously, causing data consistency problems.
Innovation Solution
Implement a primary bias feature that allows storage clusters to negotiate and agree on a primary I/O serving role without external mediation, storing the primary bias state locally, ensuring non-disruptive operations even in the absence of a mediator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a mediator is used to manage primary bias state in distributed storage system, then data consistency can be maintained, but system availability deteriorates when mediator becomes unavailable
Solution Approach 1:
The patent uses an atomic lock mechanism as an intermediary to coordinate between storage clusters when the mediator is unavailable. The atomic lock serves as a temporary intermediary that allows clusters to negotiate primary bias state without requiring continuous mediator intervention, thus maintaining availability while preserving consistency through structured negotiation protocols.
Solution Approach 2:
Storage clusters are empowered to autonomously negotiate and determine primary bias state among themselves when the mediator is unavailable. Each cluster can initiate negotiations, propose states, and reach consensus without external mediation, enabling self-service operation that maintains system availability while preserving data consistency through mutual agreement protocols.
2Productivity
If storage clusters can autonomously negotiate primary bias state without mediator, then system availability improves during mediator failure, but complexity of handling race conditions increases
Solution Approach 1:
The patent replaces complex mechanical coordination mechanisms with software-based atomic lock operations and generation number comparisons. Instead of using intricate protocols to detect and resolve race conditions, the system uses simple atomic increment operations and generation number checks that can be efficiently implemented in software, reducing the complexity of race condition handling while maintaining high availability.
Solution Approach 2:
The system performs preliminary actions by establishing atomic lock acquisition and generation number assignment before actual primary bias state negotiation. These preliminary steps create a structured framework that simplifies subsequent race condition resolution, as the atomic locks and generation numbers are already in place to prevent and detect concurrent conflicts before they can cause data inconsistency.
3Reliability
If atomic locks are held during primary bias state negotiation, then race conditions are prevented, but processing speed deteriorates due to lock release delays
Solution Approach 1:
The patent applies partial locking by holding atomic locks only for the critical portion of primary bias state negotiation where race conditions can occur, rather than maintaining locks throughout the entire negotiation process. Generation numbers are used to track and resolve only the specific moments when concurrent access could cause problems, allowing processing to proceed at full speed during non-critical phases while maintaining reliability during critical operations.
Data Source
AI summary
According to an example, a computer-implemented method comprises initiating a first process for atomically setting the primary bias state with a first node of a primary storage cluster of a multi-site distributed storage system due to a temporary loss of connectivity to a mediator or a temporary mediator failure, releasing an atomic lock for the first process on the first node of the primary storage cluster, sending the first process and an associated first generation indicator to a first node of a secondary storage cluster of the multi-site distributed storage system to handle the first process for setting the primary bias state, and initiating a second process for atomically clearing a primary bias state with the first node or any node of the primary storage cluster based on detecting a connection to the mediator or detecting that the mediator is available.