Consensus Protocol Durability Modes for Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current consensus protocols in distributed computing systems, such as RAFT, do not provide flexible durability options for write operations to secondary storage, focusing primarily on synchronizing state machines across nodes without considering durability guarantees, which can impact system recovery and performance.
Innovation Solution
The implementation extends existing consensus protocols by introducing additional durability modes (immediate, replicated delayed, replicated relaxed, delayed, and relaxed) that allow nodes to manage log buffer writes and completion callbacks based on specified durability requirements, ensuring flexible durability options for distributed data storage applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If consensus protocols focus primarily on synchronizing state machines across nodes, then consistency is improved, but flexible durability options are lost
Solution Approach 1:
The patent applies dynamics by making the durability guarantee configurable through multiple modes (immediate, replicated delayed, replicated relaxed, delayed, relaxed) rather than fixed. The system dynamically adjusts the durability behavior based on the specified mode, allowing nodes to manage log buffer writes and completion callbacks flexibly while maintaining consistency guarantees appropriate to each mode.
Solution Approach 2:
The patent changes the parameter of durability guarantee from a single fixed state to multiple configurable states. By introducing different durability modes that modify how and when data is written to secondary storage and how completion callbacks are triggered, the system transforms the durability parameter into a variable that can be adjusted based on application needs while preserving consistency.
2Reliability
If immediate durability mode is used with quorum confirmation, then data persistence is improved, but system throughput is reduced
Solution Approach 1:
The patent applies partial action by offering different durability modes that perform different levels of persistence actions. Immediate durability mode performs full persistence to secondary storage with quorum confirmation for maximum reliability, while relaxed modes perform partial persistence (only to log buffer) for higher throughput. This allows users to select the appropriate level of action based on their specific needs.
Solution Approach 2:
The patent segments the durability guarantee into multiple levels or modes. Each mode represents a different segment of the durability spectrum, from immediate full persistence to delayed buffer-only persistence. This segmentation allows the system to handle different workloads with appropriate durability levels, improving overall throughput by not forcing all operations to use the most conservative durability mode.
3Productivity
If relaxed durability mode is used without secondary storage writes, then system performance is improved, but data recovery capability is reduced
Solution Approach 1:
The patent applies dynamics by making the durability behavior configurable through multiple modes. Relaxed durability mode dynamically adjusts the system to skip secondary storage writes for improved performance, while still providing a defined durability guarantee level. This allows the system to adapt its recovery capability based on the specified mode, balancing performance and reliability.
Solution Approach 2:
The patent changes the persistence parameter by introducing different durability modes that modify when and how data is written to secondary storage. Relaxed mode changes the parameter to defer or skip secondary storage writes, improving performance while maintaining a defined level of durability guarantee appropriate to the mode.
4Adaptability or versatility
If different durability modes are implemented, then flexible durability options are improved, but device complexity increases
Solution Approach 1:
The patent applies universality by designing the durability modes to work within the existing consensus protocol framework. The same basic mechanism of log buffering, secondary storage writing, and completion callbacks serves multiple durability modes, allowing the system to provide flexible durability options without requiring entirely separate protocols for each mode.
Solution Approach 2:
The patent manages complexity by changing parameters (durability mode specification) rather than adding fundamentally new mechanisms. The different durability modes are achieved by modifying when and how existing operations (log buffer writes, secondary storage writes, completion callbacks) are executed, rather than introducing entirely new protocol elements.
Data Source
AI summary
A cluster of nodes can receive a request to perform a logging operation of a distributed data storage application. The logging operation can include writing of data to a secondary storage of each of the nodes of the plurality of nodes. The request can include an indication of a durability mode to use for the logging operation. The logging operation can be initiated at each node of the plurality of nodes according to the indication of the durability mode, a completion callback can be run at each node of the plurality of nodes according to the indication of the durability mode, a global durability state and a global commit state can be determined for the cluster, and a reply can be returned to the request once the global durability state and global commit state are consistent with the indication of the durability mode to be used for the logging operation.