Overwriting Decided Slots in Sequencer Consensus Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional sequencer protocols fail to provide reliable in-order messaging and flow control for replicated services, leading to message loss, duplication, and reordering in consensus logs due to immutable decided slots and leadership changes.
Innovation Solution
The proposed solution enhances existing consensus log protocols like Paxos, Raft, and Kafka by allowing modification of decided slots during epoch changes, enabling a new leader to re-propose lost messages in their original order without gaps or reordering, and maintaining message order as intended by clients.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If decided slots are made immutable to ensure consistency, then reliability is improved, but flexibility to handle leadership changes and message recovery is worsened
Solution Approach 1:
The patent applies dynamics by transitioning from static immutability to dynamic mutability of decided slots. The system allows decided slots to be overwritten during epoch changes, enabling the consensus log to adapt to leadership changes while maintaining consistency through controlled reconfiguration mechanisms.
Solution Approach 2:
The patent changes the parameter of slot immutability from fixed to variable. By allowing decided slots to be modified during epoch transitions, the system achieves flexibility in handling leadership changes while maintaining reliability through proper consensus protocols that ensure consistent state transitions.
2Reliability
If conventional sequencer protocols are used to ensure fault tolerance, then reliability is improved, but in-order messaging and flow control are worsened
Solution Approach 1:
The patent segments the consensus log into epochs, where each epoch manages a specific range of slots. This segmentation allows the system to maintain fault tolerance at the epoch level while ensuring in-order messaging within each epoch, resolving the contradiction between reliability and operational simplicity.
Solution Approach 2:
The patent introduces epoch transitions as an intermediary mechanism between fault tolerance requirements and in-order messaging guarantees. The epoch structure acts as a mediator that enables both fault tolerance through consensus and in-order messaging through controlled slot management during epoch changes.
3Stability of the object's composition
If decided slots cannot be modified, then consistency is maintained, but message loss and duplication occur during leadership changes
Solution Approach 1:
The patent applies preliminary action by preparing for potential message loss before it occurs. During epoch changes, the system proactively manages slot overwrites to prevent message loss and duplication, ensuring that messages are properly transitioned between epochs rather than being lost or duplicated during leadership changes.
Solution Approach 2:
The patent converts the potential harm of mutable decided slots into a benefit by using controlled slot overwrites during epoch changes. This mechanism, which could potentially cause inconsistency, is instead used to prevent message loss and handle leadership transitions, turning a possible harm into a reliability enhancement.
Data Source
AI summary
Techniques are described for safely overwriting decided slots and in-order fault tolerant consensus logs for replicated services. Using techniques described herein, a broad class of already-existing consensus log protocols may be enhanced/extended to safely overwrite decided slots and provide in-order fault tolerant consensus logs. When changing to a different epoch of a consensus log, slots determined to be unreachable may be changed/deleted even if slots after the gap were decided. A sequencer protocol establishes distributed consensus among a group of services. The sequencer protocol provides in-order execution of messages from multiple clients, and flow control from within the sequencer protocol, without offloading de-duplicate and reorder (DDRO) logic to the application layer. Fault tolerance is provided by egress cursors and ingress cursors, which provide awareness of which specific messages from each client sender should be executed next, even if those messages are not presently in the consensus log.