Multi-Staged In-Memory Checkpoint Replication with Relaxed Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional cluster-based recovery mechanisms face challenges in minimizing recovery time and achieving consistent recovery points due to high resource costs and latency associated with disk-based application state storage, as well as bottlenecks in messaging traffic and processing requirements for atomic replication protocols.
Innovation Solution
A multi-staged in-memory checkpoint replication system with relaxed consistency, utilizing a replication manager to configure primary and secondary groups of nodes for efficient replication, allowing asynchronous secondary replication and flexible consistency semantics to reduce overhead and improve performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If disk-based application state storage is used for recovery, then data persistence is achieved, but recovery time and application overhead increase significantly
Solution Approach 1:
The patent segments the replication process into multiple stages: first replicating to primary group nodes within a single atomic transaction, then asynchronously replicating to secondary group nodes. This segmentation allows the system to achieve data persistence through multi-node replication while reducing recovery time by allowing secondary replication to occur asynchronously after the primary transaction commits.
Solution Approach 2:
The patent implements dynamic consistency semantics where primary group nodes require strong consistency (atomic transaction), but secondary group nodes can accept relaxed consistency (asynchronous replication). This dynamic approach to consistency levels allows the system to optimize for both data persistence and recovery time by applying appropriate consistency requirements at different replication stages.
2Reliability
If atomic replication protocols are used to ensure consistency, then data consistency is maintained, but messaging traffic and processing overhead increase
Solution Approach 1:
The patent applies different consistency quality levels to different groups of nodes: strong atomic consistency for the primary group and relaxed asynchronous consistency for the secondary group. This local quality differentiation reduces messaging traffic and processing overhead for secondary nodes while maintaining data consistency where it matters most (primary replication).
Solution Approach 2:
The patent implements partial atomicity by requiring atomic transactions only for primary group replication, while allowing non-atomic asynchronous replication for secondary groups. This partial application of strict consistency protocols reduces the messaging overhead and processing complexity while still ensuring data consistency for the critical primary replication path.
Data Source
AI summary
A system for multi-staged in-memory checkpoint replication with relaxed consistency includes a replication manager and a cluster with a plurality of nodes. In accordance with a configurable replication policy, the replication manager may be configured to identify a primary group of nodes to store primary replicas of a specified source memory region of a source node, and to identify a node of the primary group as an initiator of replication to a secondary group of nodes. In response to a replication request, the replication manager may be configured to copy contents of one or more data blocks of the source memory region to a each node of the primary group within a single atomic transaction, and initiate an operation to copy the contents of the data blocks from the initiator to an additional node of the secondary group.