Single Master Failover Protocol for Data Replication

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing data storage systems face challenges in providing seamless scaling, high durability, and availability, especially in multi-tier e-commerce systems where resources need to be efficiently allocated and managed across multiple machines and locations.

Innovation Solution

The system implements a Web-based data storage service that supports automatic live repartitioning of data in response to anomalies and explicit requests, ensuring seamless scaling and high availability through replication, flexible schema, multiple consistency models, and efficient resource allocation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If data is distributed and replicated across multiple machines for security and availability, then reliability improves, but device complexity increases

Engineering Contradiction:
Improvedata availabilityVSAvoidsystem configuration
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The system segments data into multiple partitions that can be independently replicated across different machines. Each partition can have its own replica set, allowing granular control over data distribution and failure isolation. This segmentation enables the system to achieve high availability through replication while managing complexity by treating each partition as an independent unit.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces a coordinator component that acts as an intermediary to manage the complexity of distributed data replication. The coordinator handles partition assignment, replica management, and failover coordination, centralizing control logic and shielding individual nodes from the overall system complexity. This intermediary approach allows the system to maintain high reliability while reducing the operational burden on individual components.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Productivity

If read operations are distributed across multiple machines, then productivity improves, but measurement precision deteriorates

Engineering Contradiction:
Improveread throughputVSAvoidread consistency
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The system dynamically adjusts read routing based on consistency requirements. For strong consistency reads, the system routes requests to the master node or coordinates with replicas to ensure data freshness. For eventual consistency reads, the system can distribute reads across multiple replicas to improve throughput. This dynamic approach allows the system to optimize between productivity and measurement precision based on specific read operation requirements.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent implements different consistency parameters for different read operations. The system allows clients to specify consistency levels (e.g., strong consistency, eventual consistency) for each read operation, and adjusts the read routing and replica selection based on these parameters. This enables the system to achieve high throughput for non-critical reads while maintaining data precision for critical operations that require strong consistency.

Inventive Principle:
Principle #35Parameter changes

3Measurement precision

If write operations are centralized on a single master, then measurement precision improves, but productivity deteriorates

Engineering Contradiction:
Improvewrite consistencyVSAvoidwrite throughput
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The system segments write operations by partitioning the data space into multiple partitions, each with its own master node. This allows write operations to be distributed across multiple partition masters simultaneously, improving overall write throughput while maintaining write consistency within each partition. The segmentation approach enables parallel write processing without sacrificing the precision of write operations within individual partitions.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces a partition dimension to the write operation model. Instead of having a single centralized master for all writes, the system creates multiple partition masters across different data dimensions (partitions). Write operations are routed to the appropriate partition master based on the data key, enabling parallel write processing across multiple nodes while maintaining consistency within each partition. This dimensional approach transforms the write scalability problem from a single-point bottleneck to a distributed parallel processing model.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS12316489B2System and method for data replication using a single master failover protocol
Publication Date: 2025.05.27 AMAZON TECH INC
  • US12316489B2 patent drawing
  • US12316489B2 patent drawing
  • US12316489B2 patent drawing

AI summary

A system that implements a data storage service may store data on behalf of storage service clients. The system may maintain data in multiple replicas of various partitions that are stored on respective computing nodes in the system. The system may employ a single master failover protocol, usable when a replica attempts to become the master replica for a replica group of which it is a member. Attempting to become the master replica may include acquiring a lock associated with the replica group, and gathering state information from the other replicas in the group. The state information may indicate whether another replica supports the attempt (in which case it is included in a failover quorum) or stores more recent data or metadata than the replica attempting to become the master (in which case synchronization may be required). If the failover quorum includes enough replicas, the replica may become the master.