External Data Store Event Ordering for Distributed Replication

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Large-scale computing systems face challenges in accurately replicating data store events across distributed systems, particularly in maintaining the logical order of events and handling out-of-order replication, which demands significant developer time and computing resources, and increases storage and network resource usage.

Innovation Solution

The implementation of an external data store system that uses sequence identifiers and tombstone data stores to maintain the logical order of events, allowing for optimistic and eventually consistent replication by assigning continuous sequence numbers to events, even after deletion and recreation, and using these identifiers to determine the correct order for replication at the destination data store.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If events are replicated in strict sequence order, then replication accuracy is improved, but network resource usage and processing time increase due to waiting for out-of-order events

Engineering Contradiction:

Improvereplication accuracyVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system pre-assigns sequence identifiers to events at the source data store before replication occurs. This preliminary numbering allows the destination data store to process events out of network delivery order while maintaining logical sequence integrity, eliminating the need to wait for out-of-order events.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Sequence identifiers act as an intermediary mechanism between source and destination data stores. These identifiers enable the destination to reorder and process events correctly without requiring strict network delivery order, thus improving both replication accuracy and processing efficiency.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If events are replicated out of order, then processing time and network usage are reduced, but replication accuracy deteriorates due to incorrect event sequencing

Engineering Contradiction:

Improveprocessing speedVSAvoidreplication accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system replaces the mechanical constraint of strict sequential processing with a logical ordering system based on sequence identifiers. Events can be transmitted and initially processed in any network delivery order, but the sequence identifiers enable correct logical ordering at the destination without retransmission or waiting.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system changes the parameter of event identification from relying on network delivery order to using pre-assigned sequence identifiers. This parameter change allows events to be processed in network delivery order while maintaining logical sequence accuracy through the identifier-based ordering mechanism.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If sequence identifiers are maintained in external tombstone data stores, then event ordering accuracy is improved, but storage resource usage increases

Engineering Contradiction:

Improveevent ordering accuracyVSAvoidstorage resources

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The system extracts sequence identifier maintenance from the main data store operations and places it in external tombstone data stores. This separation allows the primary data stores to focus on data operations while external stores handle sequencing metadata, improving event ordering accuracy without significantly impacting main storage resources.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Tombstone data stores use inexpensive storage mechanisms for maintaining sequence identifiers. These external stores use minimal resources compared to primary data stores, providing accurate event ordering information without significant storage resource consumption.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Data Source

PatentUS11514077B2Replication event ordering using an external data store

Publication Date: 2022.11.29 AMAZON TECH INC

AI summary

Methods, systems, and computer-readable media for replication event ordering using an external data store are disclosed. Events are received for a key in a source data store, including a deletion event and modification events. In an external data store, a record of the deletion event is stored that includes a sequence identifier of the deletion event in a sequence. Sequence identifiers are assigned to the modification events. Some of the sequence identifiers are earlier than the deletion event in the sequence, and others of the sequence identifiers are later than the deletion event in the sequence and are assigned based at least in part on the record of the deletion event in the external data store. Individual events are replicated or not replicated to the destination data store based at least in part on their sequence identifiers.