Data Distribution Service Using Ordered Cross-Region Watch Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing systems face high latency and availability issues when aggregating or searching data across multiple regions due to the need for cross-region internet calls and varying latency between regions.
Innovation Solution
A data distribution service that enables eventually consistent replication of data objects across different geographical regions using a watch-based API architecture, persistent TCP connections, and policy-based distribution control, with features like subscription multiplexing, fault tolerance, and security measures to maintain data consistency and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If cross-region reads are made on demand when data is needed, then data availability across regions is improved, but latency increases due to internet calls between regions
Solution Approach 1:
The system performs preliminary data replication to multiple regions before actual read operations are needed. Data is proactively distributed across regions using publish/subscribe mechanisms, so that when reads are required, the data is already available locally in each region without requiring cross-region internet calls, thus eliminating latency while maintaining availability
Solution Approach 2:
The system creates copies of data objects in multiple geographic regions simultaneously. Each region maintains its own copy of the data, allowing local reads without accessing the original data location across the internet. This copying approach resolves the contradiction by providing both immediate local access (low latency) and distributed availability (high reliability)
2Adaptability or versatility
If fan out requests to each region individually to aggregate or search data, then data aggregation capability is improved, but system complexity and latency variation increase
Solution Approach 1:
The system merges the data aggregation function into the replication infrastructure itself. By using a unified publish/subscribe mechanism that operates across all regions, the system combines data distribution and aggregation capabilities into a single coordinated operation, eliminating the need for complex individual region requests and result combination logic
Solution Approach 2:
The replication system is designed to serve multiple functions simultaneously: data distribution, data aggregation, and search operations all use the same underlying publish/subscribe infrastructure. This universal approach allows the system to handle diverse data operations through a single mechanism, reducing complexity while maintaining versatility
3Reliability
If data is replicated across multiple regions, then data availability and reduced latency are improved, but data consistency and conflict resolution become more difficult
Solution Approach 1:
The system implements feedback mechanisms through acknowledgment confirmations and sequence number tracking. When data is published to a region, the system receives feedback about the data's successful replication and can use sequence numbers to detect and resolve conflicts, ensuring consistency across all regions while maintaining high availability
Data Source
AI summary
A cross-region data distribution service enables eventually consistent replication of data objects between geographical regions while maintaining data ordering guarantees. For data replication between two regions, the system leverages a data stream-based replication model where destination region leverages a watch API exposed by origin regions. The origin region watch API provides a resumable data stream for changes happening in the origin region that need to be replicated to destination. The system exposes distribution ability through distribution policies which are global objects that encode the source datasets and desired distribution topology. In order to solve data fanout scalability challenges where same data needs to be replicated to many regions, system leverages caching at origin watch servers. The system maintains data ordering by enforcing ordered replication stream provided by watch servers and using version based conditional optimistic concurrency control writes in destination servers.