Multi-Master Data Replication with Vector Clock Conflict Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Global social networking platforms face challenges in maintaining data consistency and low latency across geographically distributed data centers, leading to potential outages and poor user experiences due to the limitations of single-region data storage.
Innovation Solution
A geographically distributed, multi-master system that employs vector clocks and conflict resolution techniques to ensure consistent data replication across multiple regions, using data read/write engines, conflict resolution engines, and consistency rules engines, along with loop-back queues and message queues for asynchronous replication without the need for distributed transaction coordinators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a single geographic region, then data consistency is maintained, but latency increases and availability decreases during regional outages
Solution Approach 1:
The system segments data storage across multiple geographic regions, allowing data to be accessed from the nearest region. Each region maintains a copy of the data, enabling users to access data locally without waiting for cross-region synchronization, thus reducing latency while maintaining availability during outages in other regions.
Solution Approach 2:
The patent introduces vector clocks as an intermediary mechanism to coordinate updates across multiple regions without requiring distributed transaction coordinators. Vector clocks enable deterministic conflict resolution by providing a causal ordering of events, allowing regions to independently resolve conflicts without blocking, thereby maintaining low latency and high availability.
2Speed
If data is replicated across multiple geographic regions, then availability and latency are improved, but data consistency and conflict resolution become problematic
Solution Approach 1:
The system uses vector clocks to provide feedback about the causal state of data across regions. Each update carries its vector clock information, allowing receiving regions to determine whether to apply updates immediately or hold them for conflict resolution. This feedback mechanism ensures eventual consistency while allowing fast local reads, resolving the contradiction between speed and reliability.
Solution Approach 2:
The patent changes the parameter of conflict resolution from centralized coordination to distributed deterministic resolution using vector clocks. By changing the resolution strategy to compare vector clock values and apply conflict resolution functions locally, the system achieves consistency without sacrificing the performance benefits of multi-region replication.
3Reliability
If distributed transaction coordinators are used to ensure consistency, then data consistency is maintained, but system complexity and overhead increase
Solution Approach 1:
The patent extracts the coordination function from a centralized distributed transaction coordinator and distributes it to each region through vector clocks. Each region independently maintains vector clock state and resolves conflicts locally using deterministic functions, eliminating the need for complex centralized coordination while maintaining consistency.
Solution Approach 2:
Each region serves itself by independently resolving conflicts using vector clocks and deterministic conflict resolution functions. Regions do not need to coordinate with a central authority or other regions to resolve conflicts, as the vector clock mechanism provides sufficient information for each region to make consistent decisions autonomously, reducing system complexity.
4Reliability
If cross-region synchronization is implemented, then data consistency is improved, but storage overhead and synchronization complexity increase
Solution Approach 1:
The system performs preliminary action by pre-computing vector clock values with each data update. This preliminary tagging of data with causal information enables efficient conflict detection and resolution without requiring additional storage for conflict metadata or complex synchronization protocols, minimizing storage overhead while ensuring consistency.
Data Source
AI summary
This disclosure relates to a geographically distributed, multi-master system for storing data records, and associated methods and computer-readable media for replicating data records across geographically distributed data stores of the system in a manner that achieves consistency between data stored in geographically distributed regions as well as deterministic data replication.