Data Replication Framework Using Version Numbers for Conflict Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database replication systems face challenges in maintaining consistency and scalability, particularly in multi-master replication scenarios, where transactional conflicts and data inconsistencies arise due to increased complexity and the strain on centralized relational databases, leading to potential overloading and reduced availability.
Innovation Solution
A data replication framework that utilizes a network of interconnected hosts with local data stores, enabling writes to any host and maintaining membership awareness across all hosts, employing version numbers and recorders for conflict resolution and ensuring consistent data replication without relying on a centralized storage system, thus allowing for dynamic request routing and load balancing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multi-master replication is implemented to improve availability and scalability, then data can be written to any host independently, but transactional conflicts and data inconsistencies arise due to increased complexity
Solution Approach 1:
The system segments the centralized database into multiple distributed data stores across different hosts. Each host maintains its own data store and can independently accept write operations, eliminating the single point of failure and improving availability while distributing the system's complexity across multiple independent units.
Solution Approach 2:
Version numbers are introduced as intermediaries to manage conflicts in multi-master replication. Each data item carries a version number that allows the system to detect and resolve conflicts automatically through version comparison, reducing the complexity of conflict management while maintaining data consistency across distributed hosts.
2Stability of the object's composition
If a centralized relational database is used to store data for multiple services, then data consistency is maintained, but the database becomes overloaded and unresponsive as the number of hosts and access requests increases
Solution Approach 1:
The centralized database is segmented into multiple distributed data stores, one on each host. This segmentation distributes the access load across multiple independent storage units, preventing any single database from becoming overloaded while maintaining data consistency through the replication framework and version control mechanisms.
Solution Approach 2:
Each host maintains a local data store with locally cached data, allowing read operations to be served locally without accessing the centralized database. This local quality improvement reduces the access burden on the centralized system while maintaining data availability and consistency through periodic synchronization.
3Productivity
If localized caches are used on hosts to reduce access requests to the centralized database, then scalability is improved, but coherency issues arise and RAM resources are constrained
Solution Approach 1:
The local caches on different hosts are merged into a unified distributed data store through the replication framework. Version numbers and conflict resolution mechanisms are used to combine data from multiple sources while maintaining coherency, allowing the system to scale beyond individual host memory constraints while preserving data consistency.
Solution Approach 2:
The system changes the parameters of cache management by using version numbers as a key parameter for tracking and synchronizing cached data across hosts. This parameter change enables automatic detection of stale data and triggers appropriate synchronization actions, resolving coherency issues while maintaining scalability.
4Stability of the object's composition
If synchronous replication is used to prevent conflicts, then data consistency is maintained, but write operations must wait for acknowledgments from all databases, reducing performance
Solution Approach 1:
The system uses partial acknowledgment where a write operation is considered successful when acknowledged by a quorum of hosts rather than all hosts. This partial action approach maintains data consistency for the majority of the system while reducing the time latency for write operations, as the operation can proceed once a sufficient number of replicas confirm the write.
Data Source
AI summary
Generally described, the present disclosure is directed to an eventually consistent replicated data store that uses, for its underlying storage, a computer software library that provides a high-performance embedded database for data. The replicated data store employs a plurality of hosts interconnected to one another, allowing for writes to any host and full awareness of membership across all hosts. With the data replication framework disclosed herein, various modes are allowed to be built up on top of the core system.