Distributed Data Storage Causality-Based Reconciliation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data locking mechanisms in distributed data storage systems are costly in terms of computational resources and latency, reducing performance by requiring frequent communication across multiple data servers for every data access request, which negatively impacts response time and availability.
Innovation Solution
Implementing an eventual consistency model that allows data servers to process write and read requests without locking data fields or databases, using causality chains to manage data replacements and reconcile values during read requests, thereby eliminating the need for inter-server communication during write operations and enhancing system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data locking mechanisms are implemented to ensure data accuracy and consistency, then data consistency is improved, but system performance and response time deteriorate due to computational overhead and frequent inter-server communication
Solution Approach 1:
The patent extracts the data consistency guarantee mechanism from the request-processing path by using causal versioning and conflict detection only when necessary. Instead of locking data during every operation, the system allows concurrent access and resolves conflicts only when actual conflicts are detected through version comparison, thereby removing the performance bottleneck while maintaining consistency.
Solution Approach 2:
Each data server independently tracks its own data versions using causal version vectors and autonomously detects conflicts by comparing versions with other servers. This self-service approach eliminates the need for centralized coordination and frequent inter-server communication, allowing servers to operate independently while still ensuring overall system consistency.
2Reliability
If data locking mechanisms are implemented to ensure data accuracy, then data consistency is improved, but latency increases due to communication overhead across multiple data servers
Solution Approach 1:
The system performs preliminary version tagging and causal tracking at the time of data writes, embedding version information directly in the data. This preliminary action allows for efficient conflict detection during reads without requiring real-time communication between servers, as the version information is already available locally for comparison.
3Speed
If data is replicated across multiple data servers in different regions, then data access speed is improved, but data consistency becomes more difficult to maintain due to concurrent updates by different servers
Solution Approach 1:
The patent segments the consistency guarantee into independent causal chains for each data item, allowing different servers to operate on different segments of the data without interfering with each other. Each server maintains its own version vector and can independently determine whether to accept or reject updates based on causal version comparison, enabling parallel operation while maintaining overall consistency.
Data Source
AI summary
Methods and systems are presented for providing data consistency in a distributed data storage system using an eventual consistency model. The distributed data storage system may store data across multiple data servers. To process a request for writing a first data value for a data field, a first data server may generate, for the first data value, a first causality chain representing a data replacement history for the data field leading to the first data value. The first data server may insert the first data value without deleting pre-existing data values from the data field. To process a data read request, multiple data values corresponding to the data field may be retrieved. The first data server may then select one data value based on the causality chains associated with the multiple data values for responding to the data read request.