Client-Side Two-Phase Commit for Distributed Data Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed virtualized computing environments, ensuring data consistency across multiple servers when capturing data sets is challenging, particularly when writes are distributed across different servers, leading to potential logical inconsistencies and performance issues.
Innovation Solution
Implementing a client-side two-phase commit mechanism where client devices suspend write acknowledgments and insert a capture token into the request stream, ensuring that all writes confirmed before the capture are included, and none submitted after are included, thereby maintaining data consistency without requiring servers to wait for each other.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a capture is taken of a distributed data set partitioned across multiple servers, then data consistency can be achieved, but servers must wait for each other increasing latency
Solution Approach 1:
The system divides the capture operation into two independent phases: (1) inserting capture tokens into request streams at client devices, and (2) servers independently capturing data when tokens are received. This segmentation eliminates the need for servers to wait for each other while maintaining consistency within each server's partition.
Solution Approach 2:
The capture token is inserted into the request stream before the actual capture operation. This preliminary action marks the boundary of consistent writes in advance, allowing servers to independently identify and capture only the relevant data portion without waiting for other servers.
2Reliability
If write acknowledgments are suspended to ensure consistency, then data integrity is maintained, but request processing speed decreases
Solution Approach 1:
Write acknowledgments are suspended only locally at the client device for the specific duration needed to insert the capture token, rather than globally across the entire system. This localized suspension minimizes the impact on overall request processing speed while maintaining data integrity.
Solution Approach 2:
The system dynamically suspends and resumes write acknowledgments based on the capture operation state. Acknowledgments are suspended only when a capture token needs to be inserted, and resumed immediately afterward, allowing the system to adapt its consistency enforcement to actual operational needs.
3Reliability
If captures are taken across distributed servers, then complete data representation is achieved, but complexity of coordinating captures increases
Solution Approach 1:
The coordination complexity is extracted from the server side and moved to the client side. Client devices are responsible for inserting capture tokens into request streams, while servers simply execute local capture operations independently when tokens are received, eliminating the need for complex inter-server coordination.
Solution Approach 2:
The capture token acts as an intermediary signal that travels through the request stream from client to server. This simple token mechanism replaces complex coordination protocols, enabling distributed captures through a lightweight, standardized signaling approach.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques, including systems and methods, for capturing data sets include performing a client-side two-phase commit to ensure one or more data consistency conditions. A logical volume may represent a data set that is distributed among a plurality of physical storage devices. One or more client devices are instructed to block at least acknowledgment of write operations. When the one or more client devices have blocked at least acknowledgment of write operations, one or more servers in communication with the physical storage devices are instructed to capture corresponding portions of the data set. When the servers have been instructed to capture corresponding portions of the data set, the client devices are instructed to resume at least acknowledgment of write operations.