Synchronous Replication Timestamping for Parallel Split Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing synchronous replication techniques face challenges in maintaining timestamp consistency and data integrity during parallel splitting of operations, leading to potential data loss and replication inconsistencies between nodes.
Innovation Solution
Implement a splitter component that intercepts operations to assign the same timestamp to both the operation and its replication, ensuring consistent timestamping across nodes, and selectively replicate operations based on on-disk modifications and file access policies to maintain data integrity and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If parallel splitting of operations is implemented for synchronous replication, then replication latency is reduced and productivity is improved, but timestamp consistency between nodes deteriorates
Solution Approach 1:
The system performs preliminary timestamp assignment by extracting and storing the timestamp from the original operation request before parallel execution. This pre-assigned timestamp is then applied to both the primary and replicated operations, ensuring consistency without waiting for parallel execution to complete. The timestamp is captured in advance (preliminary action) to avoid the inconsistency that would arise from assigning timestamps after parallel execution.
Solution Approach 2:
The system introduces an intermediary mechanism (the timestamp extraction and propagation logic) that mediates between the parallel operation execution and the final state assignment. This intermediary captures the timestamp from the request and ensures it is consistently applied across both primary and replicated operations, resolving the timestamp consistency issue that would otherwise arise from parallel execution.
2Reliability
If all operations are replicated to ensure data integrity, then reliability is improved, but network bandwidth consumption increases and productivity deteriorates
Solution Approach 1:
The system applies local quality by selectively replicating only those operations that modify on-disk data, rather than uniformly replicating all operations. The replication logic examines the operation type and data modification status, and only forwards operations that actually change persistent storage state. This reduces unnecessary network traffic while maintaining data integrity for operations that matter.
Solution Approach 2:
The system performs partial replication by sending only the necessary subset of operations to the remote node. Instead of replicating all operations (excessive action), it filters to replicate only on-disk modifying operations, which is the minimum necessary to maintain data integrity. This partial action approach reduces network bandwidth consumption while preserving reliability.
3Productivity
If selective replication based on on-disk modifications is implemented, then network bandwidth usage is reduced and productivity is improved, but reliability may deteriorate due to potential data loss
Solution Approach 1:
The system applies local quality by selectively replicating only those operations that modify on-disk data, rather than uniformly replicating all operations. The replication logic examines the operation type and data modification status, and only forwards operations that actually change persistent storage state. This reduces unnecessary network traffic while maintaining data integrity for operations that matter.
Solution Approach 2:
The system uses feedback mechanisms to determine whether an operation resulted in on-disk modifications. The replication logic receives feedback about the operation outcome and uses this information to decide whether replication is necessary. This feedback-driven approach ensures that only operations that actually changed data are replicated, maintaining reliability while improving efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques are provided for timestamp consistency. An operation targeting a first storage object having a synchronous replication relationship with a second storage object is intercepted. A timestamp is assigned to the operation. A replication operation is created as a replication of the operation. The same timestamp is assigned to the replication operation. The operation is implemented upon the first storage object and the replication operation is implemented upon the second storage object.