Dynamic Overlapping Write Manager for Synchronous Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Synchronous replication systems face challenges in maintaining data consistency and scalability when handling overlapping writes and replication errors across large numbers of files, particularly in maintaining the order of execution and handling misaligned holes and writes beyond the end of a file.
Innovation Solution
The system dynamically allocates and recycles file range locks and employs an overlapping write manager to serialize writes, pre-allocates and manages overlapping write managers, and performs a quick reconcile operation to maintain data consistency and handle errors, ensuring that the order of execution is preserved and replication errors are neutralized.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous replication is implemented to maintain data consistency, then data protection reliability is improved, but system complexity increases due to handling overlapping writes and maintaining execution order across multiple files
Solution Approach 1:
The patent segments the replication system into distinct components: an overlapping write manager that handles write serialization, file range locks that manage individual file access, and a reconcile operation that handles discrepancies. This segmentation allows each component to address specific aspects of the complexity problem independently while maintaining overall system reliability through coordinated operation.
2Reliability
If file range locks are allocated for each file to handle overlapping writes, then write serialization reliability is improved, but memory consumption increases with large numbers of files
Solution Approach 1:
The patent implements a recycling mechanism for file range locks. When a file's writes are completed and the file is closed, its associated range lock is released back to the pool of available locks. This allows the same lock objects to be reused for different files over time, significantly reducing the total number of lock instances needed in memory while maintaining reliable write serialization for each file.
3Productivity
If overlapping write managers are pre-allocated for all files, then write handling productivity is improved, but resource allocation complexity increases
Solution Approach 1:
The patent implements dynamic allocation of overlapping write managers based on actual system conditions. Rather than pre-allocating a fixed number of managers for all files, the system allocates managers as needed when files are opened and writes are initiated, and releases them when files are closed. This dynamic approach improves productivity by ensuring managers are available when needed while reducing allocation complexity by responding to actual demand rather than anticipating all possible scenarios.
4Reliability
If synchronous replication is implemented to maintain zero RPO, then data consistency is improved, but performance decreases due to waiting for replication acknowledgment
Solution Approach 1:
The patent performs reconcile operations preliminarily during normal operation rather than waiting for failures or performing extensive checks after writes. By proactively comparing and reconciling data between primary and secondary storage systems during the write process, the system ensures zero RPO is maintained without requiring additional performance-critical operations later, thus preserving overall system performance while guaranteeing data consistency.
Data Source
AI summary
Techniques are provided for overlapping write handling. Overlapping write managers are used to maintain the order that write operations are executed at a first computing environment and replicated to a second computing environment. Overlapping write managers are pre-allocated as available for managing overlapping write operations. A mapping is used to track what overlapping write managers are currently allocated for particular file handles of files. Thus, if an incoming write operation targets a file handle of an already allocated overlapping write manager, then that overlapping write manager is used to execute and replicate the incoming write operation so that the order of execution of overlapping writes by the second computing environment is the same as at the first computing environment. If there is no allocated overlapping write manager for the file handle, then a new overlapping write manager is allocated and utilized.