Dependency-Aware Parallel Replication for Metadata Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Synchronous replication in computing environments faces increased latency and replication time due to the need for serial execution of metadata operations that affect common inodes, leading to inconsistencies when executed out of order between nodes.
Innovation Solution
Implementing dependency-aware parallel splitting of operations, where independent metadata operations are transmitted in parallel and dependent operations are serialized, considering additional factors like inode relationships, timestamp consistency, and qtree IDs to maintain data integrity and reduce replication latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous replication is used to provide zero recovery point objective, then data protection guarantees are improved, but replication latency increases due to serial execution of metadata operations
Solution Approach 1:
The patent segments metadata operations into independent and dependent operations based on their impact on common inodes. Independent operations can be replicated in parallel to different nodes, while dependent operations are serialized. This segmentation allows the system to achieve both fast parallel replication and data consistency without requiring all operations to be executed serially, thus reducing replication latency while maintaining zero recovery point objective.
2Stability of the object's composition
If metadata operations affecting common inodes are executed serially, then data consistency is maintained, but replication throughput decreases
Solution Approach 1:
The patent divides metadata operations into two categories: independent operations that do not affect common inodes and can be replicated in parallel, and dependent operations that affect common inodes and must be replicated serially. This segmentation enables the system to maximize parallelization for throughput while ensuring serial execution only where necessary for consistency, thereby improving overall replication throughput while maintaining data consistency.
Solution Approach 2:
The patent implements dynamic decision-making for operation replication based on real-time analysis of inode relationships. The system dynamically determines whether operations should be replicated in parallel or serially based on their specific characteristics and impact on common inodes. This dynamic approach allows the system to adapt to varying workloads and optimize both throughput and consistency under different conditions.
3Loss of time
If independent metadata operations are replicated in parallel, then replication latency is reduced, but the complexity of managing operation dependencies increases
Solution Approach 1:
The patent performs preliminary analysis of metadata operations to identify their impact on common inodes before replication begins. By pre-determining which operations are independent and which are dependent, the system can prepare replication plans in advance and execute them efficiently. This preliminary action reduces the complexity of runtime dependency management by making decisions upfront rather than during execution.
Data Source
AI summary
Techniques are provided for dependency aware parallel splitting of operations. For example, a first operation and a second operation may be replicated in parallel from a first device to a second device if the operations only target a single common inode that is an access control list inode referenced by the operations. An operation that dereferences the access control list inode can be replicated in parallel with other operations if the operation does not have the potential to delete the access control list inode from the second device. In another example, operations may be replicated to the second device in parallel if the operations only affect a single common parent directory inode and where timestamps are only moved forward in time at the second device.