Cloud Storage Dataset Replication with Mediated Synchronous I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in efficiently managing data replication and synchronization across multiple storage nodes, particularly in cloud-based environments, leading to complexities in data availability, consistency, and performance.
Innovation Solution
The implementation of a system that synchronously replicates datasets across multiple storage nodes, utilizing advanced methods for establishing and managing replication relationships, mediating I/O operations, and synchronizing metadata, ensures high availability and performance in cloud-based storage systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated across multiple storage nodes, then data availability is improved, but system complexity increases
Solution Approach 1:
The patent introduces a mediation layer that coordinates replication operations between storage nodes. This intermediary mechanism manages the complexity of multi-node replication by centralizing control logic, thereby improving data availability without proportionally increasing overall system complexity.
Solution Approach 2:
The replication system is divided into independent replication relationships between node pairs. Each replication relationship can be managed independently, allowing the system to scale by adding node pairs rather than managing all nodes simultaneously, thus reducing the complexity burden of replication across multiple nodes.
2Reliability
If synchronous replication is implemented across storage nodes, then data consistency is improved, but I/O operation complexity increases
Solution Approach 1:
A mediation mechanism is introduced to handle synchronous replication coordination. The mediator manages the complexity of coordinating I/O operations across multiple nodes by providing a centralized control point, thereby maintaining data consistency without proportionally increasing I/O operation complexity.
Solution Approach 2:
The system establishes replication relationships and mediates I/O operations in advance through structured setup procedures. By pre-configuring replication relationships and establishing mediation protocols before actual data operations, the system reduces the complexity burden during active I/O operations while maintaining synchronous consistency.
3Reliability
If replication relationships are established between multiple storage nodes, then data availability is improved, but metadata synchronization complexity increases
Solution Approach 1:
Metadata synchronization is segmented into individual replication relationship contexts. Each replication relationship maintains its own metadata state independently, allowing metadata to be synchronized on a per-replication-basis rather than requiring global metadata coordination across all nodes simultaneously, thus reducing synchronization complexity.
Data Source
AI summary
Synchronously replicating a dataset across cloud-based storage systems, including adding a cloud-based storage system to a set of storage systems that the dataset is synchronously replicated across, where access operations are applied to the dataset equivalently through all storage systems in the set, all storage systems in the set store a separate copy of the dataset, and operations to modify the dataset performed and completed through any of the storage systems in the set are reflected in access operations to read the dataset, the cloud-based storage system including one or more cloud computing instances executing a storage controller application, a virtual drive layer that includes one or more cloud computing instances with local storage for storing at least a portion of the dataset as block data, and an object storage layer for storing at least a portion of the dataset as object data.