Virtual Drive Layer for Synchronous Cloud Dataset Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud-based storage systems face challenges in synchronously replicating datasets across multiple storage systems while maintaining data integrity and ensuring seamless access and operations, particularly in scenarios involving cloud computing environments.
Innovation Solution
Implementing a virtual drive layer and an object storage layer within a cloud-based storage system that includes cloud computing instances, where datasets are synchronously replicated across storage systems, ensuring that all systems have a separate copy and operations are reflected uniformly, with a storage controller application managing these processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If datasets are synchronously replicated across multiple storage systems, then data integrity is improved, but system complexity increases
Solution Approach 1:
A virtual drive layer is introduced as an intermediary between the object storage layer and cloud computing instances. This virtual drive layer manages dataset replication across multiple storage systems, handling the complexity of synchronous replication while presenting a simplified interface to both the storage and compute layers. The virtual drive layer coordinates replication operations and ensures data integrity without requiring complex modifications to existing storage systems.
2Reliability
If synchronous replication is implemented across cloud storage systems, then data consistency is improved, but access time increases
Solution Approach 1:
The system performs preliminary actions by pre-establishing replication relationships and pre-synchronizing datasets across storage systems before actual data access occurs. The virtual drive layer maintains ready-to-access replicated copies, so when data is requested, the system can quickly retrieve it from the nearest available copy without performing synchronization operations at access time.
3Reliability
If multiple storage systems replicate datasets, then system reliability is improved, but operational coordination difficulty increases
Solution Approach 1:
The virtual drive layer is designed as a universal component that can manage replication across different types of storage systems and present a unified interface to cloud computing instances. It handles multiple functions including data replication, consistency management, and access coordination through a single standardized interface, eliminating the need for separate coordination mechanisms for each storage system.
Data Source
AI summary
Synchronously replicating a dataset across cloud-based storage systems, including adding a cloud-based storage system to a set of storage systems that the dataset is synchronously replicated across, where access operations are applied to the dataset equivalently through all storage systems in the set, all storage systems in the set store a separate copy of the dataset, and operations to modify the dataset performed and completed through any of the storage systems in the set are reflected in access operations to read the dataset, the cloud-based storage system including one or more cloud computing instances executing a storage controller application, a virtual drive layer that includes one or more cloud computing instances with local storage for storing at least a portion of the dataset as block data, and an object storage layer for storing at least a portion of the dataset as object data.