Cloud Storage Synchronous Replication via Mediation Service
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face challenges in efficiently synchronizing and replicating datasets across multiple storage systems, particularly in cloud-based environments, leading to issues with data consistency, redundancy, and failover mechanisms.
Innovation Solution
The implementation of a storage system architecture that utilizes non-volatile solid state storage units with embedded CPUs and energy reserves, along with erasure coding and redundancy schemes, to ensure data synchronization and replication across multiple storage nodes, enabling proactive data rebuilding and high availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous replication is implemented across multiple storage systems, then data consistency is improved, but system complexity and latency increase
Solution Approach 1:
A mediation service is introduced as an intermediary component that coordinates synchronous replication between storage systems. The mediation service receives replication requests, manages the coordination logic, and communicates with multiple storage systems, thereby centralizing complexity in a dedicated component rather than distributing it across all storage systems. This allows individual storage systems to maintain simpler architectures while achieving coordinated replication through the mediator.
Solution Approach 2:
The system implements feedback mechanisms where the mediation service monitors replication status and provides status information back to storage systems and clients. This feedback loop enables real-time tracking of replication progress and data consistency status, allowing the system to respond dynamically to replication events and maintain reliability without requiring complex proactive management at each storage node.
2Reliability
If synchronous replication is implemented across multiple storage systems, then data consistency is improved, but operational latency increases
Solution Approach 1:
The mediation service performs preliminary actions by pre-establishing communication channels and replication configurations between storage systems before actual data replication is needed. By preparing the replication infrastructure in advance, the system reduces the time required to execute synchronous replication operations, as the foundational setup is already in place rather than being created on-demand.
3Reliability
If redundancy schemes are implemented, then data availability is improved, but storage capacity is reduced
Solution Approach 1:
The system merges multiple storage systems into a unified replicated storage architecture where data exists across multiple physical locations. By combining the storage resources of multiple systems and coordinating them through the mediation service, the system achieves redundancy and improved availability while presenting a consolidated storage capacity to clients, effectively utilizing the aggregate capacity of all participating storage systems rather than treating them as separate isolated resources.
Data Source
AI summary
Synchronously replicating a dataset across cloud-based storage systems, including adding a cloud-based storage system to a set of storage systems that the dataset is synchronously replicated across, where access operations are applied to the dataset equivalently through all storage systems in the set, all storage systems in the set store a separate copy of the dataset, and operations to modify the dataset performed and completed through any of the storage systems in the set are reflected in access operations to read the dataset, the cloud-based storage system including one or more cloud computing instances executing a storage controller application, a virtual drive layer that includes one or more cloud computing instances with local storage for storing at least a portion of the dataset as block data, and an object storage layer for storing at least a portion of the dataset as object data.