Hybrid Cloud Data Sync via Intermediary Service
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Synchronizing data across hybrid cloud systems with heterogeneous network storage systems is challenging due to different software stacks and configurations, leading to inefficiencies in data replication and metadata synchronization.
Innovation Solution
A computing device communicates with multiple network storage systems using open standard APIs and messaging protocols, employing data versioning and event triggers to ensure accurate metadata updates, and provides granular permission levels for optimized performance across diverse systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is synchronized across heterogeneous network storage systems with different storage protocols, then data availability and accessibility across multiple cloud providers is improved, but system complexity and difficulty of integration increases
Solution Approach 1:
The patent introduces an intermediary synchronization service that mediates between heterogeneous storage systems using different protocols (S3, Swift, RPC). This service translates and coordinates data synchronization requests across protocol boundaries, enabling multi-cloud data synchronization without requiring direct integration between incompatible systems, thus resolving the technical contradiction between adaptability and system complexity
Solution Approach 2:
The synchronization service implements universal functionality by supporting multiple storage protocols (S3, Swift, RPC) through a unified interface. This multi-protocol capability allows the same system to work across different cloud providers and storage types, achieving broad adaptability while maintaining a single coordinated control point that manages the complexity
2Reliability
If real-time data replication is implemented across multiple storage systems, then data consistency is improved, but network bandwidth consumption and transfer time increases
Solution Approach 1:
The system implements periodic synchronization events where data is replicated at scheduled intervals rather than continuously in real-time. This periodic approach maintains data consistency across storage systems while significantly reducing network bandwidth consumption by only transferring updates when necessary, rather than maintaining constant synchronization streams
Solution Approach 2:
The system performs preliminary actions by identifying and queueing synchronization events before actual data transfer. This allows for optimization opportunities such as batching multiple small transfers into larger efficient transfers, pre-allocation of network resources, and prioritization of critical data, thereby reducing overall bandwidth consumption while maintaining consistency
3Measurement precision
If comprehensive metadata synchronization is performed across all storage systems, then data accuracy and version control is improved, but processing overhead and synchronization time increases
Solution Approach 1:
The system extracts and synchronizes only the essential metadata elements required for data accuracy and version control, rather than synchronizing all possible metadata attributes. This selective extraction approach maintains measurement precision for critical data integrity information while reducing the overall processing overhead and synchronization time by excluding non-essential metadata
4Reliability
If granular permission control is implemented across heterogeneous systems, then data security is improved, but system complexity and permission management overhead increases
Solution Approach 1:
The synchronization service implements universal permission management that works across all heterogeneous storage systems through a unified authorization framework. This single multi-functional permission system manages access control for all target storage systems, achieving granular data security across the entire multi-cloud environment while reducing complexity by eliminating the need for separate permission management systems for each storage provider
Data Source
AI summary
In some examples, a computing device may communicate with a plurality of network storage systems, such as a first network storage system provided by a first service provider employing a first storage protocol and a second network storage system provided by a second service provider employing a second storage protocol different from the first storage protocol. The computing device receives a first object, and determines, for the first object, a first remote bucket at the first network storage system and a second remote bucket at the second network storage system. The computing device may add a synchronization event to a queue for replicating the first object to the first remote bucket and the second remote bucket. Based on consuming the synchronization event from the queue, the computing device replicates data and metadata of the first object to the first remote bucket and the second remote bucket.