Multi-Cloud Data Replication via Cloud Array Intermediary
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud storage solutions lack secure, reliable, and scalable multi-cloud data replication capabilities, particularly across different APIs and systems, leading to inadequate security, availability, and high costs due to inefficient data transfer and management processes.
Innovation Solution
A method and system for secure and reliable multi-cloud data replication using a data replication cluster with a host node, cloud array application, and local cache, which encrypts and compresses data, employs RAID-1 replication, and unifies different cloud storage interfaces via a standard iSCSI interface, ensuring secure, efficient, and scalable data transfer across multiple cloud storage systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated across multiple cloud storage systems with different APIs, then reliability and availability are improved, but device complexity and integration difficulty increase
Solution Approach 1:
The patent implements a cloud array application layer that acts as an intermediary between the host node and multiple cloud storage systems. This application layer handles API translation and data format conversion, allowing the system to work with different cloud providers (Amazon S3, Nirvanix SDN, etc.) without requiring changes to the core replication logic. The intermediary absorbs the complexity of multiple APIs while presenting a unified interface to the replication process.
Solution Approach 2:
The cloud array application is designed with multi-functionality to support multiple cloud storage systems simultaneously. It can translate between different APIs and data formats, making the replication system universal across various cloud providers. This allows a single replication infrastructure to serve multiple cloud storage systems without requiring separate systems for each provider.
2Object-affected harmful factors
If data is encrypted and compressed during replication, then security and bandwidth efficiency are improved, but processing time and computational resources increase
Solution Approach 1:
The system performs encryption and compression of data before transmission to cloud storage systems. By preparing the data in advance with security measures and optimization, the actual data transfer is more efficient and secure. The preliminary processing ensures that data is protected and optimized before it enters the network, reducing the burden during transmission.
Solution Approach 2:
The replication system maintains continuous operation by processing data in streams and using buffering mechanisms. While encryption and compression add processing steps, the system keeps the data flow continuous through efficient pipeline processing, minimizing idle time and maintaining high utilization of network and storage resources.
3Reliability
If entire files are uploaded or downloaded as required by different cloud systems, then data integrity is maintained, but bandwidth utilization efficiency decreases
Solution Approach 1:
The patent divides data into smaller extents or blocks that can be independently managed and transferred. Instead of requiring entire files to be uploaded or downloaded, the system can transfer individual extents or blocks as needed. This segmentation allows for more efficient bandwidth utilization while maintaining data integrity through proper reconstruction of the complete data set from its segments.
Solution Approach 2:
The system changes the parameter of data transfer granularity from file-level to extent or block-level. By adjusting this parameter, the system can optimize transfer sizes based on network conditions and cloud system requirements, transferring only the necessary portions of data rather than entire files, thus improving bandwidth efficiency while maintaining integrity.
4Adaptability or versatility
If specialized software is installed for one-time data migration, then data transfer between different cloud systems is enabled, but ease of operation and scalability deteriorate
Solution Approach 1:
The patent implements a universal cloud array application that works across multiple cloud storage systems without requiring specialized software for each migration scenario. The application provides a standardized interface and translation layer that enables data transfer between different cloud providers using the same replication infrastructure, making the system adaptable to various cloud environments while maintaining ease of operation.
Solution Approach 2:
The system merges multiple cloud storage interfaces and APIs into a unified cloud array application layer. This consolidation allows the system to handle migrations between different cloud providers through a single standardized interface, eliminating the need for separate specialized software tools for each cloud pair and significantly improving ease of operation and scalability.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A multi-cloud data replication method includes providing a data replication cluster comprising at least a first host node and at least a first online storage cloud. The first host node is connected to the first online storage cloud via a network and comprises a server, a cloud array application and a local cache. The local cache comprises a buffer and a first storage volume comprising data cached in one or more buffer blocks of the local cache's buffer. Next, requesting authorization to perform cache flush of the cached first storage volume data to the first online storage cloud. Upon receiving approval of the authorization, encrypting the cached first storage volume data in each of the one or more buffer blocks with a data private key. Next, assigning metadata comprising at lest a unique identifier to each of the one or more buffer blocks and then encrypting the metadata with a metadata private key. Next, transmitting the one or more buffer blocks with the encrypted first storage volume data to the first online cloud storage. Next, creating a sequence of updates of the metadata, encrypting the sequence with the metadata private key and then transmitting the sequence of metadata updates to the first online storage cloud.