Secure Cross-Region File System Delta Transfer for Scalable Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current disaster recovery methods for cloud-based file systems lack scalability, manageability, and security in cross-region replication, particularly in managing large data volumes and ensuring consistent point-in-time views across different cloud infrastructure regions.
Innovation Solution
A file storage service generates snapshot deltas between source and target file systems, encrypts them using session keys, and transfers these deltas through high-throughput object storage to recreate snapshots in a target file system, utilizing parallel processing threads and secure key management to ensure scalability, reliability, and consistency during replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional disaster recovery methods are used for cross-region file system replication, then data protection is provided, but scalability and manageability deteriorate when handling large data volumes
Solution Approach 1:
The patent segments the file system data into incremental snapshots and further divides them into delta chunks that can be independently transferred. This segmentation allows the replication process to handle large data volumes efficiently by processing smaller units in parallel, improving scalability while maintaining data protection across regions.
Solution Approach 2:
The patent introduces a new dimension of operation by allowing concurrent upload and download processes to occur simultaneously across different regions. This dimensional change in the replication timeline enables asynchronous operations that improve scalability without compromising reliability, as each region can operate independently at different speeds.
2Productivity
If incremental snapshot replication is implemented, then data transfer efficiency is improved, but security and consistency across regions worsen without proper encryption and key management
Solution Approach 1:
The patent applies preliminary action by encrypting snapshot deltas with session keys before transfer and pre-establishing key management structures in both source and target regions. This preliminary encryption and key setup ensures that security and consistency are maintained throughout the efficient incremental replication process, resolving the contradiction between speed and security.
Solution Approach 2:
The patent introduces session keys as intermediaries that facilitate secure data transfer between regions. These keys act as mediators that enable efficient incremental replication while maintaining security, as the keys are temporarily used for encryption/decryption and then discarded, ensuring consistency without compromising security.
3Reliability
If synchronous replication is used to ensure consistency, then data integrity is maintained, but operational flexibility and speed deteriorate
Solution Approach 1:
The patent applies dynamics by allowing the replication system to adapt its synchronization level based on operational needs. The system can operate in asynchronous mode for flexible, independent regional operations while still maintaining data integrity through incremental snapshots and delta transfers, providing operational flexibility without sacrificing data integrity.
Solution Approach 2:
The patent uses periodic incremental snapshots to maintain data integrity across regions. Instead of continuous synchronous replication, the system periodically captures snapshots and transfers deltas, which maintains data integrity while allowing operational flexibility between snapshot intervals, resolving the contradiction between consistency and flexibility.
4Ease of operation
If manual replication management is implemented, then administrative control is maintained, but complexity and time consumption increase for large-scale deployments
Solution Approach 1:
The patent implements self-service by enabling automated incremental snapshot creation, delta generation, and transfer processes that require minimal administrative intervention. The system automatically manages the replication workflow between regions, reducing management complexity while maintaining administrative control through configuration options, resolving the contradiction between control and complexity.
Data Source
AI summary
Novel techniques for end-to-end file storage replication and security between file systems in different cloud infrastructure regions are disclosed herein. In one embodiment, a file storage service generates deltas between snapshots in a source file system, and transfers the deltas and associated data through a high-throughput object storage to recreate a new snapshot in a target file system located in a different region during disaster recovery. The file storage service utilizes novel techniques to achieve scalable, reliable, and restartable end-to-end replication. Novel techniques are also described to ensure a secure transfer of information and consistency during the end-to-end replication.