Cloud Data Transfer Snapshot Block Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for transferring data to a cloud service often result in data inconsistency when modifications occur during the transfer process, as they typically require preventing modifications to maintain data integrity, which can be unacceptable due to the time required for transfer.
Innovation Solution
The system initiates a snapshot of the data, transfers each block of the snapshot to the cloud service, and terminates the snapshot after all blocks have been transferred, allowing for modifications to be copied to a storage pool before they are written, enabling modifications during transfer while maintaining data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data modification is prevented during transfer to cloud service, then data integrity of the replica is maintained, but productivity is reduced due to transfer time requirements
Solution Approach 1:
The data set is divided into multiple blocks that are transferred individually to the cloud service. Each block can be transferred independently, allowing the system to maintain snapshots of specific blocks while enabling modifications to other blocks, thus improving transfer efficiency without compromising data integrity.
Solution Approach 2:
Snapshots are created for data blocks before they are transferred to the cloud service. This preliminary action captures the state of each block at the time of transfer initiation, ensuring that even if modifications occur during transfer, the original consistent state is preserved in the snapshot, maintaining data integrity while allowing continued productivity.
2Productivity
If data modification is allowed during transfer, then productivity is maintained, but data inconsistency occurs in the replica at cloud service
Solution Approach 1:
The system creates copies of data blocks in snapshot form before transfer to the cloud service. These snapshots serve as immutable replicas of the original blocks at the time of transfer initiation. Even if the original blocks are modified during transfer, the snapshots ensure that the cloud service receives consistent, unmodified copies, thereby maintaining data consistency while allowing productivity.
3Reliability
If complete data set is transferred before allowing modifications, then data integrity is ensured, but loss of time occurs during the transfer process
Solution Approach 1:
The complete data set is segmented into multiple transferable blocks, each with its own snapshot. This allows the transfer process to be divided into smaller, parallelizable units, reducing the overall transfer duration while ensuring each block's integrity through individual snapshots.
Solution Approach 2:
The snapshot mechanism enables continuous data modification actions to occur during the transfer process without interrupting the transfer itself. Unlike the conventional approach that requires complete transfer before modifications, this method maintains continuous useful action (modifications) throughout the transfer, reducing time loss.
Data Source
AI summary
Transferring data from a storage device to cloud service includes initiating a snapshot of the data, accessing each block of the data corresponding to the snapshot to transfer each block to the cloud service, and terminating the snapshot after all of the blocks have been transferred to the cloud service. At least some blocks of the storage device that are modified after initiating the snapshot may be copied from the storage device to a storage pool prior to modification. Only a first modification of a particular one of the blocks of the storage device may cause the particular one of the blocks to be copied to the storage pool. Accessing each block of the data may include accessing blocks of the storage pool. Modifying a particular one of the blocks of the storage device may include modifying a corresponding block of a storage pool.