On-demand Data Migration with Background Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Migrating large amounts of data in data storage systems often consumes significant resources and makes the data unavailable until the migration is complete, leading to inefficiencies and prolonged wait times for users.
Innovation Solution
Implementing on-demand operations that allow data migration to be declared complete before all data is transferred, enabling immediate user access and I/O requests, with remaining data transferred in the background.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data migration is performed completely before declaring migration complete, then data integrity is ensured, but user access time is prolonged and resources are consumed
Solution Approach 1:
The patent applies preliminary action by creating the target data object and declaring migration complete before all data is actually transferred. The target object is prepared in advance with placeholder data or metadata, allowing users to access it immediately. The remaining data migration continues in the background without blocking user operations, thus resolving the contradiction between ensuring data integrity and reducing user wait time.
2Reliability
If all data is migrated before allowing user access, then data completeness is guaranteed, but resource consumption increases and productivity decreases
Solution Approach 1:
The system performs preliminary actions by initializing the target data object and notifying users that migration is complete, even though data transfer is ongoing. This allows users to begin working with the target object immediately, improving productivity while the system continues migrating data in the background to ensure completeness.
Solution Approach 2:
The patent enables continuous useful action by allowing user operations on the target data object to proceed concurrently with the ongoing data migration. The migration process continues in the background without interruption, while users can read, write, and manipulate the target object, thus maintaining continuous productive work rather than idle waiting.
3Ease of operation
If migration is declared complete before full data transfer, then user access is enabled immediately, but data availability may be compromised
Solution Approach 1:
The patent introduces an intermediary mechanism where the target data object serves as a placeholder or intermediary representation of the source data. When users access the target object before full migration, the system mediates by providing available data portions or metadata, allowing partial access while the migration completes. This intermediary approach enables ease of operation while managing data availability expectations.
Data Source
AI summary
Many applications and computing environments allow users to migrate data from a source object to a target object (e.g., a file may be cut/pasted, copied, etc.). It may be advantageous to provide users with access to the data (e.g., migrated data at the target object and/or data that has yet to be migrated from the source object) before all of the data is completely migrated (e.g., a user may otherwise have to wait hours for a 2 TB file to be copied between various data volumes). Accordingly, as provided herein, migration of a source object to a target object may be declared as completed, even though the target object may not comprise all of the data that is to be migrated. In this way, an I/O request may be satisfied based upon migrated data within the target object and/or data, not yet migrated, retrieved on-demand from the source object.