Data Migration Metrics Prediction Before Repository Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data movers lack the ability to predict or track data migration metrics accurately and fail to correlate these metrics to user-defined entities at the desired level of granularity, leading to difficulties in evaluating costs and time implications for data transfers between different storage devices and environments.
Innovation Solution
A data migration/analysis system that integrates with third-party client applications via API engines, providing predictive data migration metrics and user-friendly interfaces to manage data migrations, including generating and presenting metrics without initiating the migration, and offering historical and correlation metrics to help users evaluate storage utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional data movers are used for data migration, then data migration can be performed, but the ability to predict or track data migration metrics is insufficient
Solution Approach 1:
The system performs preliminary analysis of data migration metrics before the actual migration occurs. It calculates predicted duration, cost, and other metrics in advance by analyzing data characteristics, storage system performance, and migration parameters, allowing users to make informed decisions before committing to the migration.
Solution Approach 2:
The system implements continuous monitoring and feedback mechanisms during data migration. It tracks actual migration progress, compares it with predicted metrics, and provides real-time updates on migration status, enabling dynamic adjustment and accurate tracking of migration performance.
2Measurement precision
If conventional data movers are used, then data migration is performed, but correlation of metrics with entities at desired granularity is lacking
Solution Approach 1:
The system segments metric correlation by multiple entity levels including accounts, data lakes, storage pools, and individual data objects. Each entity type can have its own metric tracking and reporting, allowing granular analysis of migration costs and performance at different organizational levels without requiring a completely new system.
3Loss of time
If data migration is initiated without prediction, then migration can proceed, but cost and duration assessment is unavailable
Solution Approach 1:
The system calculates predicted migration duration, cost, and resource requirements before migration initiation. It analyzes data size, compression ratios, network bandwidth, storage performance, and pricing models to provide accurate estimates, enabling users to evaluate whether the migration aligns with their budget and timeline constraints before committing resources.
Data Source
AI summary
A query specifying a source repository and a target repository is received from a client device. A source index is generated that corresponds to the source repository and represents a snapshot of metadata associated with data contained in the source data repository. The source index is filtered based on filtering criteria specified by the query to obtain a filtered source index. Attributes of data corresponding to the filtered source index are determined as well as data retrieval type parameters. Without initiating a data migration of the data corresponding to the filtered source index from the source repository to the target repository, predicted data migration metrics associated with the data migration are determined and presented to an end user of the client device. The end user is provided with the capability to initiate or forego the data migration based on an evaluation of the predicted data migration metrics.