Data Migration Planning With Workload-Balanced Cloud Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in migrating data from one storage infrastructure to another, such as local to cloud-based storage, resulting in downtime, computational costs, and inefficiencies due to varying database capacities and data types, without adequately balancing the workload.
Innovation Solution
A migration planning system that generates automated plans using a greedy mapping technique and swap optimization to efficiently migrate data from a first party infrastructure to a public cloud, minimizing downtime and costs by balancing workload across cloud database instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is migrated from one storage infrastructure to another, then data can be moved to cloud-based storage, but downtime occurs and data availability is disrupted
Solution Approach 1:
The system performs preliminary actions by creating a copy of the source database and preparing the target database structure before actual data migration begins. This allows the migration process to occur without disrupting the original data availability, as the source system continues to operate normally during the migration preparation and execution phases.
2Productivity
If data migration is performed without optimization, then migration can be completed, but computational costs increase and efficiency decreases
Solution Approach 1:
The migration process is segmented into distinct phases: analysis phase (assessing source database characteristics), planning phase (creating optimized migration plans), and execution phase (performing the actual migration). This segmentation allows for optimized resource allocation and computational strategies to be applied at each stage, reducing overall computational costs and improving efficiency.
Solution Approach 2:
The system changes parameters such as batch size, migration timing, and resource allocation based on the analyzed characteristics of the source database. By dynamically adjusting these parameters, the system optimizes the migration process to minimize computational costs while maintaining high efficiency.
3Productivity
If data migration is performed without workload balancing, then migration can proceed, but system performance degrades due to uneven database capacities
Solution Approach 1:
The system applies local quality by tailoring the migration strategy to the specific characteristics of each database instance involved in the migration. By analyzing the unique properties of the source and target databases, the system creates customized migration plans that optimize workload distribution for each specific case, rather than applying a one-size-fits-all approach.
Data Source
AI summary
The disclosed techniques for generating a migration plan include identifying one or more entities that are eligible for data migration to a destination database from a source database. The techniques include generating, using planning procedures that include a workload balancing procedure, a data migration plan for the eligible entities and executing the migration plan. The workload procedure includes mapping, based on data metric values of the eligible entities, different ones of the eligible entities to instances in the destination database, where the mapping is performed based on utilization metric values of the instances, and where the instances are of a storage service that collectively implements the destination database. The workload balancing procedure includes altering the mappings of entities to instances in the destination database, where the remapping is based on a standard deviation of data for entities mapped to instances in the destination database not meeting a threshold standard deviation.