Target Database Sizing With Machine Learning for Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for determining the size of a target database during migration, especially from traditional SQL databases to modern NoSQL databases, rely on manual estimation, which is error-prone and inefficient, leading to potential failures or resource wastage due to overestimation or underestimation.
Innovation Solution
A machine learning-based approach using supervised artificial intelligence techniques and statistical analysis to predict the required size of a target database by training migration models on various database types, considering factors like table structure, data format, and expected data volumes, and applying linear or exponential regression as needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If manual estimation methods are used to determine target database size, then the process is simple to implement, but the accuracy and reliability of size determination deteriorates
Solution Approach 1:
The patent replaces manual estimation methods (mechanical/human process) with machine learning models that automatically predict target database size based on source database characteristics and migration factors. This substitution maintains ease of implementation through automated processes while dramatically improving prediction accuracy by leveraging trained ML algorithms that analyze historical migration data and identify complex patterns invisible to manual methods.
Solution Approach 2:
The patent introduces machine learning models as an intermediary between the source database and target database provisioning process. These models act as intelligent mediators that process source database metadata, migration configuration, and historical data to generate accurate size predictions, thereby resolving the contradiction between simple implementation and accurate measurement.
2Productivity
If manual estimation is used for target database size, then resource allocation can be quickly determined, but the reliability of migration success deteriorates due to overestimation or underestimation
Solution Approach 1:
The patent performs preliminary actions by training machine learning models on historical migration data before actual migrations occur. These pre-trained models enable rapid, accurate predictions during migration planning, allowing quick resource allocation decisions while ensuring high reliability through predictions based on learned patterns from numerous past migrations rather than speculative manual estimates.
Solution Approach 2:
The system implements feedback mechanisms where actual migration outcomes are fed back into the training data for continuous model improvement. This feedback loop ensures that the machine learning models become increasingly accurate over time, maintaining high migration success rates while preserving the speed advantage of automated predictions.
3Measurement precision
If automated machine learning models are used to predict target database size, then the accuracy of size determination is improved, but the device complexity increases
Solution Approach 1:
The patent creates universal machine learning models that can handle multiple database type combinations (SQL to NoSQL, different NoSQL types, etc.) through a single unified system. This multi-functionality approach improves accuracy across diverse migration scenarios while avoiding the complexity of maintaining separate manual estimation processes for each database type combination.
Solution Approach 2:
The machine learning models perform self-service by automatically selecting appropriate prediction algorithms, processing input parameters, and generating size predictions without requiring complex manual configuration. The system self-manages the complexity of model selection and parameter processing, providing accurate predictions while keeping the user interface simple.
4Reliability
If conservative overestimation is used for target database size, then migration reliability is improved by avoiding underallocation, but resource wastage increases
Solution Approach 1:
The patent changes the fundamental parameter of size prediction from conservative static overestimation to dynamic ML-based prediction that adapts to specific migration characteristics. By analyzing source database structure, data distribution, and migration transformation rules, the system generates optimized size estimates that minimize wastage while maintaining reliability, replacing the one-size-fits-all conservative approach with tailored predictions.
Data Source
AI summary
An apparatus comprises a processing device configured to identify a source database of a first database type that is to be migrated to a target database of a second database type, to determine a set of migration factors for migrating the source database to the target database, and to select a given migration machine learning model for estimating a size required for the target database based at least in part on the first database type of the source database and the second database type of the target database. The processing device is further configured to estimate the size required for the target database utilizing the given migration machine learning model and the determined set of migration factors and to provision, for the target database, a given amount of storage resources based at least in part on the estimated size required for the target database.