Data Migration Rule Engine for Heterogeneous Database Type Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Migrating data between databases of different types, especially from an on-premise to a public cloud environment, is challenging due to differing data types and the lack of security-focused migration tools, often requiring manual intervention and additional devices/software.
Innovation Solution
A system and method that automatically detects data type differences, applies conversion rules, and migrates data using a Java batch program, with parallel threading and encryption, to convert and transfer data between databases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing data migration tools are used for migrating data from on-premise to public cloud, then data migration can be performed, but security concerns arise and manual intervention is required due to data type differences
Solution Approach 1:
The migration system automatically detects data type differences between source and target databases, selects appropriate conversion rules, and executes the migration without requiring manual intervention. The system serves itself by implementing automated data type detection, rule selection, and conversion execution, eliminating the need for operators to manually correct data type issues.
Solution Approach 2:
The system pre-configures multiple conversion rules for different data type scenarios before migration begins. During migration, the system has already prepared the necessary conversion logic and can immediately apply the appropriate rules when data type mismatches are detected, avoiding the need for manual rule creation or adjustment during the migration process.
2Adaptability or versatility
If additional devices and software are installed to facilitate data migration, then data migration capability is improved, but device complexity increases
Solution Approach 1:
The migration system is designed to handle multiple data types and database formats using a single unified platform. It supports various conversion rules for different data type scenarios (DATE to VARCHAR, NUMBER to DATE, etc.) within one system, eliminating the need for multiple specialized tools or additional software installations for different migration scenarios.
Solution Approach 2:
The system introduces an automated conversion rule engine as an intermediary between the source and target databases. This rule engine detects data type mismatches and automatically applies appropriate conversion rules, serving as a mediator that resolves compatibility issues without requiring additional specialized devices or software for each specific data type scenario.
3Manufacturing precision
If operators manually intervene to correct or pre-process data before migrating, then data migration accuracy is improved, but productivity decreases
Solution Approach 1:
The system automatically detects data type differences and applies conversion rules without requiring manual data correction or pre-processing. The automated detection and conversion process maintains data migration accuracy by systematically identifying and correcting data type mismatches, while eliminating the time-consuming manual intervention that would otherwise be required.
Solution Approach 2:
The system replaces the manual mechanical process of data correction and pre-processing with an automated computational system. The automated rule engine detects data type issues and applies conversions algorithmically, substituting human operators with an efficient automated process that maintains accuracy while significantly increasing migration speed and productivity.
Data Source
AI summary
Systems and methods for migrating data are provided. A migration application communicates with a source database and a target database, and receives a command to migrate data therebetween. It automatically detects that, for a source data field in the source database that is to be migrated to a target data field in the target database, the source data field is of a first data type and the target data field is of a second data type that is different from the first data type. The migration application automatically searches a rules database for a conversion rule in relation to the first data type and the second data type. The migration application automatically executes the conversion rule to convert data in the source data field to converted data that is of the second data type. The migration application automatically migrates the converted data to the target database.