Schema-Based Data Migration Using Rule-Driven Document Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data migration schemes are inflexible and fail to leverage performance enhancements when transforming data between different schemas, particularly from relational to non-relational databases, as they do not account for relationships between tables that could speed up the migration process and data access.
Innovation Solution
The systems and methods provide enhanced flexibility by using transformation rules to specify logical arrangements of data structures during migration, allowing linked groupings to be transformed into new or existing structures in the destination database, thereby optimizing data migration and access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional data migration schemes are used to transform data from relational to non-relational databases, then data can be migrated between different schemas, but the migration process is inflexible and does not leverage performance enhancements from table relationships
Solution Approach 1:
The system allows dynamic configuration of transformation rules that can adapt to different data schemas and relationships. Users can define flexible mapping rules that automatically adjust how data is transformed during migration, enabling the system to handle various relational to non-relational transformations efficiently while maintaining high productivity through optimized data structures.
Solution Approach 2:
The system changes parameters of data organization by allowing users to specify different transformation rules for different data groupings. This enables parameterized migration strategies where the same migration framework can handle different data types, relationships, and structures by adjusting transformation parameters, thus achieving both flexibility and efficiency.
2Ease of manufacture
If all tabular data is converted to separate document collections, then data can be stored in non-relational format, but computing resources are wasted and data access becomes less efficient
Solution Approach 1:
The system merges related data from multiple tables into single document structures during migration. By combining data that belongs together into unified documents with embedded relationships, the system reduces the number of separate collections needed, thereby reducing computing resources required for data access operations while maintaining non-relational storage flexibility.
Solution Approach 2:
The transformation framework provides universal rules that can handle multiple data transformation scenarios simultaneously. A single set of transformation rules can process different table groupings, relationships, and data types, making the system multi-functional and efficient rather than requiring separate processing for each data type, thus reducing overall computing resource consumption.
3Device complexity
If table relationships are not preserved during migration, then data can be migrated simply, but future data access operations become less efficient
Solution Approach 1:
The system performs preliminary transformation of data relationships during the migration process itself. By pre-establishing relationships between documents in the non-relational database using the same transformation rules applied to source table relationships, the system prepares data structures that enable efficient future access operations without requiring complex queries later, thus reducing data access time while keeping migration relatively simple.
Data Source
AI summary
Systems and methods described herein may improve data migration between databases that store data under different schemas. Such systems and methods may be used to transform data from a first schema (e.g., tabular, such as in relational databases) to second schema (e.g., document-based, such as in some non-relational databases). In some embodiments, data transformation for database migration may be performed using at least one rule (e.g., specified by a user) specifying a logical arrangement under the second schema for transforming a grouping of source data under the first schema. For example, tables under a tabular schema may be transformed into new documents and/or fields within documents under a document-based schema in a manner specified by the transform rule(s). In some embodiments, transforming source data in this manner permits organizing the migrated data efficiently to reduce computing resources needed to migrate the data between databases and/or for future data access operations.