Data Gravity Synthesis Using Live Database Activity Reflection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database migration methods are inefficient and can take months, causing data integrity issues and rendering the database inaccessible for extended periods, while maintaining online migration introduces data inconsistency due to ongoing changes.
Innovation Solution
Implementing a live database activity reflection tool that synchronizes frequently accessed tables using AI/ML models to monitor and reflect updates, ensuring data integrity and expediting the migration process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional database migration methods are used to transfer 100 TB of data, then the database can be migrated to a new platform, but the migration takes a minimum of 100 days and renders the database inaccessible for more than three months
Solution Approach 1:
The patent segments the database migration into multiple phases: (1) creating a duplicate instance with initial data copy, (2) implementing real-time data change capture and application, (3) switching traffic to the duplicate instance, and (4) completing the migration. This segmentation allows the database to remain accessible during migration by switching to the duplicate instance, thereby reducing downtime while maintaining data integrity through controlled phase transitions.
Solution Approach 2:
The patent performs preliminary actions by creating a duplicate instance of the database before the actual migration begins. This duplicate instance is pre-configured with the same structure and initially populated with data, allowing it to serve as a standby system during migration. This preliminary preparation enables seamless switching and minimizes service interruption.
2Productivity
If the database is kept online during migration, then database accessibility is maintained, but data integrity issues arise because content may be changed before migration completes
Solution Approach 1:
The patent implements a feedback mechanism through real-time monitoring of data changes in the source database. A data change capture component continuously monitors the source database for modifications, and these changes are immediately captured and applied to the duplicate instance. This feedback loop ensures that the duplicate instance remains synchronized with the source, maintaining data consistency while the source database remains online and accessible.
Solution Approach 2:
The patent introduces an intermediary data change capture component that acts as a mediator between the source database and the duplicate instance. This intermediary captures data changes from the source database and applies them to the duplicate instance, ensuring data consistency is maintained without requiring direct access to the source database during the migration process.
3Reliability
If real-time data change capture and application is implemented, then data consistency is maintained during migration, but the system complexity increases with additional monitoring and synchronization components
Solution Approach 1:
The patent implements multi-functional components that perform multiple tasks. The data change capture component not only monitors data changes but also queues and manages the transmission of change data. The data change application component both receives change data and applies it to the duplicate instance. This multi-functionality reduces the number of separate components needed, thereby managing system complexity while maintaining data consistency.
4Speed
If only frequently accessed tables are synchronized, then migration speed is improved, but completeness of data migration is reduced
Solution Approach 1:
The patent applies partial action by initially focusing synchronization efforts on frequently accessed tables that provide the most value for immediate service restoration. However, the system is designed to progressively synchronize additional tables beyond the frequently accessed ones, ensuring complete data migration. This approach allows the migration to proceed at high speed initially while still achieving full data completeness over time.
Data Source
AI summary
A system comprising a processor that may be configured to: monitor an output of a feed that outputs a respective log of each activity that occurs on a source database platform; evaluate the output of the feed against an update session taxonomy to determine whether the respective log indicates a completion of a first update session of first updates to a first source table of a source instance of a live database; when the respective log indicates the completion of the first update session, search a historical database activity repository to detect each respective log that corresponds to the first update session; identify a corresponding data change for each respective log that corresponds to the first update session; and perform a first set of data changes on a copy of the first source table.