Snapshot And Log Synchronization For Database Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data migration methods in distributed databases face inefficiencies due to limited data fragment sizes and synchronization issues during scale-in/scale-out, leading to blocked communication links and impaired system availability and performance.
Innovation Solution
A data migration method involving snapshot export of to-be-migrated data, incremental synchronization of the data migration log, and route switching based on routing rules, ensuring high availability and efficient log synchronization without blocking synchronization links.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If communication link is used to directly transmit data blocks, then data migration can be performed, but the sizes of data fragments are limited due to short delay requirements, leading to lack of flexibility
Solution Approach 1:
The patent segments the data migration process into two distinct phases: full data migration using snapshots (which can handle large data volumes without delay constraints) and incremental data migration using logs (which handles only changed data). This segmentation allows each phase to use optimized transmission methods appropriate to its needs, resolving the contradiction between fragment size flexibility and communication delay.
Solution Approach 2:
The patent performs preliminary full data migration using snapshots before incremental log-based migration. By预先 completing the bulk data transfer through snapshots, the system eliminates the need for repeated transmission of unchanged data, thereby increasing flexibility in data fragment sizing while minimizing communication delay for the critical incremental updates.
2Reliability
If synchronization is performed for large data sets, then data consistency is maintained, but communication links may be blocked, affecting efficiency of data synchronization
Solution Approach 1:
The patent extracts only the essential incremental changes (logs) from the source database after initial snapshot migration. By taking out only the necessary synchronization data (changes since the snapshot) rather than synchronizing entire large data sets repeatedly, the system maintains data consistency while minimizing communication link blocking and improving synchronization efficiency.
Solution Approach 2:
The patent performs preliminary full data migration using snapshots before incremental log-based migration. This preliminary action ensures that the bulk of data is already synchronized, so subsequent synchronization operations only need to handle incremental changes, thereby maintaining consistency without blocking communication links during large data set transfers.
3Productivity
If route switching is performed during data migration, then target database can be activated, but it may interrupt normal data read-write operations
Solution Approach 1:
The patent performs preliminary full data migration using snapshots before incremental log synchronization and route switching. By completing the bulk data transfer in advance, the system ensures that when route switching occurs, the target database is already populated with complete data, enabling faster activation without interrupting read-write operations on the source database during the migration process.
Solution Approach 2:
The patent maintains continuous data read-write operations on the source database throughout the migration process. By using snapshots for full migration and logs for incremental sync, the system ensures uninterrupted useful action (data operations) while the target database is being prepared and activated, thus maintaining availability while enabling smooth transition.
Data Source
AI summary
Embodiments of the present disclosure provide a data migration method and device, wherein the data migration method comprises: acquiring a migration request for a source database, creating a snapshot of to-be-migrated data in the source database and migrating the snapshot to a target database, reading a data migration log stored in a primary replica database of the source database and synchronizing the data migration log into the target database, and configuring a route for a target primary replica database of the target database according to routing rules of the primary replica database.