Database Cluster Data Migration via Snapshot and Write-Lock
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data migration processes in database clusters disrupt user access and are inefficient, as they require stopping the database service to ensure data consistency, particularly when handling increased user traffic, and do not support seamless migration between heterogeneous databases or during capacity expansion scenarios.
Innovation Solution
A method and apparatus for data migration in a database cluster that allows continuous user access by taking snapshots of source data nodes, recording incremental data, and migrating it to target nodes while performing write-lock operations based on preset conditions, enabling iterative migration and route switching to alleviate storage and processing pressures without interrupting the database service.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the database service is stopped to ensure data consistency during migration, then data consistency is improved, but user access efficiency deteriorates
Solution Approach 1:
The patent applies preliminary action by taking a snapshot of the source data node before migration begins. This snapshot captures the initial state of the data, allowing the system to track and synchronize incremental changes afterward. This preliminary snapshot ensures data consistency can be maintained while the database remains accessible to users during the migration process.
Solution Approach 2:
The patent implements continuity of useful action by enabling the database service to continue operating during data migration. The system continuously captures incremental data changes and synchronizes them to the target node, allowing user access to proceed uninterrupted while migration occurs in the background, thus maintaining both data consistency and user access efficiency.
2Productivity
If incremental data migration is performed without write-lock, then user access efficiency is improved, but data consistency deteriorates
Solution Approach 1:
The patent applies dynamics by implementing conditional write-lock operations. The system dynamically evaluates whether to apply write-lock based on the amount of unmigrated incremental data. When the unmigrated data exceeds a threshold, write-lock is applied to ensure consistency; when it's below the threshold, migration proceeds without write-lock to maintain user access efficiency. This dynamic approach balances data consistency and user access needs.
3Reliability
If write-lock operation is applied frequently to ensure data consistency, then data consistency is improved, but user access efficiency deteriorates
Solution Approach 1:
The patent applies parameter changes by introducing a threshold parameter for unmigrated incremental data. The system monitors the amount of unmigrated data and only applies write-lock operations when this parameter exceeds the threshold. This parameter-based control reduces the frequency of write-lock operations, maintaining data consistency while minimizing impact on user access efficiency.
4Quantity of substance
If complete data migration is performed to alleviate storage pressure, then storage capability is improved, but migration time increases
Solution Approach 1:
The patent applies preliminary action by taking a snapshot of the source data node before migration begins. This snapshot captures the initial state of the data, allowing the system to track and synchronize incremental changes afterward. This preliminary snapshot ensures data consistency can be maintained while the database remains accessible to users during the migration process.
Solution Approach 2:
The patent implements continuity of useful action by enabling the database service to continue operating during data migration. The system continuously captures incremental data changes and synchronizes them to the target node, allowing user access to proceed uninterrupted while migration occurs in the background, thus maintaining both data consistency and user access efficiency.
Data Source
AI summary
The present disclosure is related to a method and an apparatus for data migration in a database cluster. The method includes obtaining a snapshot of a source data node and recording incremental data in a to-be-migrated data shard in the source data node according to inventory data. The method includes migrating the inventory data to a target data node. The method also includes migrating the incremental data, and during the migration when the unmigrated incremental data satisfies a preset write-lock condition, instructing the source data node to perform a write-lock operation on the to-be-migrated data shard and migrate unmigrated incremental data to the target data node. The method further includes, after the migration of the incremental data is completed, instructing a coordinator node to switch a route corresponding to the to-be-migrated data shard from the source data node to the target data node.