Online Data Fragment Migration With MVCC Consistency Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed transactional databases face challenges in moving data fragments while maintaining transactional consistency and performance, particularly when ongoing transactions modify the data during movement, leading to degraded performance and unavailability.
Innovation Solution
A method and system for moving data fragments in a transactional distributed database using multi-version concurrency control, where rows are labeled with commit timestamps, allowing new versions to be created during movement, ensuring transactional isolation and preserving performance by maintaining replicas at source and destination servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data fragment movement is performed while the database is online and transactions are processing, then system availability is maintained, but transactional consistency becomes difficult to preserve
Solution Approach 1:
The system performs preliminary actions by creating a copy of the data fragment at the destination server before completing the movement. This preliminary copy allows the system to prepare for the transition while maintaining the original data at the source, ensuring that consistency can be preserved throughout the migration process.
Solution Approach 2:
The patent introduces an intermediary mechanism where the data fragment exists in a transitional state with replicas at both source and destination servers. This intermediary state allows ongoing transactions to continue accessing the data while the system gradually transitions ownership from source to destination, maintaining consistency through controlled replication and synchronization.
2Loss of information
If data fragment movement is performed by stopping processing on the data fragment, then transactional consistency is maintained, but system availability and performance degrade
Solution Approach 1:
The system dynamically adjusts the state of data fragments during movement, allowing them to transition from being exclusively at the source server to being replicated at both source and destination servers. This dynamic approach enables the system to maintain availability by allowing transactions to continue processing while the data fragment location and accessibility evolve over time.
Solution Approach 2:
The patent ensures continuity of useful action by allowing transactions to continue processing on data fragments during the movement operation. Instead of stopping processing, the system maintains continuous access to the data through replication, ensuring that the useful action of transaction processing continues uninterrupted while the data fragment is being migrated.
3Loss of information
If data fragment movement is performed with a process that highly disrupts ongoing processing, then data integrity is preserved, but quality of service degrades and performance is lost
Solution Approach 1:
The system uses copying to create a replica of the data fragment at the destination server while the original remains at the source server. This copying approach allows the data to be migrated without disrupting ongoing transactions, as the copy can gradually replace the original. The copying mechanism preserves data integrity while maintaining quality of service, as transactions can continue to access the data through the original until the migration is complete.
Data Source
Figure 1a~1b
Figure 2~3
Figure 4
AI summary
This invention comprises a method to move a source data fragment from a source data server to a destination data server while it is being changed. The method enables to preserve the data consistency using commit timestamps and comprises: configuring the destination data server to buffer changes for the data fragment; sending changes for the data fragment to both data servers; configuring the source data server to send the data fragment to the destination data server; configure the destination data server to apply all the changes while still buffering new changes for the data fragment; once the changes have been consumed, sending changes only to the destination data server; and configuring the source data server to remove the data fragment. The invention further provides a transactional distributed database system which performs the method.