Live Datastore Migration Using Two-Phase Commit Sync
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data migration systems require significant downtime and incur excessive computing processing during data migration from one storage system to another, especially when migrating data from a source datastore to a target datastore, and do not scale well, leading to production processing data issues.
Innovation Solution
A live data migration system utilizing a dual delete and dual writing process with a two-phase commit mechanism to perform cross datastore transactions, reducing processing rounds and eliminating the need for code and configuration changes at the application server level, while maintaining acceptable service levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If data is migrated from source datastore to target datastore using traditional migration systems, then data migration is completed, but significant downtime occurs and excessive computing processing is required
Solution Approach 1:
The system performs preliminary actions by establishing a secondary datastore and progressively copying data before the actual migration is needed. The two-phase commit mechanism prepares transactions in advance, allowing the system to switch datastores without downtime. This preliminary setup enables seamless migration by having the target datastore ready before the source datastore needs to be replaced.
Solution Approach 2:
The patent introduces an intermediary mechanism (the two-phase commit protocol and middleware layer) that mediates between the application and the datastore during migration. This intermediary handles the complexity of data synchronization and transaction management, allowing the application to continue operating while data is being migrated in the background, thus eliminating downtime and reducing computing overhead.
2Adaptability or versatility
If traditional data migration systems are used, then data can be moved between storage systems, but the systems do not scale well and lead to production processing data issues
Solution Approach 1:
The system implements a universal migration framework that can handle multiple datastore types and scenarios through a standardized two-phase commit interface. The middleware layer provides multi-functional capabilities including data copying, validation, rollback management, and seamless switching, making the system adaptable to various production environments and scalable across different migration scenarios without compromising reliability.
3Ease of manufacture
If code and configuration changes are made at the application server level for data migration, then migration can be performed, but the complexity increases and service levels may be disrupted
Solution Approach 1:
The patent extracts the migration complexity from the application server level and places it in a dedicated middleware layer. This separation allows the application to remain unchanged while the migration logic is handled independently by the two-phase commit mechanism and datastore management system. By taking out the migration complexity from the application layer, the system becomes easier to implement and maintain without disrupting service levels.
Data Source
AI summary
Systems, methods, and computer readable media for a live data migration system where the methods include accessing a command to perform a live data migration of a source datastore to a target datastore, accessing a delete operation, the delete operation indicating first data to be deleted from a datastore, and determining the datastore is stored in a source datastore. The methods further includes performing a two-phase commit delete of the first data from the source datastore and the target datastore, accessing a write operation, the write operation indicating second data, and performing a two-phase commit write of the second data to the source datastore and to the target datastore. The methods further include copying the source datastore to the target datastore, accessing a read operation, the read operation indicating third data to be read from the datastore, and performing the read operation of the third data from the target datastore.