Online Data Migration Server with Mirror and Incremental Sync
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing database systems face inefficiencies in data processing and quality of service due to the need to suspend services when transferring large volumes of hot data to cold data, which affects query and modification operations.
Innovation Solution
A data management method and server that perform online migration of service data by generating mirror data and incremental data, migrating them to separate service data sets based on time validity, and clearing the data in the source set once migration is complete, ensuring continuous data access and reduced downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If service data is transferred from hot data set to cold data set, then data storage organization is improved, but service quality deteriorates due to suspension of query and modification operations
Solution Approach 1:
The patent segments the data migration process into multiple stages: first migrating base data to the cold data set, then separately migrating incremental data changes. This segmentation allows the hot data set to remain accessible during base data migration while incremental changes are applied afterward, preventing service suspension.
Solution Approach 2:
The patent performs preliminary actions by creating mirror data of the hot data set before actual migration occurs. This mirror data serves as a placeholder and allows the system to prepare the cold data set in advance, enabling seamless transition without interrupting ongoing query and modification operations.
2Productivity
If data migration is performed to separate hot and cold data sets, then data management efficiency is improved, but processing time increases due to suspension of operations
Solution Approach 1:
The patent ensures continuity of useful action by maintaining the hot data set as the active working set throughout the migration process. Query and modification operations continue uninterrupted on the hot data set while migration proceeds in the background. The cold data set receives data asynchronously, ensuring continuous data management efficiency without time loss.
Solution Approach 2:
The patent implements dynamic data management by allowing the hot data set to dynamically receive new data while simultaneously migrating old data to the cold data set. The system adapts the migration timing and methodology based on current data access patterns, optimizing both efficiency and processing time.
3Reliability
If incremental data is recorded during migration process, then data consistency is improved, but system complexity increases
Solution Approach 1:
The patent implements feedback mechanisms by recording incremental data changes during the migration process and using this feedback to update the cold data set. The system continuously monitors data changes in the hot data set and applies corresponding incremental updates to the migrated data, ensuring consistency without requiring complex real-time synchronization systems.
Data Source
AI summary
A data management method is provided. To-be-migrated base data in a first service data set is obtained, mirror data that are same as the base data are generated, and the base data to a second service data set are migrated. Incremental data obtained for the base data are recorded in the migration process of the base data. Adding processing is performed on the mirror data by using the incremental data, and the incremental data are migrated to the second service data set, when the migration process of the base data is completed. The mirror data and the incremental data in the first service data set are cleared when the migration process of the incremental data is completed.