Data Migration Version Comparison to Filter Deleted Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In clustered storage systems, data migration processes often result in resource wastage due to ineffective data deletion, where data remanence occurs because the source disk does not receive deletion instructions, leading to unnecessary I/O operations.
Innovation Solution
Implement a method where the first node compares version numbers of data from the second node and third nodes to filter out data for deletion, ensuring that only up-to-date data is migrated and deleted, thereby reducing resource wastage and improving migration performance and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is migrated from source disk to target disk without version number verification, then data migration can be completed, but data remanence occurs and I/O resources are wasted
Solution Approach 1:
The patent applies preliminary action by comparing version numbers before data migration to identify and filter out data that has already been deleted or modified at the source. This pre-check prevents unnecessary migration of obsolete data, avoiding subsequent deletion operations and wasting I/O resources. The version number comparison is performed in advance to determine which data blocks need actual migration.
2Loss of energy
If version number comparison is performed before data migration, then I/O resource consumption is reduced, but system complexity increases
Solution Approach 1:
The patent implements feedback by using version numbers as metadata that provide information about the current state of data at the source. This feedback mechanism allows the migration system to make informed decisions about whether to migrate specific data blocks. The version number comparison creates a feedback loop that guides the migration process, enabling intelligent resource allocation without significantly increasing system complexity.
3Reliability
If all data is migrated regardless of deletion status, then data completeness is maintained, but resource wastage increases
Solution Approach 1:
The patent applies the extraction principle by separating data blocks into two categories: those that need migration (where source version differs from target version) and those that don't need migration (where versions match or source data is deleted). This extraction of only the necessary data blocks from the complete data set eliminates wasteful migration operations while maintaining data completeness for the data that actually needs to be transferred.
Data Source
Figure 1A~1B
Figure 2A~2B
Figure 3
AI summary
This application provides a data migration method and apparatus. The method includes: in a process in which a second node migrates data to a first node, reading, by the first node from the second node, to-be-migrated data and a version number of the to-be-migrated data, and reading, by the first node from at least one third node, a version number of data that belongs to a same first service as the to-be-migrated data, where the data of the first service is distributively stored in the second node and the at least one third node; and then when determining that the version number of the to-be-migrated data read from the second node is different from a version number of data read from any one of the third node, discarding, by the first node, the to-be-migrated data read from the second node. In this way, during data migration, the version number of the to-be-migrated data and a version number of data that is in another node and that belongs to a same service as the to-be-migrated data are compared, to filter out to-be-deleted data. This can reduce a waste of IO resources for ineffective migration and subsequent deletion.