Deduplication Data Migration Based on Extent References
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data backup systems face inefficiencies in managing deduplicated data, particularly in determining which data objects to move from primary to secondary storage, as current caching algorithms like FIFO and LRU do not effectively account for the likelihood of data being referenced again, leading to suboptimal storage space utilization and network traffic.
Innovation Solution
A method that determines the duration of data objects' residency in primary storage, calculates space occupied by unreferenced extents, and moves data objects to secondary storage based on whether they have external references, optimizing space savings and network efficiency by allowing time for matching with incoming data objects before migration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data objects are moved from primary to secondary storage using traditional caching algorithms (FIFO/LRU), then storage space is freed in primary storage, but network traffic increases and storage efficiency deteriorates because unreferenced data is moved prematurely
Solution Approach 1:
The system performs preliminary deduplication processing before moving data objects to secondary storage. By checking for extent references and calculating space savings in advance, the system ensures that only data objects with actual space savings potential are moved, preventing premature migration of still-referenced data and reducing unnecessary network traffic.
Solution Approach 2:
The system implements a feedback mechanism where data objects are monitored for extent references after being stored in primary storage. The decision to move data to secondary storage is based on feedback about whether extents are referenced by other data objects, creating a closed-loop control system that optimizes storage efficiency while minimizing network traffic.
2Volume of stationary object
If data objects are moved quickly to secondary storage, then primary storage capacity is increased, but long-term storage efficiency is reduced because data may be referenced again
Solution Approach 1:
The system performs preliminary checks for extent references before moving data objects to secondary storage. This preliminary action ensures that data objects which are still needed (referenced by other data objects) remain in primary storage, while only truly unreferenced data is moved, thereby maintaining both storage capacity and long-term storage efficiency.
Solution Approach 2:
The system enables data objects to effectively 'self-select' for migration to secondary storage based on their reference patterns. By monitoring whether extents are referenced by other data objects, the system allows data that serves no further purpose in primary storage to be automatically identified and moved, while data that is still useful remains in place.
Data Source
AI summary
In one aspect of the present description, in connection with storing a first deduplicated data object in a primary storage pool, described operations include determining the duration of time that the first data object has resided in the primary storage pool, and comparing the determined duration of time to a predetermined time interval. In addition, described operations include, after the determined duration of time meets or exceeds the predetermined time interval, determining if the first data object has an extent referenced by another data object, and determining whether to move the first data object from the primary storage pool to a secondary storage pool as a function of whether the first data object has an extent referenced by another data object after the determined duration of time meets or exceeds the predetermined time interval. Other features and aspects may be realized, depending upon the particular application.