Backend Track Delayed Release for Deduplication Reclaim
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems face challenges in managing disk fragmentation and data deduplication efficiency, particularly in content-addressable storage systems, where backend tracks are deleted without considering the likelihood of future data reuse.
Innovation Solution
Implement a process that determines whether a backend track is transitioned to a delayed release state or a free state based on a repetition probability score, using an autoregressive moving average (ARIMA) model to forecast data reuse likelihood, thereby preserving data for potential reuse and reducing fragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If backend tracks are deleted immediately without considering future data reuse, then storage space is freed up quickly, but disk fragmentation increases and data deduplication efficiency deteriorates
Solution Approach 1:
The system performs preliminary actions by transitioning backend tracks to a delayed release state before actual deletion, allowing time to assess whether the data will be reused. This preliminary transition enables the system to catch and preserve data that would otherwise be deleted, thereby maintaining deduplication efficiency while still progressing toward space reclamation.
Solution Approach 2:
The system implements feedback mechanisms by monitoring write requests against tracks in the delayed release state. When matching data is detected during this monitoring period, the system responds by preserving the track and preventing deletion. This feedback loop ensures that deletion decisions are adjusted based on actual data reuse patterns, maintaining deduplication efficiency.
2Quantity of substance
If backend tracks are deleted immediately, then storage capacity is increased, but disk fragmentation worsens
Solution Approach 1:
The system performs preliminary actions by transitioning backend tracks to a delayed release state before actual deletion, allowing time to assess whether the data will be reused. This preliminary transition enables the system to catch and preserve data that would otherwise be deleted, thereby maintaining deduplication efficiency while still progressing toward space reclamation.
Solution Approach 2:
The system applies dynamics by implementing a multi-state track management model with transitions between allocated, delayed release, and free states. This dynamic approach allows the system to adapt track status based on observed data reuse patterns, balancing storage capacity release with fragmentation prevention through flexible state changes rather than immediate deletion.
3Reliability
If backend tracks are transitioned to delayed release state, then data availability for reuse is improved, but storage space release is delayed
Solution Approach 1:
The system applies partial action by transitioning only certain backend tracks to the delayed release state based on specific criteria, rather than applying the delay universally. This selective approach ensures that data availability is improved for tracks at risk of being reused, while minimizing the impact on overall storage space release timing for tracks that will not be reused.
Solution Approach 2:
The system changes parameters by implementing a time-based state transition mechanism where tracks move from allocated to delayed release state with a specific waiting period. This parameter change (time delay) allows the system to balance data availability and storage release by adjusting the duration tracks remain in the delayed release state before being freed, optimizing both competing objectives.
Data Source
AI summary
A method for use in a storage system, comprising: detecting that a backend track is required to be deleted, the backend track currently being in an allocated state; identifying a repetition probability score for data that is stored in the backend track; detecting whether the repetition probability score satisfies a predetermined condition; when the predetermined condition is satisfied, transitioning the backend track from an allocated state into a delayed release state, removing any associations between the backend track and one or more frontend tracks, waiting for a predetermined waiting period, and transitioning the backend track from the delayed release state into a free state, wherein the backend track is transitioned into the free state only if no write requests are received at the storage system which include user data matching the data that is currently stored in the backend track.