Storage Garbage Collection via Access Window Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Garbage collection of data objects in large-scale storage environments is challenging due to the complexity of data structures and limited knowledge about consumer needs, leading to inefficient use of storage resources.
Innovation Solution
A computer-implemented method that aggregates access data, determines future storage costs, and identifies data objects with expired access windows to reclaim storage space by notifying users and generating deletion instructions when deemed cost-effective.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated garbage collection is performed on all data objects, then storage space reclamation is improved, but data loss risk increases due to limited knowledge about consumer needs
Solution Approach 1:
The system performs preliminary actions by determining access windows and expiration dates for data objects before deletion occurs. This allows the system to proactively identify candidate data objects for deletion based on predicted future access patterns, while still allowing for user feedback before final deletion takes place
Solution Approach 2:
The system introduces feedback mechanisms by notifying users about candidate data objects for deletion and receiving user responses. This feedback loop allows users to confirm or prevent deletion of data objects, ensuring that data retention decisions align with actual consumer needs while maintaining high automation for clear-cut cases
2Reliability
If manual user confirmation is required for each data object deletion, then data loss risk is reduced, but processing time increases significantly
Solution Approach 1:
The system applies partial automation by requiring user confirmation only for data objects where the predicted access window extends into the future or where uncertainty exists about data value. For data objects with clearly expired access windows and no predicted future access, the system proceeds with automated deletion without user intervention, thus reducing processing time while maintaining reliability for ambiguous cases
3Reliability
If storage space is retained for all data objects beyond current access, then data loss risk is reduced, but storage resource efficiency decreases
Solution Approach 1:
The system performs preliminary analysis to determine access windows and expiration dates for data objects before deletion occurs. This allows the system to proactively identify candidate data objects for deletion based on predicted future access patterns, optimizing storage reclamation while maintaining data availability during the access window period
Solution Approach 2:
The system dynamically changes the retention parameter for data objects based on their predicted future access patterns. Data objects with predicted future access maintain their retention status, while those with expired or non-predicted access windows are marked for deletion, thus optimizing storage resource utilization while preserving data availability when needed
Data Source
AI summary
Methods, systems, apparatus, including computer programs encoded on computer storage media, for reclaiming storage space in a storage environment. In one aspect, the method includes actions of aggregating data that is indicative of access to one or more data objects, determining a future storage cost associated with each of a plurality of data objects, determining an access window for each of the plurality of data objects, identifying a data object based on (i) the future storage cost that satisfies a predetermined threshold and (ii) a data object access window, providing a notification to a user device that requests feedback from a user indicating whether the data object can be deleted, and in response to receiving data that indicates that the data object can be deleted, generating an instruction to cause deletion of the data object upon the expiration of the access window.