Predictive Garbage Collection for SMR Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems, particularly those using shingled magnetic recording (SMR) disks, face high computational and operational costs during garbage collection due to the need to rewrite entire zones when altering individual files, and existing methods do not efficiently determine which files to delete, leading to excessive data processing and resource utilization in data centers.
Innovation Solution
The method involves predicting the survivor lifetime of individual files within a container using machine learning algorithms to determine a deletion predictor, calculating the cost and benefit of garbage collection, and scheduling it at optimal epochs where the net benefit is positive, thereby optimizing garbage collection operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If garbage collection is performed on SMR disks by rewriting entire zones, then storage density is increased, but computational cost and operational expense increase significantly
Solution Approach 1:
The patent segments the container into individual files and predicts which specific files are likely to be deleted using machine learning. Instead of treating the entire zone as a single unit that must be rewritten, the system identifies and processes only the relevant files for garbage collection, thereby reducing the computational cost while maintaining the storage density benefits of SMR disks.
2Measurement precision
If the entire collection of data is read and processed to identify files for deletion, then accurate garbage identification is achieved, but operational cost and processing time increase
Solution Approach 1:
The patent applies preliminary action by using machine learning models to predict which files are likely to be deleted before garbage collection is performed. This prediction step is executed in advance, allowing the system to identify target files without reading and processing the entire collection of data, thus improving operational efficiency while maintaining accurate garbage identification.
3Loss of energy
If frequent garbage collection is performed to maintain storage efficiency, then data storage costs are reduced, but resource utilization and processing overhead increase
Solution Approach 1:
The patent implements dynamic garbage collection scheduling based on predicted file deletion patterns. The machine learning model continuously learns from access patterns and deletion behaviors, adjusting the timing and scope of garbage collection operations dynamically. This allows the system to perform garbage collection at optimal moments when it provides maximum benefit with minimum resource utilization, rather than following a fixed frequent schedule.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A predictive method for scheduling of the operations is described. The predictive method utilizes data generated from computing an expected lifetime of the individual files or objects within the container. The expected lifetime of individual files or objects can be generated based on machine learning techniques. Operations such as garbage collection are scheduled at an epoch where computational efficiencies are realized for performing the operation.