Just-in-time segment cleaning for storage fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network-based storage systems face performance degradation due to fragmentation of free space, which is exacerbated by the resource-intensive process of segment cleaning, especially in heavily-loaded systems where it competes with client requests.
Innovation Solution
Just-in-time segment cleaning predicts and prepares segments targeted by future write operations, minimizing resource costs by cleaning only those segments that will be used soon and applying a cost analysis to determine the most efficient cleaning approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If segment cleaning is performed to alleviate fragmentation of free space, then storage system performance is improved, but resource consumption increases and competes with client requests
Solution Approach 1:
The system performs segment cleaning in advance of when it is strictly needed, based on prediction of future write operations. By proactively cleaning segments that are likely to be targeted by future writes, the system prepares storage space beforehand, reducing the need for urgent cleaning operations that would compete with client requests for resources.
Solution Approach 2:
Instead of performing complete segment cleaning of all storage segments, the system applies partial cleaning only to specific segments predicted to be targeted by future write operations. This selective approach reduces overall resource consumption while still achieving the performance benefit of reduced fragmentation in the critical areas where writes will occur.
2Stability of the object's composition
If segment cleaning is performed on all segments, then fragmentation is fully alleviated, but resource expenditure increases significantly
Solution Approach 1:
The system applies different cleaning strategies to different storage segments based on their predicted usage. Segments predicted to be targeted by future write operations receive cleaning attention, while segments not expected to be used remain uncleared. This localized approach addresses fragmentation only where it matters for future performance, rather than uniformly across all segments.
Solution Approach 2:
The storage system is divided into multiple segments, and the cleaning operation is segmented accordingly. Rather than treating all storage as a single unit requiring uniform cleaning, the system identifies and cleans only the specific segments that will be targeted by future write operations, isolating the cleaning effort to discrete, manageable portions where it will have the greatest impact.
3Use of energy by moving object
If segment cleaning is delayed to reduce resource competition, then resource availability for client requests is maintained, but fragmentation accumulates and degrades performance
Solution Approach 1:
The system performs segment cleaning in advance of future write operations by predicting which segments will be targeted. This preliminary cleaning ensures that when writes occur, the segments are already prepared and free of fragmentation, eliminating the need to delay writes waiting for cleaning or to perform cleaning during write operations.
Solution Approach 2:
The system uses feedback from predicted future write operations to guide cleaning decisions. By analyzing what writes are likely to occur and where they will target segments, the system adjusts its cleaning schedule and scope accordingly, ensuring cleaning resources are allocated to the segments that will actually benefit from them.
Data Source
AI summary
Just-in-time segment cleaning obtains the location of blocks of a storage system predicted to be targeted by a future write operation and cleans the blocks targeted for the future write operation prior to the occurrence of the future write operation. In one aspect, just-in-time segment cleaning is performed in proportion to previous user-initiated write operations. In another aspect, just-in-time segment cleaning applies a cost analysis determination to minimize the work of cleaning.