Deferred-Free Block Log for Storage Metadata Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage systems face inefficiencies due to frequent metadata modifications, leading to increased overhead and decreased performance, particularly when updating tracking structures for free blocks, which results in poor user experience and higher costs.
Innovation Solution
Implementing a deferred-free block log system where block identifiers are appended to an active log, sorted in subsets, and then merged into a sorted log, reducing the need for random updates and optimizing input/output operations by leveraging spatial locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If metadata is updated frequently to track free blocks, then storage system can maintain accurate block availability information, but storage system overhead increases and performance decreases
Solution Approach 1:
The patent implements a deferred-free block log system where block identifiers are appended to an active log and then sorted in subsets before being merged into a sorted log. This preliminary sorting action groups block updates by spatial location, enabling batch processing of metadata updates in a structured manner rather than performing random updates immediately, thus reducing overhead while maintaining tracking accuracy
Solution Approach 2:
The patent divides the block log into subsets that are sorted independently before merging them into a sorted log. This segmentation allows the system to process updates in manageable chunks, reducing the complexity of handling frequent metadata modifications and enabling more efficient batch processing of block availability information
2Ease of operation
If random updates are performed to tracking structures, then block availability can be updated immediately, but input/output operations increase significantly
Solution Approach 1:
The system performs preliminary sorting of block identifiers in subsets before merging them into the sorted log. This advance preparation organizes the update data in a structured manner that leverages spatial locality, allowing the system to minimize random I/O operations by processing updates in a more predictable sequence rather than immediately performing all updates at once
Solution Approach 2:
The patent introduces a sorted log as an intermediary structure between the active log and the final metadata updates. This intermediary sorted log serves as a buffer that collects and organizes block identifiers before they are applied to the actual tracking structures, reducing the number of direct random I/O operations to the storage system
Data Source
AI summary
A first plurality of block identifiers is sorted based, at least in part, on a measure of spatial locality. A second plurality of block identifiers is sorted based, at least in part, on the measure of spatial locality. At least the first plurality of block identifiers and the second plurality of block identifiers are incrementally merged into a third plurality of block identifiers based, at least in part, on the measure of spatial locality. A block of data corresponding to metadata associated with a plurality of block identifiers of the third plurality of block identifiers is updated.