Coalescing Metadata Changes for Virtual Block Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems incur processor and memory-intensive operations and inefficient use of transaction log space due to processing each metadata change individually, even when multiple changes affect the same block, leading to potential data corruption and limited log capacity.
Innovation Solution
Coalescing metadata changes based on the block of storage using a searchable data structure with bitmaps, where each node accumulates changes for a block, allowing for summarized transactions to be written to the transaction log, reducing redundant operations and log usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If each metadata change is recorded as a separate transaction in the transaction log, then the transaction log provides complete recording of all metadata changes, but the storage system performs multiple processor and memory-intensive operations for each metadata change even when multiple changes affect the same block, and the transaction log consumes valuable space
Solution Approach 1:
The patent merges multiple metadata change instructions that affect the same block into a single coalesced transaction. The system identifies metadata changes grouped by their target block and combines them into one transaction entry in the transaction log, rather than creating separate transactions for each change. This reduces the number of processor and memory-intensive operations (cyclic redundancy check, fetch block, modify block, store block) from being invoked once per metadata change to once per coalesced transaction, significantly improving processing efficiency while maintaining complete recording of all metadata changes.
2Reliability
If each metadata change is recorded as a separate transaction in the transaction log, then the transaction log provides complete recording of all metadata changes, but the transaction log consumes valuable space and may run out of space
Solution Approach 1:
The patent merges multiple metadata change instructions that affect the same block into a single coalesced transaction. By combining multiple individual transaction entries into one coalesced transaction, the total number of transactions in the log is reduced. This decreases the space consumption in the transaction log while still maintaining complete and accurate recording of all metadata changes, preventing the log from running out of space during high-load operations.
3Reliability
If each metadata change is recorded as a separate transaction in the transaction log, then the transaction log provides complete recording of all metadata changes, but the system can handle limited load due to log space constraints
Solution Approach 1:
The patent merges multiple metadata change instructions that affect the same block into a single coalesced transaction. This reduction in transaction volume increases the effective capacity of the transaction log, allowing the system to handle higher loads with more concurrent metadata changes. The coalescing mechanism enables the system to maintain complete and accurate recording of all metadata changes even under heavy load conditions, thereby improving system adaptability and load handling capacity.
Data Source
AI summary
A technique for preserving metadata changes in a transaction log involves coalescing metadata changes based on the block of storage in which the metadata to be changed resides. Metadata change information that accompanies a file system command is stored in nodes of a searchable data structure, wherein each node accumulates metadata changes for a respective block of storage. Once all metadata changes are specified in the searchable data structure, or after some threshold number of metadata changes have been stored, the storage processor composes a transaction for each node summarizing the metadata changes and writes the transaction to the transaction log.