Metadata Log Deferring Physical Allocation for Snapshot Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems incur unnecessary overhead in allocating and deallocating metadata pages for short-lived snapshots, leading to resource inefficiencies due to premature allocation of metadata pages before destaging.
Innovation Solution
The proposed solution involves deferring the allocation of new metadata pages until the metadata log entries referencing them are destaged, ensuring that metadata pages are only allocated if the associated snapshot has not been deleted, thereby optimizing resource usage and reducing unnecessary allocation and deallocation overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If metadata pages are allocated in advance for snapshots, then snapshot creation speed is improved, but resource wastage increases due to allocation for deleted snapshots
Solution Approach 1:
The system creates an ALI entry in the metadata log in advance to represent a future metadata page, allowing snapshot creation to proceed immediately without waiting for physical allocation. The actual physical storage allocation is deferred until the destaging process, ensuring the page is only allocated if the snapshot still exists. This resolves the contradiction by performing the preparatory action (ALI creation) immediately while delaying the resource-consuming action (physical allocation) until necessary.
2Ease of operation
If metadata pages are allocated immediately, then snapshot operations are simplified, but overhead increases due to allocation and deallocation operations
Solution Approach 1:
The ALI (abstract logical index) serves as an intermediary between the snapshot operation and the physical metadata page allocation. The ALI entry in the metadata log allows snapshot operations to proceed using a lightweight reference without triggering immediate physical allocation. The actual allocation only occurs during destaging if the snapshot persists, thereby simplifying snapshot operations while avoiding unnecessary allocation overhead.
3Productivity
If physical storage is allocated for new metadata pages early, then page availability is improved, but resource efficiency deteriorates due to premature allocation
Solution Approach 1:
The system performs preliminary action by creating the ALI entry in the metadata log, which makes the metadata page logically available for snapshot operations without committing physical storage resources. The physical storage allocation is deferred to the destaging phase, ensuring resources are only consumed when the snapshot actually needs the page. This approach maintains page availability for operations while preserving resource efficiency.
Data Source
AI summary
In at least one embodiment, processing can include receiving a metadata (MD) structure including MD pages; and performing a MD split operation with respect to a first of the MD pages, wherein said performing the MD split operation includes: generating a first ALI (abstract logical index) representing a new MD page that is unallocated and is a child of the first MD page; and storing an entry in a bucket of an in-memory MD log for the first ALI, wherein the entry denotes a mapping between the first ALI and a corresponding LI (logical index), wherein the entry indicates that the corresponding LI associated with the first ALI is invalid since the first ALI represents a new MD page which is unallocated and not associated with physical storage; and destaging the in-memory MD log, wherein said destaging includes allocating first physical storage for the new MD page.