Lazy Snapshot Delta Log Architecture for Storage Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Delta logging in storage clusters leads to massive duplication of data, causing infrastructure overload and performance degradation due to the need to log entire metadata pages during snapshot creation, despite only actual data changes being logged elsewhere.
Innovation Solution
Implement a 'lazy snapshot' approach with a delta log-based architecture that logs only delta tuples defining snapshot relationships, allowing the actual snapshot metadata page to be built during the destage flow if the snapshot is long-lived, and using one of two destaging techniques to manage dependencies between parent and child leaves.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If entire metadata pages are copied during snapshot creation, then snapshot consistency is ensured, but delta log infrastructure becomes overloaded and NVRAM bandwidth is wasted
Solution Approach 1:
The patent extracts only the essential snapshot relationship information (parent-child leaf associations) from the full metadata pages and stores this in the delta log. The complete metadata pages are not copied during snapshot creation, but only the necessary relational metadata is logged, reducing delta log overhead while maintaining snapshot consistency through the preserved parent-child relationships.
Solution Approach 2:
The patent performs preliminary action by pre-establishing the parent-child relationship metadata in the delta log during snapshot creation, but delays the actual copying of full metadata pages until destage operations. This allows snapshot relationships to be recorded without immediately incurring the full cost of metadata page duplication, improving delta log performance while ensuring consistency is maintained when pages are eventually copied.
2Productivity
If delta logging is applied to snapshot creation, then logging overhead is reduced, but dependency management between parent and child leaves becomes complex
Solution Approach 1:
The patent segments the metadata management into two distinct parts: (1) lightweight delta log entries that record only parent-child relationship metadata, and (2) full metadata pages that are managed separately during destage operations. This segmentation allows efficient logging of snapshot relationships without immediately copying full pages, while the destage process handles the complex dependency resolution by rebuilding child leaves from parent pages when needed.
Solution Approach 2:
The patent introduces the destage process as an intermediary mechanism that resolves the complexity of dependency management. Instead of handling all dependency resolution during snapshot creation, the destage process acts as a mediator that rebuilds child leaf pages from parent pages when dependencies are detected, simplifying the snapshot creation process while maintaining data integrity through controlled dependency resolution.
3Reliability
If full metadata pages are logged in delta log, then data integrity is maintained, but NVRAM bandwidth is wasted and performance degrades
Solution Approach 1:
The patent extracts only the essential relational metadata (parent-child leaf associations) from the full metadata pages and stores this extracted information in the delta log. This extraction approach maintains data integrity by preserving the critical relationships needed for snapshot reconstruction, while avoiding the waste of NVRAM bandwidth that would result from logging complete metadata pages that contain redundant information.
Data Source
AI summary
A method, computer program product, and computer system for creating, by a computing device, a child leaf of a snapshot associated with a parent leaf of a volume. At least one entry associated with one of the parent leaf and the child leaf may be added to a delta log. Only delta changes are written to the child leaf. One of a first destaging technique and a second destaging technique may be used for the child leaf.