Computing Object Snapshot Batching for Lower Backup Cluster IOPS
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management systems face inefficiencies in managing computing object snapshots due to the high resource consumption of frequent consolidation jobs for expired snapshots, leading to significant disk IOPS and CPU cycles.
Innovation Solution
The system defers consolidation jobs until multiple chained incremental snapshots can be consolidated together, using heuristic thresholds to reduce redundant read and write operations by merging expired snapshots with unexpired ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If frequent consolidation jobs are performed for expired snapshots, then data management reliability is improved, but disk IOPS and CPU cycles increase significantly
Solution Approach 1:
The patent combines multiple expired incremental snapshots into a single consolidated snapshot, merging their data blocks together. This reduces the number of separate consolidation jobs needed, thereby decreasing disk IOPS and CPU cycles while maintaining data integrity and reliability.
Solution Approach 2:
The consolidated snapshot serves multiple functions: it preserves expired data for compliance and recovery purposes, reduces storage space requirements, and decreases system resource consumption. This multi-functional approach resolves the contradiction by achieving reliability through consolidation while improving productivity through reduced resource usage.
2Reliability
If individual expired snapshots are consolidated separately, then data integrity is maintained, but redundant read and write operations increase
Solution Approach 1:
Multiple expired incremental snapshots are merged into a single consolidated snapshot in one operation. This eliminates redundant read and write operations that would occur if each snapshot were consolidated separately, while still maintaining data integrity through the consolidation process.
3Quantity of substance
If consolidation jobs are performed immediately upon snapshot expiration, then storage space is reclaimed promptly, but system resource consumption increases
Solution Approach 1:
The system performs preliminary identification and grouping of expired snapshots before consolidation. By preparing the consolidation list in advance and batching multiple snapshots together, the system reclaims storage space efficiently while minimizing peak resource consumption compared to immediate individual consolidation.
Data Source
AI summary
Methods, systems, and devices for data management are described. A backup cluster may store incremental or base snapshots of computing objects. When a snapshot expires, a data management system (DMS) that manages the backup cluster may in some cases merge or consolidate the expired snapshot with a non-expired snapshot to create a new merged snapshot. In some cases, however, consolidation may be deferred until a chain of multiple expired snapshots satisfies one or more heuristic thresholds, to conserve resources. Example heuristic thresholds may be a length of the chain, an amount of space reclaimable by consolidating the snapshots, an age of the expired snapshots in the chain of incremental snapshots, or an amount of free space on the backup cluster.