Snapshot Queue Pacing with Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Scheduled snapshot operations for large numbers of storage objects can lead to a surge of concurrent snapshot requests, causing system performance issues and latency spikes due to overlapping snapshot rules and simultaneous execution across thousands of objects.
Innovation Solution
A method that tracks and deduplicates snapshot requests using a centralized timer and storage object ID-based hashing to space out requests across a pacing window, ensuring consistent execution times and reducing duplicate requests, while maintaining a reference count to manage PIT copies and user-facing snapshots.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple snapshot rules are executed simultaneously against thousands of storage objects, then snapshot operations can be performed according to each rule's schedule, but system performance degrades due to surge of concurrent snapshot requests and latency spikes
Solution Approach 1:
The patent segments the snapshot request processing by introducing a snapshot queue that divides concurrent requests into sequential batches. Each snapshot rule's requests are submitted to the queue and processed in order, preventing simultaneous execution of multiple snapshot operations on the same storage objects. This segmentation resolves the contradiction by maintaining high productivity through continuous processing while ensuring reliability through controlled, non-overlapping execution.
Solution Approach 2:
The patent applies preliminary action by submitting all snapshot requests to the queue in advance with their desired execution times determined before actual processing. The system pre-calculates execution times based on storage object IDs and pacing windows, then processes requests in predetermined sequence. This allows the system to maintain high productivity by having requests ready while ensuring reliability by executing them in a controlled, non-concurrent manner.
2Adaptability or versatility
If duplicate snapshot requests are processed independently, then each snapshot rule can execute its full set of operations, but redundant PIT copies are created wasting storage resources
Solution Approach 1:
The patent merges duplicate snapshot requests by tracking reference counts for each storage object in the snapshot queue. When multiple snapshot rules target the same storage object, the system combines their requests into a single PIT copy operation. The reference count mechanism ensures that one PIT copy serves multiple snapshot rules, maintaining adaptability by still honoring each rule's requirements while eliminating storage resource waste through deduplication.
Solution Approach 2:
The patent uses copying by creating a single PIT copy that is referenced by multiple snapshot rules rather than creating separate copies for each rule. The reference count table tracks how many snapshots reference each PIT copy, allowing the system to maintain versatile snapshot rule execution while minimizing actual data copying operations, thus preventing storage resource waste.
3Loss of time
If snapshot requests are processed immediately upon submission, then response time is minimized, but latency spikes occur due to thundering herd problem
Solution Approach 1:
The patent implements periodic action by processing snapshot requests in paced intervals rather than immediately. The system determines desired execution times based on storage object IDs and pacing windows, spacing out requests to prevent thundering herd problems. This periodic processing maintains acceptable response times by continuously working through the queue while eliminating latency spikes through controlled, distributed execution timing.
Data Source
AI summary
A method, computer program product, and computer system for submitting, by a computing device, at least one snapshot request of a plurality of snapshots requests into a snapshot queue. How many snapshots of a plurality of snapshots are referencing a given point in time copy may be tracked, wherein the given point in time copy may be mapped to at least a portion of the plurality of snapshots. A desired executing time for the at least one snapshot request in the snapshot queue may be determined. The at least one snapshot request may be dequeued.