Storage Replication Copy Prioritization via Dynamic Concurrency Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in efficiently managing concurrent copy operations for data replication, particularly in ensuring data freshness and preventing Recovery Point Objective (RPO) violations, which can lead to resource-intensive replication activities impacting ongoing host IO operations.
Innovation Solution
A method that triggers synchronization activities on storage objects based on a replication policy, prioritizing copy requests in a queue according to attributes such as time-to-Recovery-Point-Objective-violation, time-to-alert, and generation time, ensuring that copy operations are executed efficiently even when the number of concurrent copy operations reaches a maximum, by using rules to manage concurrency and prevent RPO violations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of concurrent copy operations is increased to improve data replication speed, then productivity is improved, but device complexity and resource consumption increase
Solution Approach 1:
The system dynamically adjusts the number of concurrent copy operations based on current system load and queue depth. The copy manager monitors the number of active copy operations and dynamically creates or suspends copy tasks to maintain optimal throughput while preventing resource exhaustion. This dynamic adaptation resolves the contradiction by allowing high productivity when resources are available while reducing complexity when resources are constrained.
Solution Approach 2:
A copy manager component is introduced as an intermediary between the synchronization activity trigger and the actual copy operations. This mediator manages the queue of copy requests, controls the number of concurrent operations, and prioritizes copy tasks based on RPO violations and alert status. By introducing this intermediary layer, the system can handle high volumes of replication requests without directly exposing the complexity of concurrent operation management to the rest of the system.
2Reliability
If more copy operations are executed concurrently to prevent RPO violations, then reliability is improved, but use of energy increases
Solution Approach 1:
The system implements feedback mechanisms that monitor RPO status and alert conditions for each storage object. When an RPO violation is detected or an alert is generated, the copy manager receives feedback and prioritizes copy operations for the affected objects. This feedback-driven approach ensures that copy operations are executed with high reliability when needed (to prevent or correct RPO violations) while avoiding unnecessary energy consumption during normal operating conditions.
Solution Approach 2:
The system changes operational parameters dynamically based on system state. The maximum number of concurrent copy operations is not fixed but is adjusted based on current queue depth, system load, and priority conditions. When RPO violations occur, the system parameter changes to allow more aggressive copy operation execution. During normal conditions, the system reduces concurrent operations to minimize energy consumption, thus resolving the contradiction between reliability and energy use.
3Reliability
If copy requests are prioritized based on multiple attributes, then reliability is improved, but device complexity increases
Solution Approach 1:
The system applies different priority levels and management strategies to different copy requests based on their local characteristics. Copy requests are evaluated individually for attributes such as RPO violation status, alert condition, and queue depth. Objects in RPO violation state receive highest priority with aggressive copy execution, while objects in normal state follow standard queue processing. This local quality approach ensures high reliability for critical objects without applying complex priority management uniformly across all objects, thus reducing overall system complexity.
Data Source
AI summary
A method, computer program product, and computer system for triggering, by a computing device, a synchronization activity on a storage object. A request may be executed for a copy operation resulting from the synchronization activity to copy the storage object from a source to a destination when a current number of concurrent copy operations is below a maximum number of copy operations. The request for the copy operation resulting from the synchronization activity to copy the storage object from the source to the destination may be placed in a queue when the current number of concurrent copy operations is at least at the maximum number of copy operations. One or more attributes of the request in the queue may be identified. The request for the copy operation may be executed according to a replication policy based upon, at least in part, the one or more attributes of the request in the queue.