Storage Replication Token Pool for Latency Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage system replication processes negatively impact client I/O requests due to the need for continuous access to storage arrays during data transfer, leading to reduced availability and increased latency.
Innovation Solution
A resource pool of tokens is dynamically managed to authorize and optimize the replication engine's transfer of replicated information, increasing tokens when client traffic is low and latency is within thresholds, and decreasing them when latency and traffic levels indicate high demand, thereby balancing data transfer with client request processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If continuous data transfer from source storage to destination storage is performed, then data replication reliability is improved, but client I/O request processing performance deteriorates due to storage array access conflicts
Solution Approach 1:
The system dynamically adjusts the replication rate based on real-time monitoring of client I/O workload and storage array performance metrics. When client traffic increases or latency thresholds are exceeded, the system automatically reduces or pauses replication transfers. When client workload decreases, replication resumes at higher rates. This dynamic adaptation resolves the contradiction by making replication reliability conditional on system capacity rather than constant.
Solution Approach 2:
The system implements periodic evaluation of client I/O performance metrics and storage array availability during the replication process. Instead of continuous uninterrupted transfer, replication occurs in periodic bursts separated by evaluation intervals. This allows the system to periodically assess whether client I/O performance remains acceptable and adjust replication accordingly, resolving the conflict between continuous replication and continuous client service.
2Reliability
If storage arrays are accessed during replication transfer, then data redundancy is improved, but storage array availability for client operations deteriorates
Solution Approach 1:
The system applies different access priorities to different data blocks within the storage array. Client I/O requests receive high priority when workload is heavy, while replication transfers receive low priority or are suspended. When client workload is light, replication transfers can access storage arrays with higher priority. This local quality differentiation in access priority resolves the contradiction by allowing both client operations and replication to access storage arrays at different times with appropriate priority levels.
Solution Approach 2:
The system introduces a replication controller as an intermediary between the replication engine and the storage arrays. This mediator monitors client I/O performance metrics and mediates access requests from the replication engine to the storage arrays. When client performance degrades, the mediator blocks or throttles replication access requests. When client performance is acceptable, the mediator permits replication access. This intermediary resolves the contradiction by filtering and controlling access conflicts before they reach the storage arrays.
3Speed
If replication transfer rate is increased, then data synchronization speed is improved, but system latency for client requests increases
Solution Approach 1:
The system dynamically adjusts replication transfer rate based on real-time monitoring of client request latency metrics. When latency approaches or exceeds predefined thresholds, the system automatically reduces replication transfer rate or pauses transfers. When latency remains below thresholds and system capacity is available, replication proceeds at maximum rate. This dynamic rate adjustment resolves the contradiction by making synchronization speed adaptive rather than fixed.
Solution Approach 2:
The system implements feedback control where client request latency measurements feed back to the replication controller, which then adjusts replication transfer rate accordingly. The feedback loop continuously monitors latency metrics and closes the control loop by modifying replication behavior. This feedback mechanism resolves the contradiction by using actual system performance data to automatically balance synchronization speed against client request latency.
Data Source
AI summary
Machine implemented method and system for transferring replicated information from a first storage location managed by a storage operating system at a first storage system node and accessible to a client computing system to a second storage location managed by a second storage system node are provided. A resource pool having a plurality of tokens is maintained for authorizing a replication engine to transfer replicated information from the first storage location to the second storage location. The number of available tokens is increased when traffic due to client requests for accessing the first storage location is less than a first threshold level. The number of available tokens is decreased for reducing transfer of information via the replication engine, when latency in responding to the client requests reaches a second threshold value and the traffic due to client requests reaches the first threshold value.