Concurrent Remote IO Processing for Storage Replication Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for maintaining consistency between replicas in computer networks with replicated logical production volumes face inefficiencies due to serialized processing of IO commands, leading to increased latency in acknowledging write operations.
Innovation Solution
Implementing concurrent remote IO processing by generating a notification (early IO) that enables pre-processing on both primary and secondary storage arrays, allowing them to prepare for incoming data updates in parallel, thereby reducing the time elapsed between receiving a write command and sending an acknowledgement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If serialized processing of IO commands is used to maintain consistency between replicas, then data consistency is ensured, but processing latency increases
Solution Approach 1:
The patent applies preliminary action by implementing pre-processing steps before the actual data write operation. The primary storage array sends a notification to the secondary storage array containing metadata about the impending IO operation, allowing the secondary array to prepare in advance. This includes allocating buffer space, setting up data structures, and pre-establishing write paths, so that when the actual data arrives, the secondary array can immediately write without preparation delays.
Solution Approach 2:
The patent segments the IO processing into distinct phases: a notification phase where metadata is exchanged and pre-processing occurs, and a data write phase where actual data transfer happens. This segmentation allows the secondary storage array to perform preparatory operations independently and in parallel with the primary array's data preparation, reducing the critical path latency while maintaining consistency through the structured two-phase approach.
2Reliability
If pre-processing is performed sequentially on primary and secondary storage arrays, then consistency is maintained, but processing speed decreases
Solution Approach 1:
The patent merges the pre-processing operations of the primary and secondary storage arrays into a concurrent execution model. Both arrays receive notifications and perform their respective pre-processing tasks (metadata identification, memory location identification, locking) simultaneously rather than sequentially. This merging of operations maintains consistency through the notification mechanism while doubling the effective pre-processing throughput.
Solution Approach 2:
The patent ensures continuity of useful action by having both primary and secondary storage arrays engaged in productive pre-processing activities simultaneously. Rather than one array waiting for the other to complete preparation, both arrays continuously perform useful work in parallel - the primary array preparing data for transmission while the secondary array prepares reception buffers and data structures, maximizing resource utilization and reducing idle time.
Data Source
AI summary
In a data storage system in which a first storage array and a second storage array maintain first and second replicas of a production volume, the first storage array is responsive to a write command from a host to send a notification to the second storage array indicating that the replicated production volume will be updated. The notification has information that enables the second storage array to implement pre-processing steps to prepare for subsequent receipt of data associated with the write command. Both storage arrays implement the pre-processing steps at least partly concurrently. When the data associated with the write command is subsequently received, the first storage array writes the data to cache and then sends a copy of the data to the second storage array, i.e. in series. The second storage array then writes the data to cache. Elapsed time between receipt of the write command and returning an acknowledgment to the host may be improved by concurrent pre-processing.