Replication I/O Queue Prioritization for Lower-Latency Data Repositories
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in managing data replication and storage operations, particularly in flash-based storage systems, due to redundant processes performed by both higher and lower level controllers, leading to increased latency and reduced reliability.
Innovation Solution
Implementing a direct-mapped flash storage system where higher level processes initiate and control data operations directly on flash drives without address translation by lower level controllers, utilizing non-volatile RAM for buffering and offloading device management tasks from storage drives, and employing erasure coding and mirroring schemes for redundancy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If both higher level controllers and lower level controllers perform data replication operations, then data redundancy is achieved, but system latency increases and reliability decreases due to redundant processes
Solution Approach 1:
The patent extracts the data replication function from the lower level controller and assigns it exclusively to the higher level controller. This eliminates redundant replication operations by having only one controller perform the replication task, thereby reducing latency while maintaining data redundancy and integrity through the replicated datasets stored at different locations.
2Ease of operation
If address translation is performed by lower level controllers, then data access is facilitated, but operation latency increases due to additional translation steps
Solution Approach 1:
The patent merges the address translation function with the higher level controller that also performs data replication. By combining these functions at the higher level, the system eliminates the need for separate address translation steps at the lower level, reducing access latency while maintaining ease of operation through centralized control.
3Adaptability or versatility
If device management tasks are performed by storage drives themselves, then device autonomy is improved, but overall system productivity decreases due to wasted drive capacity
Solution Approach 1:
The patent extracts device management tasks from the storage drives and assigns them to the higher level controller. This allows storage drives to be dedicated entirely to data storage operations, maximizing their capacity utilization and productivity, while the higher level controller handles management functions that require coordination across multiple drives.
Data Source
AI summary
Providing Quality of Service (QoS) for replicating datasets including: receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository and the target data repository; adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues for the target data repository; selecting, by the target data repository, one or more queues from the plurality of queues; and servicing an operation from each of the selected one or more queues.