Toggling Storage Replication Modes for Latency and Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data replication systems, such as synchronous and asynchronous RDF systems, face challenges with high latency and data consistency issues, especially over long distances, and require significant overhead in managing host control and cycles, leading to potential data loss during failures.
Innovation Solution
A system that toggles between storage configurations to enable asynchronous data transfer between storage groups, using maps to associate synchronous and asynchronous data, and maintains data integrity by transferring pending data and establishing relationships between storage groups to ensure consistency and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous RDF transfer is used, then data consistency between primary and secondary storage devices is ensured, but write latency increases due to waiting for acknowledgment
Solution Approach 1:
The system dynamically switches between synchronous and asynchronous replication modes based on operational requirements. The primary storage device can toggle between providing synchronous data transfer (for critical consistency) and asynchronous data transfer (for lower latency), allowing flexible adaptation to different workload conditions and failure scenarios
Solution Approach 2:
The system changes the replication parameter from synchronous to asynchronous mode by modifying the acknowledgment handling mechanism. In asynchronous mode, the primary storage device sends data to the secondary device without waiting for acknowledgment, fundamentally changing the transfer parameter to reduce latency while maintaining the ability to switch back when needed
2Loss of time
If semi-synchronous RDF transfer is used, then write latency is reduced by immediate acknowledgment, but data integrity is compromised due to potential data loss in failures
Solution Approach 1:
The system performs preliminary actions by maintaining multiple storage groups (first, second, and third storage groups) with predefined relationships. Before a failure occurs, the system has already established cascaded relationships where the second storage group receives data from the first, and the third storage group receives data from the second, ensuring data integrity is preserved through pre-configured redundancy
Solution Approach 2:
The second storage group acts as an intermediary in the cascaded replication chain. It receives data from the first storage group and forwards it to the third storage group, mediating the data transfer to ensure that even if the direct path fails, data integrity is maintained through the intermediate node that has already processed and acknowledged the data
3Loss of time
If asynchronous RDF transfer is used, then write latency is minimized with immediate acknowledgment, but data consistency cannot be guaranteed between storage devices
Solution Approach 1:
The system segments the replication process into multiple independent storage groups with distinct roles. The first storage group handles primary data, the second storage group handles intermediate replication, and the third storage group handles final backup. This segmentation allows each segment to operate asynchronously while maintaining overall system consistency through their structured relationships
Solution Approach 2:
The system implements periodic toggling between synchronous and asynchronous modes for different storage groups. The primary storage device periodically switches between providing synchronous data transfer to the second storage group and asynchronous data transfer to the third storage group, ensuring that data consistency is maintained periodically while minimizing overall latency
4Reliability
If multiple storage groups are managed with toggling configurations, then data consistency and latency are optimized, but system complexity increases
Solution Approach 1:
The storage device is designed with multi-functionality to handle multiple storage groups and replication modes within a single device architecture. The RDF adapter can universally manage synchronous and asynchronous transfers to multiple different storage groups, eliminating the need for separate dedicated systems for each function and reducing overall system complexity despite the enhanced capabilities
Data Source
AI summary
Toggling between a first storage configuration and a second storage configuration includes disabling asynchronous data transfer to a third storage group from one of the first storage group and the second storage group and enabling asynchronous data transfer to the third storage group from an other one of the first storage group and second storage group. Toggling between a first storage configuration and a second storage configuration may also include, prior to disabling asynchronous data transfer to the third storage group, transferring pending data to the third storage group. Toggling between a first storage configuration and a second storage configuration may also include, after enabling asynchronous data transfer to the third storage group, transferring data to the third storage group that had been pending prior to disabling asynchronous data transfer to the third storage group.