Multi-Modal Write Cache for Cloud Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Public cloud data centers face challenges in implementing persistent write caches due to the lack of persistent memory provided by vendors and the high financial cost of persistent storage volumes, which are not ideal for burst write caching and I/O workloads that exceed provisioned performance.
Innovation Solution
A multi-modal write caching technique that utilizes both persistent and non-persistent write caches, where data is written to a persistent cache for base workloads and a non-persistent cache for burst workloads, with data mirrored across different fault domains or availability zones to ensure resilience and cost-effectiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If persistent storage volumes are used to implement write caches, then data persistence and reliability are improved, but financial cost increases due to provisioned performance requirements
Solution Approach 1:
The write cache is segmented into two distinct components: a persistent write cache for base workloads and a non-persistent write cache for burst workloads. This segmentation allows the system to maintain reliability for critical operations while using cost-effective temporary storage for overflow traffic, thereby resolving the contradiction between data persistence and financial cost.
Solution Approach 2:
The system dynamically changes the persistence parameter of the write cache based on workload conditions. During normal operations, data is written to persistent storage ensuring reliability. During burst workloads exceeding provisioned performance, the system switches to non-persistent storage, changing the persistence parameter to optimize cost while maintaining system functionality.
2Productivity
If persistent storage volumes with high provisioned performance are used, then write acknowledgment performance is improved, but financial cost increases
Solution Approach 1:
The write cache performance is made dynamic by switching between persistent and non-persistent storage modes based on real-time workload conditions. During base workloads, persistent storage ensures reliable performance. During burst workloads, the system dynamically switches to non-persistent storage to handle peak traffic without incurring additional provisioned performance costs, thereby optimizing write acknowledgment performance relative to financial cost.
Solution Approach 2:
The storage system is designed with multi-functionality to serve dual purposes: persistent storage for reliability and non-persistent storage for cost-effective burst handling. This universal approach allows the same storage infrastructure to adapt to different performance requirements, improving write acknowledgment performance across varying workload conditions without proportionally increasing financial cost.
3Quantity of substance
If non-persistent storage volumes are used for write caches, then financial cost is reduced, but data loss occurs upon server failures
Solution Approach 1:
The write cache is segmented into persistent and non-persistent components, each serving specific workload types. Base workloads are handled by persistent storage ensuring data safety, while burst workloads use non-persistent storage for cost efficiency. This segmentation mitigates the data loss risk of non-persistent storage by limiting its use to temporary overflow traffic that can be regenerated from persistent sources.
Solution Approach 2:
The persistent write cache acts as an intermediary between the non-persistent write cache and primary storage. During burst workloads, the non-persistent cache handles immediate writes while the persistent cache serves as a safety intermediary, ensuring that critical data is not lost even if non-persistent storage fails. This intermediary relationship reduces data loss risk while maintaining cost benefits.
Data Source
AI summary
Techniques are provided for implementing and managing a multi-modal write cache for a data storage system. For example, a storage control system is configured to perform a write caching method which comprises the storage control system receiving an input/output (I/O) write request from a client application to write data to a primary storage volume, comparing a current I/O workload associated with the client application to an I/O workload threshold, and writing the data of the I/O write request to one of (i) a persistent write cache in a persistent storage volume and (ii) a non-persistent write cache in a non-persistent storage volume, based at least in part on a result of comparing the current I/O workload to the I/O workload threshold.