Cloud Gateway Data Rate Control via SCSI Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems face challenges in managing data rate between storage arrays and cloud storage, particularly due to bandwidth and buffering capacity limitations, which can lead to performance gradients and potential data intake overloads, affecting service level objectives (SLOs) and user experience.
Innovation Solution
A cloud gateway with a feedback program is implemented to control data rate between the storage array and cloud storage by adjusting data streams based on available bandwidth and buffering capacity, using signals such as SCSI VU commands to reduce or pause data transmission, ensuring optimal data handling and preventing service disruptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data transmission rate is increased to improve productivity, then data intake speed to cloud storage is improved, but bandwidth and buffering capacity limitations cause performance gradients and service disruptions
Solution Approach 1:
The cloud gateway implements a feedback mechanism that continuously monitors cloud storage performance and sends control signals back to the storage array. This feedback loop enables dynamic adjustment of data transmission rates based on real-time cloud storage capacity and performance conditions, preventing overloads while maintaining optimal throughput.
Solution Approach 2:
The system dynamically adjusts the data transmission rate between the storage array and cloud gateway based on real-time conditions. The cloud gateway modifies data stream characteristics (rate, timeouts, pausing) in response to changing cloud storage availability, transforming a static transmission system into an adaptive one that responds to performance gradients.
2Productivity
If data transmission rate is increased to improve productivity, then data movement efficiency is improved, but bandwidth limitations cause data intake overloads
Solution Approach 1:
The cloud gateway serves as an intermediary device between the storage array and cloud storage. It mediates the data transmission by buffering data streams and controlling the data rate based on cloud storage availability, preventing direct overload of the cloud storage system while maintaining efficient data movement.
Solution Approach 2:
The system uses partial action by transmitting data at controlled rates rather than maximum capacity. The cloud gateway selectively manages data streams, adjusting transmission levels to match cloud storage capacity, preventing excessive data intake that would cause overloads while still moving data efficiently.
3Reliability
If buffering capacity is increased to handle data rate variations, then service level objective compliance is improved, but device complexity increases
Solution Approach 1:
The cloud gateway performs self-service by autonomously monitoring its own buffering capacity and cloud storage performance, then automatically adjusting data transmission rates accordingly. This self-regulating mechanism eliminates the need for external complex control systems or manual intervention to maintain service level objectives.
Solution Approach 2:
The system changes operational parameters (data rate, timeouts, stream pausing) rather than physically increasing buffering infrastructure. The cloud gateway dynamically modifies transmission parameters based on available buffering capacity, achieving reliable service level compliance without adding complex hardware buffering resources.
Data Source
AI summary
In a storage system in which a storage array accesses cloud storage via a cloud gateway, the cloud gateway provides signals to control data rate between the storage array and the cloud. The signals may include commands to reduce or stop all volume streams or a select subset of volume streams. Data rate may be controlled on the basis of available buffering capacity of the cloud gateway and available bandwidth between the cloud gateway and cloud storage. The signals may be SCSI VU commands, where the cloud gateway is a SCSI initiator and the storage array is a SCSI target.