I/O Driven Rate Adaptation for Storage Bottleneck Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data transfer mechanisms fail to adaptively adjust their rate in response to storage device bottlenecks at the receiver-end, leading to high latency and packet losses due to overdriving, which existing network-focused adaptation mechanisms do not address.
Innovation Solution
An I/O driven, feedback-based rate control mechanism that utilizes a ring buffer to measure storage I/O bottlenecks, employing both long and short time-scale rate adaptation to adjust transfer rates, ensuring the storage device can keep up with data transfer speeds and prevent buffer overflow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If network-focused adaptation mechanisms are used to utilize link capacities, then network throughput is improved, but storage device bottlenecks at the receiver-end are not addressed, leading to overdriving, high latency and packet losses
Solution Approach 1:
The patent implements feedback-based rate control where the receiver monitors storage device performance and sends feedback signals to the sender to adjust the data transfer rate. This closed-loop feedback mechanism allows the system to adapt to storage bottlenecks dynamically, preventing overdriving while maintaining high network throughput utilization.
Solution Approach 2:
The patent introduces dynamic rate adaptation that adjusts the data transfer rate in real-time based on storage device capacity. The system transitions from static rate settings to dynamic adjustment, allowing the transfer rate to respond to changing storage conditions and prevent packet losses while maximizing throughput.
2Productivity
If data is written directly to storage device without buffering, then storage I/O capacity is fully utilized, but buffer overflow occurs when storage device cannot keep up with transfer speed
Solution Approach 1:
The patent introduces an intermediary buffer between the network receiver and storage device. This buffer acts as a mediator that decouples the network data arrival from storage write operations, allowing the system to maintain high storage I/O utilization while preventing data loss when storage device cannot keep up with transfer speeds.
Solution Approach 2:
The patent employs buffering as a cushioning mechanism that absorbs fluctuations in data flow. The buffer provides a cushion against storage device performance variations, preventing overflow conditions before they occur while maintaining efficient storage I/O utilization through controlled write operations.
3Reliability
If transfer rate is reduced to match storage device speed, then packet losses are prevented, but network throughput is underutilized
Solution Approach 1:
The patent implements dynamic rate adjustment that allows the transfer rate to adapt to storage device capacity in real-time. Rather than using a fixed reduced rate, the system dynamically optimizes the transfer rate to match storage performance, preventing packet losses while maximizing network throughput utilization when storage capacity allows.
Solution Approach 2:
The patent changes the transfer rate parameter dynamically based on storage device performance metrics. By adjusting this key parameter in response to storage capacity variations, the system prevents packet losses during storage bottlenecks while maintaining high throughput during periods of available storage capacity.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
Described is an I/O driven, feedback based rate control mechanism that allows a transfer session to self-adjust its transfer rate in response to the speed of the storage device at the destination. The described system may operate at the application layer, which makes cross-platform deployment an easier task