Dynamic Buffer Allocation for Storage Access Request Flow Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems face issues with packet drops due to RX ring buffer congestion and inefficient memory allocation, leading to increased latency and decreased throughput in managing storage access requests from multiple client computers.
Innovation Solution
A system and method that dynamically allocate buffer memory space based on client-specific parameters, using a processor and Network Interface Controller (NIC) to manage RX ring buffer space, prevent overflow, and optimize memory usage by dividing RAM into portions for incoming data and pending storage, allowing for real-time adjustment according to connection needs and Quality of Service parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If static memory allocation is used for each client connection, then memory management is simple, but memory efficiency decreases and buffer overflow occurs
Solution Approach 1:
The patent implements dynamic memory allocation where the buffer memory space is adjusted in real-time based on client-specific parameters such as connection status, data injection rate, and quality of service requirements. This allows the system to optimize memory usage for each client connection dynamically, preventing buffer overflow while maximizing throughput.
Solution Approach 2:
The system changes memory allocation parameters dynamically based on observed client behavior and network conditions. By monitoring parameters like data injection rate and connection status, the system adjusts buffer sizes accordingly, transitioning from static to adaptive memory management that resolves the contradiction between simplicity and efficiency.
2Reliability
If RX ring buffer size is increased to prevent packet drops, then packet loss decreases, but memory consumption increases and latency increases
Solution Approach 1:
Instead of uniformly increasing buffer size for all connections, the patent applies local quality by allocating buffer space specifically to connections that require it based on their individual characteristics. This targeted approach ensures packet delivery reliability for critical connections without unnecessarily increasing overall memory consumption and latency.
Solution Approach 2:
The buffer allocation is made dynamic rather than static, allowing the system to adjust buffer sizes in real-time based on actual network conditions and client needs. This prevents permanent over-allocation of memory while ensuring adequate buffering when needed, thus reducing latency while maintaining reliability.
3Productivity
If dynamic memory allocation is implemented based on client parameters, then memory efficiency improves and throughput increases, but system complexity increases
Solution Approach 1:
The system implements self-service by automatically monitoring client connections and dynamically adjusting buffer allocations without requiring external intervention or complex manual configuration. The flow control mechanism serves itself by adapting to changing conditions, which reduces operational complexity despite the dynamic nature of the allocation.
Solution Approach 2:
The patent incorporates feedback mechanisms where the system continuously monitors client behavior, buffer usage, and network conditions, then uses this information to adjust memory allocation dynamically. This closed-loop control achieves high throughput while managing complexity through automated decision-making based on observed system state.
4Adaptability or versatility
If TCP window field is used for flow control, then connection-level flow control is achieved, but it cannot prevent RX buffer overflow and causes static memory waste
Solution Approach 1:
The patent introduces an intermediary flow control mechanism that operates between the TCP layer and the RX buffer management. This intermediate layer provides more granular control over buffer allocation, preventing overflow by monitoring actual buffer usage and coordinating with TCP flow control, thus combining the adaptability of TCP with reliable overflow prevention.
Data Source
AI summary
A method and a system for controlling the access of a plurality of client computers to storage media, the system including: a processor, a Random-Access Memory (RAM) device; and a Network Interface Controller (NIC), configured to establish a plurality of connections with the clients. The processor may dynamically allocate a buffer memory space to each connected client computer on the RAM device, and the NIC may be configured to receive at least one storage access request from at least one client, over at least one computer network connection. The RAM device may accumulate data of the at least one storage access request in the buffer allocated to the respective connected client computer, and the processor may be configured, upon completion of the accumulation of data, to propagate the buffered data to at least one storage device of the storage media.