Network Flow Shaper for Incast Buffer Overflow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The 'incast' problem arises in information handling systems where data transfer bottlenecks occur at the receiving edge switch or end device due to buffer overload, leading to packet drops, especially in scenarios with large data sets and parallel data requests from multiple sources.
Innovation Solution
A network device with a control unit that inspects data packets for indicators of incast communication patterns and implements a data flow shaper by distributing buffer load across multiple devices, using a central controller to manage network traffic and adjust data transfer rates and bandwidth to prevent buffer overflow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transferred in parallel from multiple sources to a single destination, then data transfer speed is improved, but buffer overload occurs at the receiving edge switch causing packet drops
Solution Approach 1:
The patent segments the data flow by introducing intermediate buffer nodes between data sources and the receiving switch. Each intermediate node buffers and forwards packets individually, dividing the large parallel data stream into smaller manageable units that can be processed without overwhelming the receiving switch buffer.
Solution Approach 2:
The patent introduces intermediate buffer nodes as mediators in the data path. These intermediate nodes act as buffers that receive packets from multiple sources and forward them to the destination, preventing direct buffer overload at the receiving switch while maintaining parallel data transfer capability.
2Reliability
If buffer capacity at the receiving edge switch is increased to prevent packet drops, then packet delivery reliability is improved, but device complexity and cost increase
Solution Approach 1:
Instead of increasing the buffer capacity at a single receiving switch, the patent segments the buffering function across multiple intermediate nodes distributed in the network. Each intermediate node maintains a small buffer, collectively providing sufficient buffer capacity without requiring any single switch to have excessive buffer resources.
Solution Approach 2:
The patent moves the buffer capacity from a single dimension (one switch's buffer) to multiple dimensions (distributed across several intermediate nodes). This spatial distribution of buffering capacity allows the system to handle larger data volumes without requiring any single device to have proportionally larger buffer resources.
3Reliability
If a data flow shaper is implemented to distribute buffer load, then packet drops are reduced, but network device complexity increases
Solution Approach 1:
The patent implements self-service by having intermediate buffer nodes automatically buffer and forward packets based on local conditions without requiring complex centralized control. Each intermediate node independently manages its own buffer, reducing the need for sophisticated network-wide coordination while still achieving load distribution.
Solution Approach 2:
The patent applies preliminary action by having intermediate nodes buffer packets before they reach the receiving switch. This advance buffering prevents the receiving switch from being overwhelmed by sudden data arrivals, smoothing out traffic patterns proactively rather than reactively.
Data Source
AI summary
A system and method includes a network device comprising a control unit, a first port coupled to the control unit and configured to couple the network device to a first device using a first network link. The control unit is configured to receive a data packet from the first device on the first port, inspect the data packet for an indicator of an incast communication pattern, and implement a data flow shaper on a network when the indicator is present in the data packet.


