InfiniBand Shared Memory Flow Control for Buffer Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
InfiniBand architecture faces inefficiencies due to dedicated FIFO queues for each queue pair and virtual lane, leading to wasted memory space and delayed communication transfers between InfiniBand and external systems, as well as challenges in flow control and interleaving different types of traffic.
Innovation Solution
Implementing a shared memory structure for combined queuing of virtual lane and queue pair traffic, using linked lists to manage buffers dynamically and establish thresholds for flow control, allowing queue pairs to share buffers and advertise credits based on available space, thereby optimizing memory usage and reducing delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If dedicated FIFO queues are used for each queue pair and virtual lane, then flow control is simplified, but memory space is wasted and device complexity increases
Solution Approach 1:
The patent combines multiple dedicated FIFO queues into a single shared memory structure that serves multiple queue pairs and virtual lanes. This consolidation allows the system to maintain flow control capabilities while significantly reducing the total memory space required, as buffers can be dynamically allocated and shared among multiple QPs and VFs based on actual traffic needs rather than reserving worst-case capacity for each individual queue.
Solution Approach 2:
The shared memory structure is designed to serve multiple functions simultaneously - it acts as a buffer for multiple queue pairs and virtual lanes, manages flow control for different traffic types, and provides dynamic allocation capabilities. This universal structure replaces the need for separate dedicated queues, reducing overall system complexity while maintaining operational simplicity through unified management.
2Ease of operation
If dedicated FIFO queues are used for each queue pair and virtual lane, then flow control is simplified, but device complexity increases
Solution Approach 1:
The patent merges multiple individual queue management structures into a single shared memory framework. Instead of managing separate FIFO queues for each queue pair and virtual lane, the system uses one unified structure with centralized control logic, thereby reducing device complexity while preserving flow control functionality through standardized management procedures.
Solution Approach 2:
The shared memory structure serves as a universal container that handles multiple queue pairs and virtual lanes through a single management interface. This eliminates the need for duplicate queue management logic across multiple dedicated structures, simplifying the overall device architecture while maintaining the ability to perform flow control operations.
3Reliability
If separate queues are used for each queue pair, then traffic isolation is ensured, but communication transfer efficiency between InfiniBand and external systems decreases
Solution Approach 1:
The patent merges the queue structures for InfiniBand and external system communications into a single shared memory space. This allows traffic from different sources (InfiniBand QPs and external systems) to be interleaved and processed together, eliminating the need for separate queue management and reducing transfer delays while maintaining proper traffic isolation through logical separation within the shared structure.
Solution Approach 2:
The system performs preliminary organization of traffic within the shared memory structure, pre-positioning packets from different sources in an interleaved manner that facilitates efficient subsequent processing. This preliminary arrangement eliminates the need for complex copying operations between separate queues and external system buffers, directly improving transfer efficiency.
4Quantity of substance
If shared storage space is used for virtual lane and queue pair traffic, then memory efficiency is improved, but flow control becomes problematic
Solution Approach 1:
The patent segments the shared memory structure into distinct logical regions or buckets, each associated with specific queue pairs or virtual lanes. This segmentation allows the system to maintain efficient shared storage while implementing flow control by tracking and managing credits at the segment level rather than requiring complex global management of the entire shared space.
Solution Approach 2:
The shared memory structure is organized with local quality characteristics, where different regions have different properties or associations with specific QPs or VFs. This allows flow control to be implemented locally at each segment rather than requiring global coordination, simplifying the overall flow control management while maintaining memory efficiency through sharing.
Data Source
AI summary
A system and method for implementing flow control, at the link and/or transport layers, for InfiniBand receive traffic. A shared memory structure may be used for combined queuing of virtual lane and queue pair traffic. Each virtual lane is allocated a number of memory buffers; a packet is dropped if buffering it would cause its virtual lane to exceed its allocation of buffers. For each active queue pair, a linked list of buffers is maintained in the structure. Each queue pair is dedicated zero or more of its virtual lane's buffers, and may also use a set of buffers shared among multiple queue pairs. Thresholds are established in a queue pair's dedicated set of buffers and/or the shared set of buffers. As each threshold number of buffers is used, a queue pair can advertise fewer message credits. RNR-NAKs are issued when no more buffers are available to a queue pair.