Stride-Based Receive Queue for Network Interface Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data communication methods in high-performance computing, such as InfiniBand, incur substantial overhead and memory footprint due to the need for continuous production and consumption of work queue elements (WQEs) for handling 'push' operations, leading to latency and resource consumption.
Innovation Solution
Implementing a stride-based receive queue model where work items point to buffers divided into fixed-size segments (strides) in memory, allowing data to be written only to the necessary strides, with remaining strides used for subsequent packets, and dynamically managing buffer allocation based on traffic characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional WQE-based receive queue operations are used for handling push operations, then data can be received and written to memory, but memory footprint and overhead increase substantially due to continuous production and consumption of WQEs
Solution Approach 1:
The receive queue is segmented into multiple strides, each stride representing a fixed-size memory region. Instead of using traditional WQE structures for each packet, the system divides the receive buffer into discrete strides that can be incrementally consumed. This segmentation eliminates the overhead of continuous WQE production and consumption while maintaining efficient data reception through stride-based scattering.
2Loss of time
If conventional WQE-based receive queue operations are used, then data reception can be handled, but latency increases due to continuous production and consumption operations
Solution Approach 1:
The receive queue is pre-configured with strides that represent predetermined memory regions. Instead of creating WQEs dynamically for each incoming packet, the system prepares the stride structure in advance. When data arrives, the NIC simply scatters packets into the next available stride without requiring complex WQE production and consumption operations, thereby reducing latency and operational complexity.
3Quantity of substance
If buffers are allocated for each WQE in the receive queue, then data can be written to specific memory locations, but resource consumption increases due to continuous buffer allocation and deallocation
Solution Approach 1:
The stride-based receive queue creates a universal buffer management structure where strides serve multiple purposes: they represent both memory regions and receive queue entries simultaneously. Each stride can be used to receive data from any source without requiring separate buffer allocations, eliminating the need for continuous buffer allocation and deallocation while maintaining flexibility through the stride pointer mechanism.
Data Source
AI summary
A method for communication includes posting in a queue a sequence of work items pointing to buffer consisting of multiple strides of a common, fixed size in a memory. A NIC receives data packets from a network containing data to be pushed to the memory. The NIC reads from the queue a first work item pointing to a first buffer and writes data from a first packet to a first number of the strides in the first buffer without consuming all of the strides in the first buffer. The NIC then writes at least a part of the data from a second packet to the remaining strides in the first buffer. When all the strides in the first buffer have been consumed, the NIC reads from the queue a second work item pointing to a second buffer, and writes further data to the strides in the second buffer.