NIC Request Aggregation for Small Remote Memory Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current supercomputer performance is hindered by the need for numerous small remote memory operations, which incur significant network packet overhead, leading to inefficiencies in commodity systems and high costs in proprietary systems, with software-based request aggregation consuming CPU resources.
Innovation Solution
A system that aggregates small remote memory-operation requests into larger messages using network interface cards (NICs) with asynchronous buffering, sorting requests into queues based on destination NICs and cores, reducing the need for individual transmissions and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If numerous small remote memory operations are used, then memory access flexibility is improved, but network packet overhead increases significantly
Solution Approach 1:
The patent merges multiple small remote memory operations into larger aggregated requests. The NIC buffers and aggregates multiple memory requests from multiple cores into fewer, larger packets that can be transmitted together over the network, reducing the total number of packets and associated overhead while maintaining memory access flexibility through structured buffering and sorting mechanisms
2Loss of energy
If software-based request aggregation is used, then network overhead is reduced, but CPU resources are consumed
Solution Approach 1:
The patent implements self-service by moving the request aggregation functionality from software to hardware. The NIC includes dedicated buffering and sorting hardware that automatically aggregates memory requests without requiring CPU intervention. The hardware sorts requests by destination core and buffers them in dedicated queues, enabling autonomous aggregation that reduces network overhead while freeing CPU resources
Solution Approach 2:
The patent replaces the software-based mechanical system with a hardware-based system. Instead of using software processes to aggregate requests, the invention uses dedicated hardware circuits within the NIC that perform buffering and sorting operations. This substitution of mechanical (software) with physical (hardware) mechanisms enables more efficient, lower-overhead aggregation without CPU resource consumption
3Speed
If link bandwidth is increased, then data transmission speed is improved, but sustaining line rate on small packets becomes more difficult and expensive
Solution Approach 1:
The patent merges multiple small packets into fewer, larger packets at the NIC level. By buffering and aggregating multiple memory requests into larger transmission units, the system reduces the number of individual packets that need to be sustained at high line rates, thereby reducing the difficulty and cost of maintaining high bandwidth on small packets while preserving overall data transmission speed
Data Source
AI summary
A network interface card (NIC) receives a stream of commands, a respective command comprising memory-operation requests, each request associated with a destination NIC. The NIC buffers asynchronously the requests into queues based on the destination NIC, each queue specific to a corresponding destination NIC. When first queue requests reach a threshold, the NIC aggregates the first queue requests into a first packet and sends the first packet to the destination NIC. The NIC receives a plurality of packets, a second packet comprising memory-operation requests, each request associated with a same destination NIC and a destination core. The NIC buffers asynchronously the requests of the second packet into queues based on the destination core, each queue specific to a corresponding destination core. When second queue requests reach the threshold, the NIC aggregates the second queue requests into a third packet and sends the third packet to the destination core.