Network Controller Direct HBM Access for Low-Latency Data Paths
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face challenges in achieving low latency data transfer due to the need for multiple memory copies when moving data between network interfaces and high-bandwidth memory, resulting in significant latency that is not suitable for real-time applications.
Innovation Solution
Implementing a network interface controller (NIC) with logic to directly access high-bandwidth memory (HBM) without intermediate copies to system memory, using descriptor queues and smart filter logic to determine direct data paths, thereby reducing latency through direct DMA transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If data is transferred through system memory using traditional DMA paths, then data transfer reliability is maintained, but data transfer latency increases significantly
Solution Approach 1:
The patent extracts the data transfer path from the traditional system memory route and creates a direct bypass path from the network interface controller to the high bandwidth memory. This extraction eliminates the intermediate copying step through system memory, directly reducing data transfer latency while maintaining transfer reliability through the DMA engine's direct memory access capabilities.
Solution Approach 2:
The patent introduces an intermediary component - the smart filter logic within the network interface controller - that determines whether data should take the direct path to high bandwidth memory or the traditional path through system memory. This intermediary enables dynamic path selection based on data characteristics and application requirements, optimizing latency without compromising reliability.
2Speed
If direct DMA transfers to high bandwidth memory are implemented, then data transfer speed increases, but device complexity increases due to additional logic requirements
Solution Approach 1:
The patent merges the smart filter logic directly into the network interface controller's existing architecture, combining multiple functions (filtering, path determination, and DMA initiation) into a single integrated unit. This merging approach increases data transfer speed by enabling direct access to high bandwidth memory while minimizing the increase in device complexity through functional integration rather than adding separate discrete components.
Solution Approach 2:
The network interface controller is designed with multi-functional capability, serving both traditional system memory access and direct high bandwidth memory access through its integrated smart filter logic. This universality allows the same hardware component to handle multiple data path scenarios, increasing speed for appropriate workloads without proportionally increasing overall device complexity.
3Productivity
If multiple memory copies are performed for data transfer, then data path flexibility is maintained, but productivity decreases due to increased latency
Solution Approach 1:
The patent implements dynamic path selection capability through the smart filter logic, which adapts the data transfer path based on real-time conditions and data characteristics. This dynamic approach enables the system to choose the direct high bandwidth memory path when low latency is critical, thereby improving productivity for real-time processing applications while maintaining the option to use traditional paths when flexibility is more important.
Solution Approach 2:
The smart filter logic performs preliminary determination of the optimal data path before actual data transfer begins. By pre-evaluating data characteristics and selecting the appropriate path in advance, the system avoids unnecessary memory copies and reduces latency, thereby improving productivity for time-sensitive operations while maintaining path flexibility when needed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A network controller (430) is coupled to a memory (435) associated with a hardware accelerator (425) and includes a first port (510) to couple to a host system, wherein the host system comprises system memory (440) and a second port to receive data over a network. The network controller (430) comprises circuitry (605, 610, 615) to determine that the data is to be written directly to the memory (435) instead of to the system memory (440) and write the data to the memory (435) for consumption by the hardware accelerator (425).