Network Node Accelerator Latency Reduction via Queuing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network communication systems experience high processing and sending latencies due to the involvement of host computing and processing units in data transfer, which hinders efficient data processing and transmission in network nodes.
Innovation Solution
A network node architecture that employs a host, an accelerator, and a network adapter, where a work queue and notifications queue are used to manage message processing and transmission, allowing direct memory access (DMA) between the accelerator and network adapter, minimizing the need for host involvement and reducing latency through efficient queuing and control index management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If host computing units are involved in data transfer and processing, then data can be processed with centralized control, but processing latency increases and processing speed decreases
Solution Approach 1:
The patent extracts the host computing units from the critical data path between network adapter and accelerator. The host is removed from the direct data transfer route, allowing data to flow directly between network adapter and accelerator memory without host intervention, thereby reducing processing latency while maintaining centralized control through alternative mechanisms like doorbell messages and completion queues
Solution Approach 2:
The accelerator is enabled to service itself by directly accessing its own memory and processing data independently of the host. The accelerator can autonomously retrieve data from network adapter memory, process it, and store results without requiring host computing unit intervention, thus reducing latency and improving processing speed
2Speed
If data is transferred through host memory and host processing units, then data can be managed with centralized control, but transfer speed decreases and latency increases
Solution Approach 1:
The patent segments the data path into distinct functional regions: network adapter memory, accelerator memory, and host memory. By separating the critical data transfer path from host memory involvement, data can be transferred directly between network adapter and accelerator at high speed, while host memory is used only for control structures and non-critical data, thus improving transfer speed without overly complicating the data path structure
Solution Approach 2:
The patent introduces control structures such as doorbell messages and completion queues as intermediaries between the host and the accelerator-network adapter data path. These intermediaries enable centralized control and coordination without requiring the host to be directly involved in the high-speed data transfer, thus maintaining control while improving transfer speed
3Productivity
If host computing units process all data, then centralized management is achieved, but productivity decreases due to processing bottlenecks
Solution Approach 1:
The patent extracts the accelerator from the host processing workload and positions it as an independent processing unit in the data path. The accelerator handles data processing tasks directly from network adapter memory without requiring host computing unit intervention, thereby increasing overall data processing throughput while reducing the time the host spends on processing operations
Solution Approach 2:
The patent merges the network adapter and accelerator into a tightly coupled processing pipeline where data flows directly from network adapter to accelerator without host intervention. This merging creates a high-throughput path for data processing that operates independently of the host, increasing productivity while minimizing host processing time through coordinated operation of the combined adapter-accelerator system
Data Source
AI summary
A method in a network node that includes a host and an accelerator, includes holding a work queue that stores work elements, a notifications queue that stores notifications of the work elements, and control indices for adding and removing the work elements and the notifications to and from the work queue and the notifications queue, respectively. The notifications queue resides on the accelerator, and at least some of the control indices reside on the host. Messages are exchanged between a network and the network node using the work queue, the notifications queue and the control indices.