Control Plane Function in Hardware Data Processor for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed computing systems face performance degradation due to internal latencies, particularly in the control plane of computing devices, which delays data generation and transfer between hardware data processors and network adapters, hindering the efficiency of operations like neural network training.
Innovation Solution
Implementing a control plane function within the hardware data processor, utilizing a direct memory access (DMA) engine and reducing host processor involvement in data transfer operations, allows for faster data generation and transfer by minimizing latency and software overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If control plane function is implemented at host processor, then system management and control are simplified, but data generation and transfer latency increases
Solution Approach 1:
The control plane function is extracted from the host processor and relocated to the hardware data processor. This allows the host processor to focus on computation while the hardware data processor handles control operations independently, thereby reducing data generation and transfer latency without sacrificing management simplicity.
Solution Approach 2:
A dedicated control plane interface is introduced as an intermediary between the hardware data processor and the network adapter. This intermediary handles control operations locally at the hardware data processor, eliminating the need for host processor involvement in data transfer operations and reducing overall latency.
2Device complexity
If host processor is involved in data transfer operations, then data transfer control is centralized, but software overhead and latency increase
Solution Approach 1:
The data transfer control function is segmented from the host processor and assigned to the hardware data processor. This segmentation creates independent control paths that operate in parallel with computation operations, eliminating software overhead and improving data transfer speed while maintaining organized control structure.
Solution Approach 2:
The hardware data processor is empowered to perform data transfer operations autonomously without requiring host processor intervention. The control plane interface enables the hardware data processor to self-manage data transfer operations, reducing software overhead and improving productivity.
3Reliability
If control operations are performed at host processor, then system coordination is maintained, but computation efficiency is reduced
Solution Approach 1:
Control operations are extracted from the host processor and relocated to the hardware data processor. This extraction allows the host processor to dedicate full resources to computation while the hardware data processor independently handles control operations, maintaining system coordination through the control plane interface while improving computation efficiency.
Data Source
AI summary
In one example, an apparatus comprises: a first local memory, a computation engine configured to generate local data and to store the local data at the first local memory, and a controller. The apparatus is coupled with a host processor and a second device via an interconnect, the second device comprising a second local memory, the host processor hosting an application. The controller is configured to: receive, from the second device, a first message indicating that first data is stored in the second local memory; based on the first message: fetch the first data from the second local memory via the interconnect; control the computation engine to perform a computation operation on the first data to generate second data to support the application hosted by the host processor; and transmit, to the second device, a second message indicating that the second data is stored in the first local memory.