Root FPGA Routing for RDMA in Disaggregated Data Centers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network interface controllers (NICs) face challenges in efficiently managing increased network bandwidth and traffic volumes, leading to suboptimal performance in data centers with disaggregated resources.
Innovation Solution
The implementation of a novel network interface controller system that utilizes a communication subsystem with multiple FPGAs, enabling remote direct memory access (RDMA) messages to be processed efficiently through a unified address space, with a root FPGA and worker FPGAs connected via high-speed serial links, and a NIC coupled via a high-bandwidth I/O link, such as PCIe, to facilitate accelerated network processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If current NICs are used to handle increased network bandwidth, then network traffic volume increases, but processing efficiency deteriorates
Solution Approach 1:
The system divides the address space management into multiple FPGAs (root FPGA and worker FPGAs), each handling specific segments of the unified address space. This segmentation allows parallel processing of RDMA messages across multiple devices, maintaining high processing efficiency despite increased network traffic volume.
Solution Approach 2:
The root FPGA acts as an intermediary that receives RDMA messages from the NIC, determines the target worker FPGA based on the unified address space, and routes messages appropriately. This intermediary approach enables efficient message distribution across multiple FPGAs without creating bottlenecks at the NIC.
2Productivity
If multiple FPGAs are used to process RDMA messages, then processing capacity increases, but system complexity increases
Solution Approach 1:
Each worker FPGA is designed with universal functionality to handle various types of RDMA operations (read, write, atomic) across its assigned address space segments. The root FPGA provides universal address space management and message routing capabilities. This multi-functionality reduces the need for specialized hardware for each operation type, simplifying the overall system architecture.
Solution Approach 2:
Instead of having each FPGA manage its own local address space independently, the system implements a unified address space where all FPGAs share a common addressing scheme. This inversion of the traditional distributed address space model simplifies message routing and target identification, reducing system complexity despite multiple FPGAs.
3Speed
If high-speed serial links are used to connect FPGAs, then data transfer speed increases, but latency requirements become more stringent
Solution Approach 1:
The root FPGA performs preliminary actions by pre-determining the target worker FPGA based on the unified address space before message transmission. This preliminary routing decision eliminates the need for complex runtime address resolution at the worker FPGAs, reducing processing latency despite high-speed data transfer requirements.
Solution Approach 2:
The system replaces traditional mechanical switching and routing mechanisms with direct high-speed serial links between the NIC and FPGAs. This substitution eliminates intermediate switching stages and reduces the number of processing hops, thereby minimizing latency while maintaining high data transfer speeds.
Data Source
AI summary
Technologies for network interface controllers (NICs) include a computing device having a NIC coupled to a root FPGA via an I/O link. The root FPGA is further coupled to multiple worker FPGAs by a serial link with each worker FPGA. The NIC may receive a remote direct memory access (RDMA) message from a remote host and send the RDMA message to the root FPGA via the I/O link. The root FPGA determines a target FPGA based on a memory address of the RDMA message. Each FPGA is associated with a part of a unified address space. If the target FPGA is a worker FPGA, the root FPGA sends the RDMA message to the worker FPGA via the corresponding serial link, and the worker FPGA processes the RDMA message. If the root FPGA is the target, the root FPGA may process the RDMA message. Other embodiments are described and claimed.