GPU as Programmable Packet Transfer Mechanism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network interface performance has increased, but packet processing has not kept pace, leading to bottlenecks due to CPU overhead, memory latency, and serialization of packet processing by multi-queue network interface controllers.
Innovation Solution
Combining a Graphics Processing Unit (GPU) with a Network Interface Controller (NIC) to offload packet processing functions, utilizing high thread-level parallelism and pre-processing data packets on the GPU before they reach the CPU, effectively creating a virtual NIC that enhances packet processing performance and reduces bottlenecks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If packet processing is performed on CPU, then processing flexibility is maintained, but processing speed is limited due to CPU overhead and serialization
Solution Approach 1:
The patent extracts packet processing functions from the CPU and implements them on the GPU. The GPU handles packet classification, routing decisions, and data plane processing, while the CPU focuses on control plane functions. This separation allows parallel processing of packets on the GPU, eliminating CPU serialization bottlenecks and significantly improving packet processing speed.
Solution Approach 2:
The patent introduces a GPU-based packet processing engine as an intermediary between the network interface controller and the CPU. This intermediary layer offloads packet processing tasks, enabling high-speed parallel processing while maintaining system flexibility through programmable packet processing functions on the GPU.
2Productivity
If multi-queue network interface controllers are used, then network throughput is improved, but packet processing becomes serialized creating bottlenecks
Solution Approach 1:
The patent segments packet processing into multiple independent queues on the GPU, each handling specific packet streams simultaneously. This segmentation enables parallel packet processing across multiple GPU threads, eliminating the serialization bottleneck while maintaining high network throughput from multi-queue NICs.
Solution Approach 2:
The patent transitions packet processing from a single-threaded CPU approach to a multi-threaded GPU approach, adding the dimension of parallelism. By distributing packet processing across thousands of GPU cores, the system achieves both high throughput from multi-queue NICs and high-speed parallel processing.
3Productivity
If packet processing is offloaded to GPU, then processing speed increases through parallelism, but system complexity increases
Solution Approach 1:
The patent makes the GPU universally applicable for packet processing by implementing a programmable packet processing engine. The same GPU infrastructure handles diverse packet processing functions including classification, routing, QoS, and encryption, eliminating the need for separate dedicated hardware for each function and managing complexity through software programmability.
4Ease of operation
If CPU processes all packets, then processing control is centralized, but memory latency and bus bandwidth become bottlenecks
Solution Approach 1:
The patent extracts time-sensitive packet processing operations from the CPU and executes them on the GPU with direct memory access capabilities. This extraction eliminates CPU memory latency and bus bandwidth bottlenecks by allowing the GPU to process packets independently without frequent CPU-GPU data transfers.
Data Source
AI summary
Discloses is an apparatus including a network interface controller (NIC), memory, and an accelerator. The accelerator can include a direct memory access (DMA) controller configured to receive data packets from the NIC and to provide the data packets to the memory. The accelerator can also include processing circuitry to generate processed data packets by implementing packet processing functions on the data packets received from the NIC, and to provide the processed data packets to at least one processing core. Other methods, apparatuses, articles and systems are also described.