DPU TCP Offload for Direct GPU Memory Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer networks rely on host CPUs to handle TCP data packet processing, which consumes CPU cycles and increases latency due to memory transfer times between CPU and GPU.
Innovation Solution
A DPU is used to handle TCP processing, directly transferring data packets to GPU memory via DMA, allowing GPUs to process TCP data in real-time without CPU intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If host CPU handles TCP data packet processing, then TCP connections are managed properly, but CPU cycles are consumed and latency increases
Solution Approach 1:
The patent extracts TCP processing functionality from the host CPU and relocates it to a separate DPU device. The DPU receives TCP packets via network interface, manages TCP connections, reorders packets, and transfers data to GPU memory, thereby freeing the host CPU from these tasks while maintaining reliable TCP connection management.
Solution Approach 2:
The DPU acts as an intermediary device between the network interface and the GPU. It receives packets from the network, processes TCP protocols, reorders packets if needed, and transfers data to GPU memory, serving as a mediator that handles TCP management responsibilities away from the host CPU.
2Productivity
If data is transferred from CPU memory to GPU memory, then data processing can proceed, but transfer time increases latency
Solution Approach 1:
The patent extracts the data transfer operation from the traditional CPU-memory-GPU path and redirects it through the DPU. The DPU transfers data directly to GPU memory using DMA transfers, eliminating the intermediate step of copying data to CPU memory first, thereby reducing transfer latency and improving throughput.
Solution Approach 2:
The patent replaces the software-based data copying mechanism (CPU reading from memory and writing to GPU memory) with a hardware-based DMA transfer system in the DPU. This substitution eliminates the need for CPU intervention in data transfer, reducing latency and improving efficiency.
3Ease of operation
If host CPU processes all TCP packets, then complete control is maintained, but processing speed decreases
Solution Approach 1:
The patent segments TCP processing responsibilities between the host CPU and the DPU. The host CPU retains control over high-level TCP connection management, while the DPU handles packet-level operations including reception, reordering, and data extraction. This segmentation enables parallel processing and improves overall packet processing speed.
Solution Approach 2:
The DPU serves as an intermediary that handles time-critical packet processing operations. It receives packets from the network interface, performs rapid reordering based on sequence numbers, and transfers data to GPU memory, thereby accelerating packet processing while the host CPU maintains supervisory control.
Data Source
AI summary
Apparatuses, systems, methods, and techniques to obtain information transmitted in data packets and store data, based at least in part on the information, in GPU memory. In at least one embodiment, the information is obtained and stored in GPU memory without using a central processing unit (CPU). In at least one embodiment, a data processing unit (DPU) recieves incoming data packets, stores information based at least in part on the data packets in DPU memory, and intiates a transfer directly to GPU memory. In at least one embodiment, the data packets are TCP data packets.