Method and system for transparent TCP offload with dynamic zero copy sending

Abstract

Certain aspects of a method and system for transparent transmission control protocol (TCP) offload with dynamic zero copy sending are disclosed. Aspects of a method may include enabling direct memory access of data for a particular user application directly from a user buffer to a buffer in a single integrated circuit, for example, a network interface card (NIC), without copying the data to a kernel buffer, based on an occurrence of a number of memory page faults. At least one page in the user buffer comprising data for the particular user application to be transmitted may be marked as a copy-on-write to prevent modification of contents of the buffer before receipt of at least one acknowledgement packet.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS / INCORPORATION BY REFERENCE [0001] This patent application makes reference to, claims priority to and claims benefit from U.S. Provisional Patent Application Ser. No. 60 / 700,544, filed on Jul. 18, 2005. [0002] This application also makes reference to: [0003] U.S. application Ser. No. ______ (Attorney Docket No. 17037US02) filed on even date herewith; [0004] U.S. application Ser. No. ______ (Attorney Docket No. 17546US02) filed on even date herewith; [0005] U.S. application Ser. No. ______ (Attorney Docket No. 17547US02) filed on even date herewith; [0006] U.S. application Ser. No. ______ (Attorney Docket No. 17553US02) filed on even date herewith; [0007] U.S. application Ser. No. ______ (Attorney Docket No. 17554US02) filed on even date herewith; and [0008] U.S. application Ser. No. ______ (Attorney Docket No. 17556US02) filed on even date herewith. [0009] Each of the above referenced applications is hereby incorporated herein by reference in t...

AI Technical Summary

Problems solved by technology

There may be various disadvantages to this approach.

The TTOE may also require some kind of manual configuration by an IT administrator, for example, by explicitly specifying an IP subnet address for accelerated connections to select which of the TCP flows will be offloaded and the offload engine is very complex as it needs to implement TCP packet processing.

While being efficient in reducing the transmit packet processing, LSO does not help with receive packet processing.

The multiple ACKs require consumption of scarce and expensive bandwidth, thereby reducing throughput and efficiency.

While this mechanism does not require any additional hardware from the NIC besides multiple hardware queues, it may have various performance limitations.

The aggregation may be limited to the amount of packets available to the host in one interrupt period.

If the interrupt period is short, and the number of flows is not small, the number of packets that are available to the host CPU for aggregation on each flow may be small, resulting in limited or no LRO aggregation, even if the number of hardware queues is large.

The additional copying provides a challenge for present memory subsystems, and as a result, these memory subsystems are unable to support high rates such as 10 Gbps.

In general, one challenge faced by TCP implementers wishing to design a flow-through NIC, is that TCP segments may arrive out-of-order with respect to the order placed in which they were transmitted.

This may prevent or otherwise hinder the immediate processing of the TCP control data and prevent the placing of the data in a user buffer.

In instances where the TCP segments are dropped or otherwise discarded, the sending side may have to re-transmit all the dropped TCP segments and in some instances, may result in about a fifty percent (50%) decrease in throughput or bandwidth utilization.

There may be various disadvantages to this approach.

The TOE may also require some kind of manual configuration by an IT administrator, for example, by explicitly specifying an IP subnet address for accelerated connections to select which of the TCP flows will be offloaded and the offload engine is very complex as it needs to implement TCP packet processing.

While being efficient in reducing the transmit packet processing, LSO does not help with receive packet processing.

The multiple ACKs require consumption of scarce and expensive bandwidth, thereby reducing throughput and efficiency.

Copy operations are CPU and memory bandwidth intensive, limiting system performance.

The host processing power may be consumed by the copying of data between user space and kernel space in the TCP / IP stack.

Although a local DMA engine may offload copying operations from the CPU, it does not relieve the memory bandwidth required.

The memory bandwidth may be a severe bottleneck in high speed networking applications as platforms are shifting to multiple CPU architectures, with multiple cores in each CPU architecture, all sharing the same memory.

Images