Method and apparatus for data re-assembly with a high performance network interface

Abstract

An intelligent network interface card (INIC) or communication processing device (CPD) works with a host computer for data communication. The device provides a fast-path that avoids protocol processing for most messages, greatly accelerating data transfer and offloading time-intensive processing tasks from the host CPU. The host retains a fallback processing capability for messages that do not fit fast-path criteria, with the device providing assistance such as validation even for slow-path messages, and messages being selected for either fast-path or slow-path processing. A context for a connection is defined that allows the device to move data, free of headers, directly to or from a destination or source in the host. The context can be passed back to the host for message processing by the host. The device contains specialized hardware circuits that are much faster at their specific tasks than a general purpose CPU. A preferred embodiment includes a trio of pipelined processors devoted to transmit, receive and utility processing, providing full duplex communication for four Fast Ethernet nodes.

Description

CROSS REFERENCE TO RELATED APPLICATIONS [0001] This application claims the benefit under 35 U.S.C. §120 of (is a continuation of) U.S. patent application Ser. No. 10 / 005,536, filed Nov. 7, 2001, which in turn claims the benefit under 35 U.S.C. §120 of (is a continuation of) U.S. patent application Ser. No. 09 / 384,792, filed Aug. 27, 1999, now U.S. Pat. No. 6,434,620, which in turn: 1) claims the benefit under 35 U.S.C. §119 of Provisional Patent Application Ser. No. 60 / 098,296, filed Aug. 27, 1998, 2) claims the benefit under 35 U.S.C. §120 of (is a continuation-in-part of) U.S. patent application Ser. No. 09 / 067,544, filed Apr. 27, 1998, now U.S. Pat. No. 6,226,680, and 3) claims the benefit under 35 U.S.C. §120 of (is a continuation-in-part of) U.S. patent application Ser. No. 09 / 141,713, filed Aug. 28, 1998, now U.S. Pat. No. 6,389,479. [0002] U.S. Pat. No. 6,226,680 and U.S. Pat. No. 6,389,479 both claim the benefit under 35 U.S.C. §119 of Provisional Patent Application Ser. No....

AI Technical Summary

Benefits of technology

[0013] The present invention offloads network processing tasks from a CPU to a cost-effective intelligent network interface card (INIC). An advantage of this approach is that a vast majority of network message data is moved directly from the INIC into its final destination. Another advantage of this approach is that the data may be moved in a single trip across the system memory bus. The offloading allows the CPU to avoid header processing, data copying, and checksumming. Since network message data does not need to be placed in a CPU cache, the CPU cache may be free for storage of other important instructions or data. Interrupts may be reduced to four interrupts per 64 k SMB read and two interrupts per 64 k SMB write. Other advantages include a reduction of CPU reads over the PCI bus and fewer PCI operations per receive or transmit transaction.

Problems solved by technology

Network processing as it exists today is a costly and inefficient use of system resources.

This is particularly expensive because while the CPU is moving this data it can do nothing else.

While moving the data the CPU is typically stalled waiting for the relatively slow memory to satisfy its read and write requests.

Even today's advanced pipelining technology doesn't help in these situations because that relies on the CPU being able to do useful work while it waits for the memory controller to respond.

Moving all this data with the CPU slows the system down even after the data has been moved.

After the data has been moved, the former resident of the cache will likely need to be pulled back in, stalling the CPU even when we are not performing network processing.

The most obvious expense is calculating the checksum for each TCP segment (or UDP datagram).

The TCP connection object must be located when a given TCP segment arrives, IP header checksums must be calculated, there are buffer and memory management issues, and finally there is also the significant expense of interrupt processing, discussed below.

Each of these segments may result in an interrupt to the CPU.

While this is possible, it is not terribly likely.

And delays in interrupt processing may mean that we are able to process more than one incoming network frame per interrupt.

Interrupts tend to be very costly to the system.

While the processor pipeline is an extremely efficient way of improving CPU performance, it can be expensive to get going after it has been flushed.

Finally, each of these interrupts results in expensive register accesses across the peripheral bus (PCI).

We noted earlier that when the CPU has to access system memory, it may be stalled for several hundred nanoseconds.

When it has to read from PCI, it may be stalled for many microseconds.

The most troubling thing about this is that since interrupt lines are shared on PC-based systems, we may have to perform this expensive PCI read even when the interrupt is not meant for us.

Other peripheral bus inefficiencies also exist.

Images