Network Protocol Stack Latency Reduction via Incremental Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data transmission systems experience significant latency due to the need for complete data availability before protocol processing can commence, especially in connection-oriented protocols like TCP, which restricts low-latency transmission paths and introduces delays in message transmission over networks connecting high-performance computing devices.
Innovation Solution
A method where an application forms parts of a message and requests a software interface to generate a message template with protocol headers, allowing incremental construction and processing of the message, with the network protocol stack completing headers and transmitting the message once all parts are available, utilizing a network interface device to minimize latency by performing protocol processing closer to the transmission point.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the network protocol stack waits for all data to be available before commencing protocol processing, then the completeness and accuracy of message transmission is ensured, but the transmission latency is significantly increased
Solution Approach 1:
The patent applies preliminary action by initiating protocol processing (header generation, error check calculation) before all message data is available. The protocol stack starts processing with the data that is currently available and continues as additional data arrives, rather than waiting for complete data availability before starting any processing.
Solution Approach 2:
The patent segments the message transmission process into independent stages that can proceed concurrently. Different parts of the protocol processing (header formation, error checks, data copying) are divided into separate operations that can be performed on available data segments while waiting for the rest of the data to arrive.
2Reliability
If connection-oriented protocols restrict transmission to only the protocol stack that negotiated the connection, then connection management and data integrity are maintained, but low-latency transmission paths are limited
Solution Approach 1:
The patent introduces an intermediary mechanism where the protocol stack negotiates a transmission path in advance and then hands off data to alternative transmission paths for actual data movement. This allows the connection management functions to remain with the original protocol stack while enabling faster transmission through dedicated low-latency paths.
Solution Approach 2:
The patent extracts the data transmission function from the protocol stack that negotiated the connection, separating connection management from actual data transmission. The protocol stack retains control for connection management while delegating the time-critical data transmission to optimized paths.
3Reliability
If protocol processing is performed at the host kernel, then comprehensive protocol control is achieved, but a low-latency transmission path to applications is not provided
Solution Approach 1:
The patent segments the protocol stack into multiple parts, with certain layers remaining at the host kernel for comprehensive control and other layers positioned closer to applications for low-latency transmission. This spatial segmentation allows different protocol functions to be performed at optimally located system levels.
Solution Approach 2:
The patent introduces an intermediary layer or mechanism that bridges the host kernel protocol stack and applications. This intermediary enables the protocol stack to maintain comprehensive control while providing applications with direct access to low-latency transmission paths without requiring full kernel mediation for every data operation.
4Adaptability or versatility
If a modern processor does not have transmission state and instructions available in cache, then processing flexibility is maintained, but significant latency is introduced
Solution Approach 1:
The patent applies preliminary action by pre-loading transmission state and instructions into processor cache before actual data transmission occurs. The system prepares and caches the protocol processing state, configuration data, and instruction sequences in advance so that when transmission is needed, the processor can execute with minimal cache misses and reduced latency.
Data Source
Figure 1
Figure 2~3
AI summary
A method of transmitting data in accordance with a connection-oriented network protocol, the method being for use at a data processing system coupled to a network and supporting first and second packet processing engines, the method comprising: initiating a connection at the first packet processing engine; the first packet processing engine passing first network protocol state to the second packet processing engine sufficient to permit the second packet processing engine to transmit a specified amount of data over the connection; forming application layer data for transmission; and the second packet processing engine performing packet processing of the application layer data in dependence on the first network protocol state so as to form one or more network messages and causing transmission of one or more network messages over the connection.