NIC Packet Preprocessing Order Maintenance via Descriptor Ring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In virtual machine environments, offloading packet preprocessing from a virtual machine monitor (VMM) to a network interface controller (NIC) can result in out-of-order packet delivery due to the high processing speed of the NIC, leading to communication latency and bandwidth degradation, especially when the VMM initiates offload in the middle of a packet flow.
Innovation Solution
A two-stage mechanism is implemented where the VMM sends an instruction to the NIC to initiate preprocessing of data packets in a specified flow, followed by an acknowledgment, and only after the NIC has emptied its queue, the VMM enables the rule for delivering preprocessed packets to the virtual machine, ensuring all packets are delivered in the proper order without degrading communication bandwidth or latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If packet preprocessing is offloaded from VMM to NIC, then processing efficiency is improved, but packet delivery order may be violated
Solution Approach 1:
The NIC pre-processes packets in advance by creating descriptor rings and preparing packet metadata before the VMM needs them. This preliminary action allows the VMM to efficiently allocate buffers and process packets without disrupting delivery order, as the preprocessing work is completed beforehand in a controlled manner.
Solution Approach 2:
A descriptor ring structure serves as an intermediary between the NIC and VMM. The NIC writes preprocessed packet information to the descriptor ring, which the VMM then processes. This intermediary layer decouples the preprocessing function from the packet delivery path, allowing efficient offloading while maintaining order through the VMM's controlled processing of the descriptor ring.
2Productivity
If NIC processes packets at high speed, then bandwidth utilization is improved, but packet ordering may be compromised
Solution Approach 1:
The packet processing function is segmented into distinct stages: the NIC handles high-speed preprocessing and descriptor ring population, while the VMM handles buffer allocation and final packet delivery. This segmentation allows the NIC to operate at high speed for bandwidth utilization while the VMM ensures precise packet ordering through its controlled processing of the descriptor ring.
Solution Approach 2:
The descriptor ring structure provides a feedback mechanism where the NIC's preprocessing results are systematically reported to the VMM. The VMM processes descriptors in order and manages packet delivery accordingly, using the feedback from the NIC's high-speed processing to maintain ordering accuracy without sacrificing bandwidth utilization.
3Reliability
If VMM continues to preprocess all packets, then packet order is maintained, but processing latency increases
Solution Approach 1:
The NIC performs preliminary preprocessing of packets including header parsing and descriptor ring population before the VMM needs to process them. This preliminary action reduces the VMM's workload and processing latency while maintaining packet order, as the VMM only needs to allocate buffers and manage the descriptor ring rather than perform full preprocessing.
Solution Approach 2:
The time-consuming preprocessing functions are extracted from the VMM and transferred to the NIC. The NIC handles header parsing, checksum calculation, and descriptor ring population, extracting these heavy processing tasks from the VMM's path. This extraction reduces VMM processing latency while the VMM maintains packet order through its controlled buffer allocation and descriptor processing.
Data Source
AI summary
Network interface apparatus includes a host interface and a network interface, which receives packets in multiple packet flows destined for one or more virtual machines running on a host processor. Packet processing circuitry receives a first instruction from the host processor to offload preprocessing of the data packets in a specified flow in accordance with a specified rule, and initiates preprocessing of the data packets while writing one or more initial data packets from the specified flow to a temporary buffer. Upon subsequently receiving a second instruction to enable the specified rule, the initial data packets are delivered from the temporary buffer, after preprocessing by the packet processing circuitry, directly to a virtual machine to which the specified flow is destined, followed by preprocessing and delivery of subsequent data packets in the specified flow to the virtual machine.