Virtio Zero-Copy Packet Path for Live VM Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtio-based systems for network packet processing in virtual machines (VMs) incur significant overhead due to the need for copying data between virtio memory rings and physical network device rings, which hampers efficient live VM migration.
Innovation Solution
Implementing a zero-copy mechanism for both data packet transmission (Tx) and reception (Rx) paths by bypassing the macvtap driver, allowing direct interaction between virtio devices in the VM and the virtual function (VF) network driver, thereby eliminating the need for data copying.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the macvtap driver is used to interface between vhost-net module and physical network device, then the system provides a mechanism to get packets in and out of the actual network device, but the Tx and Rx buffers from the VM's virtio memory rings need to be copied into the network device's physical Tx and Rx rings, causing substantial overhead
Solution Approach 1:
The patent extracts and removes the macvtap driver from the data path between vhost-net and physical network device. By eliminating this intermediate driver, the system achieves direct communication between the virtio memory rings and physical network device rings, removing the copying overhead that the macvtap driver introduced while maintaining packet transmission functionality.
Solution Approach 2:
The patent merges the vhost-net module directly with the physical network device interface by establishing a direct mapping between virtio memory rings and physical network device rings. This consolidation eliminates the separate macvtap driver layer and its associated copying operations, allowing buffers to be shared directly between VM and physical device.
2Speed
If the vhost-net module is used to reduce virtualization overhead, then latency is significantly reduced, but the copy mechanism in the Tx and Rx data packet paths still causes substantial overhead
Solution Approach 1:
The patent removes the copy mechanism from the data path by eliminating the macvtap driver that performed buffer copying. The vhost-net module retains its low-latency characteristics while the harmful copying operation is extracted and eliminated, achieving zero-copy packet transmission.
Solution Approach 2:
The patent introduces a direct memory mapping mechanism as an intermediary between vhost-net and the physical network device, allowing buffers to be shared without copying. This intermediary layer enables direct access to VM memory from the physical network device, eliminating the need for the macvtap driver's copy operations.
3Ease of operation
If the VM memory space is managed by a VM monitor such as Qemu with layout translation, then the vhost-net module can directly translate buffer addresses, but the VM memory is not directly accessible by the VF driver, requiring copy mechanisms
Solution Approach 1:
The patent extracts the address translation function from the VM monitor (Qemu) and moves it to the vhost-net module. By taking out the translation responsibility from Qemu, the system allows the VF driver to directly access VM memory through the vhost-net module's translation mechanism, eliminating the need for additional copy operations.
Solution Approach 2:
The vhost-net module serves as an intermediary between the VM monitor's managed memory space and the VF driver. It provides direct address translation capabilities that allow the VF driver to access VM memory directly without requiring copies, bridging the gap between Qemu's memory management and the network driver's access needs.
Data Source
AI summary
A new approach is proposed that contemplates systems and methods to support virtio-based data packet path optimization for live virtual machine (VM) migration for Linux. Specifically, a data packet receiving (Rx) path and a data packet transmitting (Tx) path between a VM running on a host and a virtual function (VF) driver configured to interact with a physical network device of the host to receive and transmit communications dedicated to the VM are both optimized to implement a zero-copy solution to reduce overheads in packet processing. Under the proposed approach, the data packet Tx path utilizes a zero-copy mechanism provided by Linux kernel to avoid copying from virtio memory rings/Tx vrings in memory of the VM. The data packet Rx path also implements a zero-copy solution, which allows a virtio device of the VM to communicate directly with the VF driver of the network device while bypassing a macvtap driver entirely from the data packet Rx path.