Smart IO Device Command Queues for Virtual Machine Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtualization systems require guest domains to go through a software Hypervisor for all IO access, leading to performance issues due to multiple kernel transitions and inefficiencies in control operations, which also restrict the use of the same OS distribution on virtualized and non-virtualized servers.
Innovation Solution
A smart IO device with unrestricted and restricted command queues allows guest domains to directly configure and control IO resources, offloading Hypervisor involvement and enabling direct hardware access for both datapath and control operations, using a command queue creation function and address translation to validate and translate addresses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all IO accesses go through the Hypervisor software layer, then resource isolation and control are maintained, but performance deteriorates due to multiple kernel transitions and increased CPU demands
Solution Approach 1:
The command queue is segmented into multiple types (unrestricted, restricted, and proxy) with different access permissions. Unrestricted CQs allow direct guest access for performance-critical operations, restricted CQs maintain Hypervisor control for security, and proxy CQs handle privileged operations. This segmentation enables simultaneous optimization of performance and security by routing different IO accesses through appropriate pathways.
Solution Approach 2:
The proxy command queue acts as an intermediary mechanism between guest domains and the Hypervisor. When a guest domain needs to perform privileged operations, it submits commands through the proxy CQ, which then handles the interaction with the Hypervisor. This intermediary approach reduces the frequency of direct Hypervisor involvement while maintaining proper control and isolation.
2Reliability
If multiple kernel transitions are used for IO access in virtualized environments, then resource control and isolation are ensured, but CPU load increases and execution efficiency decreases
Solution Approach 1:
The system performs preliminary configuration by creating and configuring multiple types of command queues before guest domain operation. The Hypervisor pre-establishes unrestricted CQs for direct access, restricted CQs for controlled access, and proxy CQs for privileged operations. This preliminary setup eliminates the need for frequent kernel transitions during runtime, as guests can directly use the pre-configured unrestricted CQs for common operations.
Solution Approach 2:
Guest domains are enabled to self-service by directly accessing unrestricted command queues and configuring their own IO resources without Hypervisor intervention. The gatekeeper module automatically handles access control and resource allocation based on pre-configured policies, allowing guest domains to manage their IO operations independently while maintaining system-wide resource control.
3Productivity
If direct hardware access is enabled for guest domains, then IO performance improves, but system security and resource isolation may be compromised
Solution Approach 1:
Different command queues are assigned different access qualities and permissions. Unrestricted CQs provide direct guest access with full functionality for specific IO operations, while restricted CQs maintain Hypervisor-controlled access for other operations. The gatekeeper module applies local quality control by determining which CQ type to use based on the specific IO request, enabling direct hardware access where safe while maintaining security where needed.
Solution Approach 2:
The system changes the access parameters of command queues based on the operation type and guest domain permissions. By dynamically selecting between unrestricted, restricted, and proxy CQ modes, the system adjusts the level of direct hardware access granted to guests. This parameter-based control allows the system to optimize performance for appropriate operations while maintaining security boundaries through parameter-driven access control.
Data Source
AI summary
A system and method for accelerating input/output (IO) access operation on a virtual machine, The method comprises providing a smart IO device that includes an unrestricted command queue (CQ) and a plurality of restricted CQs and allowing a guest domain to directly configure and control IO resources through a respective restricted CQ, the IO resources allocated to the guest domain. In preferred embodiments, the allocation of IO resources to each guest domain is performed by a privileged virtual switching element. In some embodiments, the smart IO device is a HCA and the privileged virtual switching element is a Hypervisor.