Hypervisor PCIe Configuration Space Mapping for VM Boot

Solution Overview

Problem

The existing virtual machine (VM) boot process is prolonged due to slow trap-and-emulate mechanisms for Enhanced Configuration Access Mechanism (ECAM) accesses and the need for extensive bus scans during VM startup, which are time-consuming and inefficient.

Innovation Solution

The hypervisor optimizes ECAM access by reserving host physical memory pages to directly map configuration spaces of absent PCIe device functions to a non-presence value and populating pages for present functions with fixed hardware resources, eliminating the need for traps and reducing the number of ECAM accesses during boot.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If trap-and-emulate mechanism is used for ECAM accesses in virtualized systems, then configuration space access is enabled for VMs, but access speed becomes slow and VM boot time increases

Engineering Contradiction:

Improveconfiguration space accessVSAvoidECAM access speed

Core Design Contradiction:

ReliabilityVSSpeed

Solution Approach 1:

The patent segments the configuration space access by identifying which PCIe device functions are present versus absent. For absent functions, it creates a separate handling path using a non-presence value in host physical memory, while present functions continue to use the hypervisor-mediated trap-and-emulate path. This segmentation allows fast direct access for absent functions while maintaining correct emulation for present functions.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary action by pre-populating host physical memory with a non-presence value at a specific address before the VM boot process. This pre-prepared data structure allows the guest OS to quickly determine absent PCIe functions without triggering hypervisor traps, thereby speeding up the configuration space access during VM startup.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If extensive bus scans are performed during VM startup to detect PCIe devices, then complete device configuration is achieved, but boot time becomes prolonged

Engineering Contradiction:

Improvedevice configuration completenessVSAvoidVM boot time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent performs preliminary action by pre-populating host physical memory with a non-presence value at a specific address before the VM boot process. This pre-prepared data structure allows the guest OS to quickly determine absent PCIe functions without triggering hypervisor traps, thereby speeding up the configuration space access during VM startup.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent extracts the time-consuming trap-and-emulate operations from the common path of ECAM access. By identifying absent PCIe device functions and providing direct access to their configuration spaces through pre-populated host memory, it removes the need for hypervisor mediation in these cases, thereby extracting the performance bottleneck from the boot process.

Inventive Principle:

Principle #2Taking out (Extraction)

3Reliability

If hypervisor traps every ECAM access for emulation, then accurate device configuration is maintained, but the number of trap operations increases system overhead

Engineering Contradiction:

Improveconfiguration accuracyVSAvoidhypervisor overhead

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the configuration space access by identifying which PCIe device functions are present versus absent. For absent functions, it creates a separate handling path using a non-presence value in host physical memory, while present functions continue to use the hypervisor-mediated trap-and-emulate path. This segmentation allows fast direct access for absent functions while maintaining correct emulation for present functions.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent extracts the time-consuming trap-and-emulate operations from the common path of ECAM access. By identifying absent PCIe device functions and providing direct access to their configuration spaces through pre-populated host memory, it removes the need for hypervisor mediation in these cases, thereby extracting the performance bottleneck from the boot process.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS11880301B2Enabling efficient guest access to peripheral component interconnect express (PCIe) configuration space

Publication Date: 2024.01.23 VMWARE INC

AI summary

Techniques for enabling efficient guest OS access to PCIe configuration space are provided. In one set of embodiments, a hypervisor can reserve a single host physical memory page in the host physical memory of a host system and can populate the single host physical memory page with a value indicating non-presence of PCIe device functions. The hypervisor can then create, for each guest physical memory page in a guest physical memory of a virtual machine (VM) corresponding to a PCIe configuration space of an absent PCIe device function in the VM, a mapping in the hypervisor's second-level page tables that maps the guest physical memory page to the single host physical memory page.