Hypervisor Page Table Switching for VM Integrity Protection

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Virtual machine (VM) systems face challenges in protecting guest integrity drivers from kernel-level attacks, such as rootkit attacks, which can tamper with memory pages and compromise the integrity of the VM's security mechanisms.

Innovation Solution

Implementing a guest integrity driver that operates in an integrity mode, where the hypervisor monitors and protects specific memory pages, and switches between two sets of page tables to prevent malicious code from interfering with the protection mechanisms, ensuring only trusted software can enter integrity mode and execute protected hypercalls.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the guest integrity driver is implemented in a typical virtual machine environment, then the VM can operate independently with other VMs, but the driver becomes vulnerable to kernel-level attacks such as rootkit attacks that can tamper with memory pages and compromise driver integrity

Engineering Contradiction:
Improvedriver integrityVSAvoidkernel-level attacks
Core Design Contradiction:
ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The patent divides the memory protection mechanism into separate components: a first set of page tables for normal operation and a second set of page tables for integrity mode. This segmentation allows the system to isolate the integrity monitoring function from the normal VM operation, preventing kernel-level attacks from compromising the driver while maintaining independent VM operation.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The hypervisor acts as an intermediary between the guest integrity driver and the memory protection mechanisms. The hypervisor switches between different page tables based on whether the driver needs normal operation or integrity protection, mediating between the driver's operational needs and security requirements without requiring the driver to directly manage complex protection mechanisms.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If the hypervisor monitors all memory pages for integrity protection, then the driver is protected from tampering, but the monitoring overhead and system complexity increase significantly

Engineering Contradiction:
Improvememory protectionVSAvoidpage table switching mechanism
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

Instead of monitoring all memory pages continuously, the system uses partial action by switching between different page tables. The first page tables enable normal operation with standard monitoring, while the second page tables provide enhanced integrity protection only when needed. This partial application of protection mechanisms reduces overall system complexity while maintaining effective memory protection.

Inventive Principle:
Principle #16Partial or excessive action

3Productivity

If the guest integrity driver can access all memory pages freely, then the driver operates efficiently, but malicious code can tamper with the driver code or data without detection

Engineering Contradiction:
Improvedriver operation efficiencyVSAvoidmemory tampering
Core Design Contradiction:
ProductivityVSObject-affected harmful factors

Solution Approach 1:

The patent implements dynamic page table switching based on the operational state of the guest integrity driver. When the driver needs to access memory for normal operations, the first page tables are activated for efficient access. When integrity protection is required, the system switches to the second page tables that provide monitored access. This dynamic switching maintains driver efficiency during normal operation while providing security when needed.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentUS10430223B2Selective monitoring of writes to protected memory pages through page table switching
Publication Date: 2019.10.01 VMWARE INC
  • US10430223B2 patent drawing
  • US10430223B2 patent drawing
  • US10430223B2 patent drawing

AI summary

Mechanisms to protect the integrity of memory of a virtual machine are provided. The mechanisms involve utilizing certain capabilities of the hypervisor underlying the virtual machine to monitor writes to memory pages of the virtual machine. A guest integrity driver communicates with the hypervisor to request such functionality. Additional protections are provided for protecting the guest integrity driver and associated data, as well as for preventing use of these mechanisms by malicious software. These additional protections include an elevated execution mode, termed “integrity mode,” as well as protections on the memory pages that store the guest integrity driver. To prevent spurious alerts associated with the GI driver accessing its own data, the hypervisor maintains two page tables. In one copy, pages storing data for the GI driver are not protected and in the other, those pages are protected. The hypervisor switches the page tables when entering and exiting integrity mode.