Virtual Machine Code Protection via Shadow Page Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual machines face challenges in managing translated program code, particularly when guest program code changes during execution, leading to inconsistencies in cached native instructions, which existing techniques struggle to address efficiently without incurring significant overhead.
Innovation Solution
The system employs a dynamic compiler and shadow page tables to track and protect translated guest program code by marking virtual memory pages as read-only, detecting modifications, and invalidating cached code sections, thereby ensuring consistency and reducing overhead through the use of a guest shadow page table and a compiler shadow page table.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If translated native ISA instructions are cached in a code cache for efficient execution, then execution speed is improved, but consistency is lost when guest program code changes during execution
Solution Approach 1:
The system performs preliminary actions by marking virtual memory pages as read-only before translation occurs. This preventive measure ensures that self-modifying code attempts are blocked before they can corrupt the translation cache, maintaining code consistency while allowing efficient cached execution of stable code segments.
Solution Approach 2:
The system implements feedback mechanisms through shadow page tables that track the executable status of memory pages. When a write attempt occurs on a translated page, the shadow page table detects this change and triggers invalidation of the corresponding translated code, ensuring the code cache remains synchronized with the current guest program state.
2Stability of the object's composition
If shadow page tables are used to track and protect translated code from self-modifying operations, then code consistency is maintained, but system overhead increases
Solution Approach 1:
The system segments the protection mechanism by implementing shadow page tables at the page level rather than requiring full virtual memory table duplication. This segmentation allows selective tracking of only those pages that contain translated code, reducing the overall overhead while maintaining consistency for protected segments.
Solution Approach 2:
The shadow page tables implement local quality by providing detailed tracking and protection information only for specific virtual memory pages that contain translated executable code, rather than uniformly applying protection to all memory. This localized approach minimizes overhead by focusing resources only where translation consistency is critical.
3Reliability
If write protection is applied to virtual memory pages containing translated code, then protection from self-modification is achieved, but legitimate self-modifying code execution is blocked
Solution Approach 1:
The system implements dynamic protection by making shadow page table entries modifiable during execution. Pages are marked as read-only when translation occurs, but this protection can be dynamically removed when self-modification is detected and the page is re-translated, allowing legitimate self-modifying code to execute while maintaining protection during stable translation periods.
Solution Approach 2:
The system applies preliminary anti-action by preemptively marking pages as read-only before translation to prevent unauthorized modification. However, this protection is designed to be reversible - when self-modification is detected through the shadow page table, the system lifts the protection after invalidating the old translation, thereby allowing legitimate self-modifying code while blocking malicious or erroneous modifications.
Data Source
AI summary
One embodiment provides a system that protects translated guest program code in a virtual machine that supports self-modifying program code. While executing a guest program in the virtual machine, the system uses a guest shadow page table associated with the guest program and the virtual machine to map a virtual memory page for the guest program to a physical memory page on the host computing device. The system then uses a dynamic compiler to translate guest program code in the virtual memory page into translated guest program code (e.g., native program instructions for the computing device). During compilation, the dynamic compiler stores in a compiler shadow page table and the guest shadow page table information that tracks whether the guest program code in the virtual memory page has been translated. The compiler subsequently uses the information stored in the guest shadow page table to detect attempts to modify the contents of the virtual memory page. Upon detecting such an attempt, the system invalidates the translated guest program code associated with the virtual memory page.