Virtualization Layer Guest Agent Memory Protection

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

The virtualization layer in malware detection architectures is unable to discern higher-level context about guest processes and lacks mechanisms to protect the integrity, confidentiality, and availability of the guest agent, making it susceptible to malicious attacks.

Innovation Solution

The virtualization layer implements a security mechanism that alters page permissions in nested page tables to restrict access to guest agent memory pages, ensuring integrity and confidentiality, and uses interrupt service routines to guarantee execution time, thereby protecting the guest agent from malicious attacks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the virtualization layer monitors guest processes for malware detection, then detection capability is improved, but the guest agent becomes vulnerable to malicious attacks due to lack of protection mechanisms

Engineering Contradiction:
Improvemalware detection capabilityVSAvoidguest agent security
Core Design Contradiction:
Measurement precisionVSReliability

Solution Approach 1:

The patent segments the virtualization architecture by introducing a nested page table hierarchy that separates guest virtual addresses into multiple translation stages. The guest agent operates in a protected segment with isolated memory access rights, while the virtualization layer maintains monitoring capabilities through controlled address translation. This segmentation allows the guest agent to perform malware detection without being directly exposed to attacks from guest processes.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces nested page tables as an intermediary layer between the guest agent and physical memory. This intermediary structure enables the virtualization layer to mediate all memory access requests from the guest agent, allowing security policies to be enforced while maintaining the agent's functionality. The nested page tables act as a trusted mediator that protects the agent from malicious attacks while permitting legitimate monitoring operations.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If the virtualization layer restricts access to guest agent memory pages to protect integrity and confidentiality, then security is improved, but access to guest process context information is limited

Engineering Contradiction:
Improveguest agent protectionVSAvoidguest process context
Core Design Contradiction:
ReliabilityVSLoss of information

Solution Approach 1:

The patent applies local quality by implementing different access permissions for different regions of the guest agent's address space. Critical security-related memory pages are protected with restricted access through nested page table entries, while other regions maintain appropriate access levels for monitoring. This selective permission scheme allows the virtualization layer to protect sensitive information while still accessing necessary guest process context for malware detection.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent adds another dimension to memory address translation by introducing a nested page table hierarchy that operates in addition to the standard guest page tables. This additional translation layer provides a new dimension for controlling access to guest agent memory, enabling fine-grained security policies without preventing the virtualization layer from obtaining necessary context information through controlled translations.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

3Measurement precision

If the guest agent operates at the same privilege level as malware, then monitoring capability is improved, but the guest agent becomes highly susceptible to malicious attacks

Engineering Contradiction:
Improvemonitoring capabilityVSAvoidsusceptibility to attacks
Core Design Contradiction:
Measurement precisionVSObject-affected harmful factors

Solution Approach 1:

The patent implements a nested doll structure by placing nested page tables within the virtualization layer that encapsulate and protect the guest agent. The guest agent retains its monitoring capabilities within this nested protective structure, while the nested page tables provide an additional layer of indirection and protection. This nesting allows the agent to monitor malware at close proximity while being protected by the surrounding nested page table hierarchy.

Inventive Principle:
Principle #7Nested doll (Nesting)

Solution Approach 2:

The patent provides beforehand cushioning by pre-configuring nested page table entries that establish protected access paths for the guest agent before any malicious attacks occur. These pre-established protection mechanisms cushion the agent against attacks by blocking malicious memory access attempts before they can reach the agent's critical data structures, while still allowing legitimate monitoring operations to proceed.

Inventive Principle:
Principle #11Beforehand cushioning (Prior cushioning)

Data Source

PatentUS10642753B1System and method for protecting a software component running in virtual machine using a virtualization layer
Publication Date: 2020.05.05 MAGENTA SECURITY HOLDINGS LLC
  • US10642753B1 patent drawing
  • US10642753B1 patent drawing
  • US10642753B1 patent drawing

AI summary

A computing device features one or more hardware processors and a memory that is coupled to the one or more processors. The memory comprises software that supports virtualization, including a virtual machine operating in the guest mode and a virtualization layer operating in the host mode. The virtual machine is configured to execute a plurality of processes including a guest agent process. The virtualization layer is configured to protect the guest agent process operating within the virtual machine that provides metadata to the virtualization layer by restricting page permissions for memory pages associated with the guest agent process when the guest agent process is inactive.