Hypervisor-Based System Call Monitoring via MSR Shadowing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern computer security monitoring capabilities are insufficient due to the limitations imposed by PatchGuard, which restricts the ability to monitor fundamental process interactions such as system calls, leading to reduced robustness in detecting malicious intrusions.
Innovation Solution
An in-band software monitor intercepts system calls made from a Model Specific Register (MSR) within the kernel, while an out-of-band hypervisor traps read and write requests to the MSR, allowing for obscured presence and enhanced monitoring operations without triggering PatchGuard or impacting system stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If Direct Kernel Object Manipulation (DKOM) is used to monitor system calls, then monitoring capabilities are enhanced, but system stability deteriorates due to PatchGuard triggering Blue Screen of Death
Solution Approach 1:
The patent introduces an intermediary mechanism (hypervisor or kernel driver) that mediates between the monitoring application and the system call interface. This intermediary captures and analyzes system call information without requiring direct modification of protected kernel objects, thereby avoiding PatchGuard detection while maintaining monitoring capabilities.
Solution Approach 2:
The patent creates copies of system call information and kernel object data through legitimate interfaces rather than directly manipulating the original protected objects. By working with copies obtained through approved methods, the system can analyze and monitor without triggering PatchGuard's integrity checks.
2Reliability
If approved monitoring methods are used to avoid PatchGuard, then system stability is maintained, but monitoring capabilities are reduced
Solution Approach 1:
The patent implements a multi-functional monitoring system that can operate through multiple approved interfaces (system call interception, process creation monitoring, network activity tracking, file system monitoring). This universal approach compensates for the limitations of individual approved methods by providing comprehensive monitoring capabilities across different system domains.
Solution Approach 2:
The patent divides the monitoring functionality into multiple independent modules, each responsible for a specific aspect of system monitoring (process monitoring, network monitoring, file system monitoring, registry monitoring). This segmentation allows each module to use appropriate approved methods for its specific domain while collectively providing comprehensive monitoring coverage.
3Difficulty of detecting and measuring
If kernel modifications are made for monitoring, then monitoring robustness is improved, but ease of operation deteriorates due to complexity of implementation
Solution Approach 1:
The patent implements self-service mechanisms where the monitoring system automatically performs tasks such as registering for system call notifications, setting up process creation callbacks, and configuring network filters without requiring manual kernel object manipulation. This automation reduces implementation complexity while maintaining robust monitoring capabilities.
Data Source
AI summary
A security system and method efficiently monitors and secures a computer to defend against malicious intrusions, and includes an in-band software monitor disposed within a kernel in communication with an operating system (OS) of the computer. The monitor intercepts system calls made from an MSR (Model Specific Register), to execute monitoring operations, and subsequently returns execution to the OS. An out-of-band hypervisor communicably coupled to the OS, has read shadow means for trapping read requests to the MSR, and write mask means for trapping write requests to the MSR. The hypervisor includes means for responding to the trapped read and write requests so that presence of the monitor is obscured.