Kernel Driver Malware Protection via System Call Virtualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sandboxing techniques are ineffective when the host system is already infected with malware and do not adequately filter system calls based on operation types or use advanced malware detection algorithms, failing to provide comprehensive protection for applications and confidential information.
Innovation Solution
A kernel-level driver intercepts system calls addressed to protected applications, determining if security rules allow or block the calls, and if none exist, executes them in a secure execution environment using virtualized components, thereby isolating potential threats and protecting critical data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sandboxing is used to isolate untrusted programs, then access to critical system areas is restricted, but the protection is ineffective when the host system is already infected by malware
Solution Approach 1:
The system segments system calls into different categories (allowed, blocked, redirected) and processes them through different paths. The protection driver segments the execution flow by intercepting system calls and routing them to appropriate handlers based on security rules, creating isolated execution paths for protected and unprotected operations.
Solution Approach 2:
The protection driver acts as an intermediary layer between applications and the operating system kernel. It intercepts system calls and introduces an analysis engine that evaluates security rules before allowing execution, thereby mediating between potentially malicious applications and the vulnerable host system.
2Reliability
If sandboxing limits program execution to exclude access to critical areas, then system security is improved, but filtering of system calls based on operation types and malware detection algorithms is not enabled
Solution Approach 1:
The system dynamically adapts its protection behavior based on the analysis of each system call. The analysis engine evaluates security rules in real-time and dynamically decides whether to allow, block, or redirect specific system calls, making the protection mechanism adaptable to different types of threats and operations rather than applying static restrictions.
Solution Approach 2:
The system changes the processing parameters of system calls based on security analysis. Depending on the security rule evaluation, system calls are transformed into different execution paths: allowed calls proceed normally, blocked calls are terminated, and redirected calls are sent to virtualized handlers, effectively changing the execution parameter based on security requirements.
3Reliability
If all system calls are executed in a secure virtual environment, then comprehensive protection is achieved, but system performance and resource consumption increase
Solution Approach 1:
Instead of virtualizing all system calls, the system applies partial virtualization only to those system calls that require protection according to security rules. The analysis engine identifies which system calls need redirection to virtual handlers, allowing legitimate calls to execute natively for performance while redirecting only suspicious or critical calls to the secure environment.
Solution Approach 2:
The system applies different processing qualities to different system calls based on their security requirements. Critical system calls that pose security risks are redirected to virtualized handlers with full protection, while benign system calls execute directly without virtualization overhead, giving each system call the appropriate level of protection rather than uniform treatment.
Data Source
AI summary
Disclosed are systems, methods and computer program products for protecting applications deployed on a host computer from malware using virtualization. An exemplary malware protection system may include a kernel-level driver configured to intercept system calls addressed to an object of a protected application. The system also includes an analysis engine configured to determine if there are security rules associated with one or more of the intercepted system call, the object of the protected application, and the actions allowed on the object of the protected application. The security rules indicate whether the system call is allowed or not allowed to be executed on the host computer. If there is no security rule associated with the system call, the system call is executed in a secure execution environment of the host computer using a virtual copy of the object of the protected application.