Kernel Syscall Hooking to Block BYOVD EDR Termination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems are ineffective in preventing malicious actors from exploiting vulnerable drivers to execute arbitrary code and gain control over computer systems, as traditional security measures fail to detect and mitigate rootkit attacks that operate at a low level within the operating system.
Innovation Solution
Deploying a hooked version of a syscall in the operating system kernel to modify the execution status of processes, specifically identifying and inhibiting execution of protected processes to prevent unauthorized changes, thereby thwarting BYOVD cyberattacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional security measures are used to detect and mitigate rootkit attacks, then general security protection is provided, but they fail to detect and mitigate rootkit attacks that operate at a low level within the operating system
Solution Approach 1:
The patent introduces a new dimension of security monitoring by deploying a hooked version of the syscall within the operating system kernel itself. This allows detection at the system call interface level, creating a new observational dimension that captures rootkit activities before they can execute malicious operations, thereby resolving the detection limitation of traditional external security tools.
Solution Approach 2:
The hooked syscall acts as an intermediary between the rootkit and the operating system kernel. By intercepting and monitoring syscall requests, it mediates the interaction between potentially malicious rootkit code and critical system functions, enabling detection and prevention without requiring changes to the rootkit itself or the core kernel code.
2Object-affected harmful factors
If vulnerable drivers are exploited to execute arbitrary code with elevated privileges, then unauthorized control over the system is gained, but this bypasses traditional security measures
Solution Approach 1:
The patent implements preliminary action by deploying the hooked syscall in advance, before any rootkit exploitation occurs. The hooked syscall is pre-configured to monitor and control specific system calls that could be used for malicious purposes, establishing a defensive barrier before the vulnerable driver can be exploited to execute arbitrary code.
Solution Approach 2:
The hooked syscall applies preliminary anti-action by proactively identifying and blocking malicious syscalls before they can execute harmful operations. When a syscall matching known malicious patterns is detected, the hooked version prevents its execution, countering the potential exploitation of vulnerable drivers before unauthorized control can be established.
3Reliability
If a hooked version of syscall is deployed to monitor and inhibit malicious processes, then protection against rootkit attacks is improved, but system complexity increases
Solution Approach 1:
The hooked syscall serves multiple functions simultaneously: it monitors system calls, identifies malicious patterns, blocks harmful operations, and logs security events. By consolidating these security functions into a single syscall interception mechanism, the patent reduces overall system complexity compared to implementing separate monitoring, detection, and response systems.
Solution Approach 2:
The hooked syscall implements self-service by autonomously monitoring and blocking malicious operations without requiring constant external intervention. The security mechanism is self-contained within the kernel, automatically detecting and responding to threats based on predefined criteria, thereby reducing the operational complexity of security management.
Data Source
AI summary
Methods, storage systems and computer program products implement embodiments of the present invention that include deploying, in a memory used by an operating system kernel of a computer, a hooked version of a syscall that is configured to modify an execution status of processes executing on the computer. Subsequent to one or more protected processes being specified, a notification of a call to the syscall from the hooked version of the syscall is received, the call requesting to change the execution status of a given process executing on the computer. Finally, upon ascertaining that the given process is one of the protected processes, execution of the syscall is inhibited.