Memory Page Execution Exception to Block Packer Evasion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current antivirus software struggles to detect malicious code hidden by packers, which change the byte-level appearance of malware, leading to evasion of signature-based detection systems and a significant decline in detection rates due to the large number of packers and their rapid evolution.
Innovation Solution
Implementing a mutual exclusion requirement that memory pages are either executable or writable but not both, with exception handlers to allow limited writing and scanning of memory pages before execution, thereby preventing packers from evading detection and allowing existing AV scanning technology to effectively detect known malware samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If packers are used to hide malware, then malware can evade signature-based detection systems, but antivirus detection capability deteriorates
Solution Approach 1:
The system performs preliminary actions by setting memory pages to non-executable state before malware can execute, and by intercepting page protection changes before they allow evasion. The antivirus scanner is invoked early in the process to analyze code before it runs, preventing malware from establishing its malicious operations.
Solution Approach 2:
The system introduces an intermediary layer between the packer and the execution environment. The exception handler acts as a mediator that monitors and controls memory page protection changes, preventing packers from silently modifying page attributes to evade detection while allowing legitimate operations to proceed.
2Reliability
If manual reverse-engineering of packers is performed to create unpackers, then detection of packed malware improves, but time and resource consumption increases significantly
Solution Approach 1:
The system implements a universal solution that works against any packer without requiring packer-specific knowledge. The memory protection mechanism and exception handler provide a generic framework that automatically handles packed malware regardless of the packing method used, eliminating the need for manual reverse-engineering of each packer variant.
Solution Approach 2:
The system enables self-service by allowing the antivirus scanner to automatically analyze suspicious code in memory without requiring manual intervention. The exception handler automatically triggers scanning when protected pages are accessed, and the system autonomously responds to packing evasion attempts without human assistance.
3Object-affected harmful factors
If memory pages are set to non-executable to prevent dynamic code execution, then malware execution is blocked, but legitimate dynamic code execution is also prevented
Solution Approach 1:
The system dynamically adjusts memory page protection based on runtime conditions. Pages are initially set to non-executable for security, but the exception handler dynamically changes protection attributes when legitimate execution is detected through the scanning process, allowing flexible adaptation between security and functionality requirements.
Solution Approach 2:
The system changes memory protection parameters controlled through exception handling. When code is scanned and approved, the page protection parameters are modified from non-executable to executable state, allowing legitimate dynamic code execution while maintaining security for unverified code.
4Reliability
If exception handlers are implemented to enforce page protection, then malware evasion is prevented, but system complexity increases
Solution Approach 1:
The system extracts the complexity of exception handling into a separate, dedicated handler module. This isolation allows the core security logic to remain simple while delegating the complex exception management to a specialized component that can be optimized and maintained independently.
Data Source
AI summary
To detect possible malicious code that is unpacked at runtime before it is executed, antivirus software requires that any dynamically created code be scanned before it can be executed by a host computer system. This requirement may be enforced by requiring memory pages to be either executable or writable, but not both. Before changing from writable but not executable to executable but not writable, the page is scanned for malicious code. To prevent packers from evading this scanning, the software may enforce the execution exception to prevent packers from changing whether a page is executable and thereby evading the scanning of dynamically created code. The software may also include exception handlers to allow a program to write to a page that contains the code being executed, but also limit such an operation (e.g., to a single step) to avoid evasion of the antivirus software.