Kernel Memory Allocation Instrumentation for Shellcode Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing malware detection techniques are inadequate for detecting and mitigating dynamic shellcode exploits that deploy executable code within heap memory and allocate additional executable memory.

Innovation Solution

A compute instance is instrumented to detect kernel memory allocation functions, particularly those that allocate heap memory and make it executable, allowing for the detection of dynamic shellcode exploits when code executing from heap memory allocates additional heap memory and makes it executable.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional malware detection techniques are used, then existing security measures are maintained, but dynamic shellcode exploits in heap memory remain undetected

Engineering Contradiction:

Improvemalware detection capabilityVSAvoiddetection of dynamic shellcode exploits

Core Design Contradiction:

ReliabilityVSDifficulty of detecting and measuring

Solution Approach 1:

The system performs preliminary detection by instrumenting kernel memory allocation functions before shellcode exploits can execute. By intercepting calls to VirtualAlloc and VirtualProtect, the system identifies malicious heap memory allocation attempts prior to code execution, preventing the exploit from completing its malicious intent.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system introduces an intermediary detection layer between the shellcode exploit and the kernel memory management functions. By placing a monitor that intercepts calls to VirtualAlloc and VirtualProtect, the system creates a security barrier that detects malicious behavior without allowing direct access to kernel memory allocation functions.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If heap memory allocation is allowed for legitimate applications, then application functionality is maintained, but malicious code execution is enabled

Engineering Contradiction:

Improveapplication memory allocation flexibilityVSAvoidmalicious code execution

Core Design Contradiction:

Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The system applies local quality by differentiating between legitimate and malicious memory allocation requests. Through instrumentation of kernel functions, the system monitors the specific context of each allocation call, allowing legitimate applications to maintain their memory allocation flexibility while blocking allocations that exhibit characteristics of shellcode exploits.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system implements feedback mechanisms by monitoring memory allocation patterns and providing real-time detection signals. When suspicious allocation behavior is detected (such as rapid sequential allocations followed by executable permission changes), the system generates feedback to block the operation while allowing normal application behavior to continue uninterrupted.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20250173429A1Malware mitigation based on runtime memory allocation

Publication Date: 2025.05.29 SOPHOS LTD

AI summary

A compute instance is instrumented to detect certain kernel memory allocation functions, in particular functions that allocate heap memory and/or make allocated memory executable. Dynamic shell code exploits can then be detected when code executing from heap memory allocates additional heap memory and makes that additional heap memory executable.