Automated Malware Unpacking via Dynamic Memory Tracking

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current antivirus systems face significant challenges in detecting malware packed using various packers, as these tools change the byte-level appearance of malware, evading signature-based detection methods, and require manual reverse-engineering of packers, which is impractical due to the large number of packers and their rapid evolution.

Innovation Solution

The packing manager system provides an automated method to track memory areas written and executed by executables, allowing for unpacking of malware packed by multiple arbitrary packers without reverse-engineering, using a dynamic content detection module, exception handlers, and an analysis module to invoke AV scanning before execution.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If signature-based detection methods are used, then detection of known malware is effective, but malware packed by packers evades detection by changing byte-level appearance

Engineering Contradiction:
Improvemalware detection accuracyVSAvoiddetection capability against packed malware
Core Design Contradiction:
ReliabilityVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary unpacking of the malware binary before detection. The unpacker extracts the packed malware to its original form, and then signature-based detection is applied to the unpacked version. This preliminary action restores the malware to its detectable state before the detection process begins, allowing signature matching to work effectively against packed malware.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The unpacker serves as an intermediary component between the packed malware and the signature-based detection system. It translates the obfuscated packed format back into the original malware format, enabling the detection system to work with familiar signatures while the malware remains in its packed, evasive form in the wild.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If manual reverse-engineering of packers is performed to create unpackers, then detection of specific packed malware is improved, but the process is slow and expensive and cannot keep up with the large number of evolving packers

Engineering Contradiction:
Improvedetection capability for specific packersVSAvoidrate of packer coverage
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The system implements a universal unpacking approach that does not require separate manual reverse-engineering for each packer. The generic unpacking mechanism can handle multiple packer types by recognizing common unpacking patterns and behaviors, allowing a single system to deal with hundreds or thousands of different packers without requiring individual attention to each one.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system automatically identifies and unpacks malware using packers without requiring human intervention for each new packer variant. The automated detection of unpacking behavior and the self-configuring nature of the system allow it to adapt to new packers as they emerge, maintaining high productivity without manual reverse-engineering efforts.

Inventive Principle:
Principle #25Self-service

3Adaptability or versatility

If emulation is used to unpack malware, then unpacking of arbitrary packers is achieved, but the process is very slow and requires large amounts of virtual memory

Engineering Contradiction:
Improvecapability to unpack arbitrary packersVSAvoidunpacking speed
Core Design Contradiction:
Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system extracts only the essential unpacking functionality from the packed malware without requiring full emulation of the malware's execution environment. By identifying and extracting the unpacking code and data, the system avoids the overhead of emulating entire instruction sets and memory management, significantly reducing time and memory requirements while still achieving unpacking of arbitrary packers.

Inventive Principle:
Principle #2Taking out (Extraction)

4Productivity

If dynamic translation is used to improve emulation speed, then unpacking speed increases, but the method often fails to run correctly because packers check code integrity

Engineering Contradiction:
Improveunpacking speedVSAvoidcorrectness of unpacking execution
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

Instead of dynamically translating and executing unpacking code, the system creates a copy of the unpacking environment and data structures needed to extract the malware. This copying approach allows the system to preserve the original code integrity checks while still performing unpacking operations on the copied data, maintaining both speed and correctness without triggering packer integrity verification.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS7996904B1Automated unpacking of executables packed by multiple layers of arbitrary packers
Publication Date: 2011.08.09 GEN DIGITAL INC
  • US7996904B1 patent drawing
  • US7996904B1 patent drawing
  • US7996904B1 patent drawing

AI summary

The packing manager provides an automated method that allows existing AV scanning technology to be applied to detect known malware samples packed by one or more packers that are potentially proprietary. The packing manager tracks the memory areas to which an executable binary writes and executes, and so can unpack programs packed by multiple arbitrary packers without requiring reverse-engineering of the packers or any human intervention. By tracking page modification and execution of an executable binary at run time, the packing control module can detect the instant at which the program's control is first transferred to a page whose content is dynamically generated, so AV scanning can then be invoked. Thus, code cannot be executed under the packing control manager without being scanned by an AV scanner first.