Obfuscated Malware Detection in JIT Compiled Code
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing malware detection systems struggle to identify obfuscated malware in Just-in-Time (JIT) compiled software due to obfuscation techniques, leading to difficulties in static analysis and pattern matching, resulting in false positives and negatives.
Innovation Solution
The proposed solution involves dynamic execution of JIT compiled code in a monitored environment, where runtime calls are analyzed using techniques such as signature matching, heuristic matching, n-gram analysis, and recurrent neural networks to detect malware, and metadata is hashed and compared to a cache of previously analyzed code to identify and deobfuscate malicious code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If rule-based security systems are used for malware detection, then detection speed is improved, but detection accuracy deteriorates due to obfuscation
Solution Approach 1:
The system transitions from static rule-based analysis to dynamic runtime behavior analysis. By executing code in a controlled environment and monitoring actual runtime calls, the system adapts to obfuscated malware that changes its appearance statically while maintaining consistent behavioral patterns during execution.
Solution Approach 2:
A virtualized execution environment serves as an intermediary between the malware sample and the analysis system. This sandbox environment allows safe execution and observation of runtime behavior without risking the host system, enabling detection of obfuscated malware through its actual operational characteristics rather than static patterns.
2Reliability
If static analysis and pattern matching are used, then false positives are reduced, but detection capability against obfuscated malware deteriorates
Solution Approach 1:
The system monitors dynamic runtime behavior including function calls, system calls, and execution flow patterns. This dynamic approach reveals the true intent of obfuscated malware during execution, overcoming the limitations of static analysis where obfuscation hides malicious patterns.
Solution Approach 2:
The system replaces mechanical pattern matching with behavioral observation. Instead of searching for specific code patterns that may be obfuscated, the system observes what the code actually does during execution, substituting static pattern recognition with dynamic behavior monitoring.
3Difficulty of detecting and measuring
If deobfuscation techniques are implemented, then detection capability is improved, but system complexity and false positives increase
Solution Approach 1:
The virtualized execution environment acts as an intermediary that naturally deobfuscates code through execution. Malware authors cannot effectively obfuscate behavior that must manifest during runtime, so the execution environment reveals true intent without requiring complex deobfuscation algorithms.
Solution Approach 2:
The system allows malware to execute and reveal its own behavior without external intervention for deobfuscation. The malware's own execution serves to expose its malicious patterns, eliminating the need for separate complex deobfuscation processes that increase system complexity.
4Measurement precision
If dynamic execution in monitored environment is used, then detection accuracy is improved, but processing time increases
Solution Approach 1:
The analysis process is segmented into distinct phases: initial hash comparison for quick rejection of known safe code, followed by selective dynamic execution only for suspicious or unknown samples. This segmentation reduces overall processing time by avoiding full dynamic analysis of all code.
Solution Approach 2:
The system performs partial dynamic execution, monitoring only critical runtime behaviors and function calls necessary for detection rather than complete execution. This partial action approach maintains high detection accuracy while reducing the time overhead of dynamic analysis.
Data Source
AI summary
Systems and method for the detection of obfuscated malware in JIT code are provided. JIT code portions may be retrieved from a repository, and metadata associated with the JIT code may be determined. The JIT code and associated metadata may be hashed to determine a hash value that is compared to a cache of hash values corresponding to previously analyzed JIT code portions. If the JIT code portion has not been previously analyzed the JIT code portion may be parsed into a byte stream for execution in a JIT runtime environment in a virtual machines. The runtime data may be analyzed using signature matching, heuristic rule matching, n-gram analysis, and RNN analysis. If malware is detected, the JIT code portion, associated metadata, and analysis results may be provided for generation of an alert.