Obfuscated Executable Detection via Instruction Disassembly
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional security software struggles to detect obfuscated executables, particularly those with low entropy, as they evade detection by maintaining random patterns, rendering them undetectable using traditional entropy checks.
Innovation Solution
A security module disassembles executable files to identify valid and illogical instructions, evaluating sections to determine if they pose a security risk by calculating ratios of invalid and illogical instructions, and considering both when determining if the executable file is obfuscated.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional security software uses entropy checks to detect obfuscated executables, then detection capability is improved for high-entropy files, but detection capability deteriorates for low-entropy obfuscated files that maintain random patterns
Solution Approach 1:
The patent divides the executable file into multiple sections and analyzes each section independently for valid and invalid instructions. This segmentation allows the security module to detect obfuscation patterns within specific sections even when overall entropy is low, resolving the contradiction between detecting high-entropy files and low-entropy obfuscated files.
Solution Approach 2:
The patent introduces an intermediary analysis layer between the executable file and the detection system. Instead of directly analyzing entropy, the security module uses a disassembly intermediary to translate machine code into assembly instructions, then evaluates validity and logic to detect obfuscation. This intermediary approach enables detection of low-entropy obfuscated files while maintaining capability for high-entropy files.
2Measurement precision
If security software disassembles and analyzes instructions to detect obfuscation, then detection accuracy for obfuscated files is improved, but processing time and computational resources increase
Solution Approach 1:
The patent segments the executable into sections and analyzes only the first section for validity, then evaluates additional sections only if needed. This selective segmentation reduces processing time compared to analyzing the entire file, while maintaining high detection accuracy for obfuscated executables.
Solution Approach 2:
The patent applies partial action by analyzing only the first section initially and determining security risk based on that section's instruction validity. Additional sections are analyzed only if the first section indicates potential obfuscation. This partial analysis approach reduces computational resources while maintaining sufficient detection accuracy.
3Reliability
If the security module evaluates multiple sections of the executable file, then detection reliability is improved, but device complexity and processing overhead increase
Solution Approach 1:
The patent segments the executable into multiple sections and establishes a systematic evaluation process for each section. This segmentation with clear evaluation criteria improves detection reliability by comprehensive analysis, while the structured approach prevents uncontrolled complexity increase.
Solution Approach 2:
The patent implements feedback mechanisms where the analysis of the first section informs whether additional sections need evaluation. The ratio of invalid to total instructions calculated from the first section provides feedback on security risk, determining the extent of further analysis required. This feedback loop improves reliability while managing processing complexity.
Data Source
AI summary
A computer-implemented method for detecting an obfuscated executable may include identifying an executable file programmed to execute on a target architecture. The method may also include disassembling a first section of the executable file and determining whether the first section of the executable file comprises a valid instruction. The method may further include determining, based on whether the first section of the executable file comprises a valid instruction, whether the executable file poses a security risk. Various other methods, computer-readable media, and systems are also disclosed.