Malicious Code Detection via API Call Interception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting malicious code in the address space of a trusted process are inadequate, as they either require extensive monitoring of access operations or only survey limited situations, failing to effectively identify malicious code in random access memory.
Innovation Solution
A system and method that intercept function calls made by a process, determine attributes associated with these calls, and perform malware analysis based on applied rules, including collecting information about the application, identifying the process, and analyzing the code's presence in the address space to determine if it is malicious.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If signature analysis is used to detect malicious programs, then detection accuracy for known malware is improved, but malicious programs can easily hide using polymorphic code and obfuscation techniques
Solution Approach 1:
The system performs preliminary action by injecting a debugger into the target process before the malware can execute its malicious code. This allows the system to set up breakpoints and monitoring points in advance, capturing the malware's behavior as it attempts to execute, thereby detecting polymorphic and obfuscated code that would evade signature-based detection.
Solution Approach 2:
The system uses a debugger as an intermediary between the antivirus application and the target process. The debugger intercepts and monitors API function calls, allowing the system to observe malware behavior indirectly without triggering anti-emulation techniques. This intermediary approach enables detection of malicious code that would otherwise hide from direct analysis.
2Reliability
If heuristic analysis with API function call monitoring is used, then detection of malicious behavior is improved, but the system complexity and resource consumption increase
Solution Approach 1:
The system segments the monitoring process by focusing on specific API function calls related to malicious behavior patterns rather than monitoring all system calls. This selective approach divides the complex monitoring task into manageable segments, reducing system complexity while maintaining reliable detection of malicious behaviors such as unauthorized file operations, registry modifications, and network communications.
3Reliability
If proactive analysis with API function call interception is used, then detection of malicious code execution is improved, but malicious programs can use anti-emulation techniques to detect and evade analysis
Solution Approach 1:
The system uses a debugger as an intermediary that operates at a lower level than traditional emulators. This intermediary approach allows the system to monitor API function calls and detect malicious code execution without triggering anti-emulation techniques, as the debugger presents itself as a legitimate debugging tool rather than an emulation environment.
Solution Approach 2:
The system performs preliminary action by injecting the debugger into the target process and setting up monitoring before the malware executes. This allows the system to capture the malware's behavior as it attempts to execute, detecting anti-emulation techniques before they can trigger evasion mechanisms.
4Reliability
If comprehensive monitoring of access operations is performed to detect malicious code, then detection coverage is improved, but the time required for analysis and false positives increase
Solution Approach 1:
The system segments the monitoring approach by focusing on specific API function calls and behaviors associated with malicious code execution rather than monitoring all access operations. This selective monitoring reduces analysis time and false positives while maintaining comprehensive detection coverage for relevant malicious activities.
Solution Approach 2:
The system applies partial action by monitoring only the most critical API function calls and behaviors that are most likely to indicate malicious code execution. This approach provides sufficient detection coverage without the overhead of comprehensive monitoring of all access operations, thereby reducing analysis time and false positives.
Data Source
AI summary
Disclosed are system and method for detecting malicious code in address space of a process. An exemplary method comprises: detecting a first process executed on the computer in association with an application; intercepting at least one function call made by the first process to a second process; determining one or more attributes associated with the at least one function call; determining whether to perform malware analysis of code associated with the at least one function call in an address space associated with the second process based on application of one or more rules to the one or more attributes; and upon determining to perform malware analysis of the code, determining whether the code in the address space is malicious.