Shellcode Detection via Function Call Redirection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods fail to effectively detect malicious shellcode injected into computer systems, particularly in web browsers, due to techniques like heap spraying and self-decoding JavaScript, which make it difficult to identify and prevent execution before it causes harm.
Innovation Solution
Redirecting function calls from decoding functions, such as the JavaScript Unescape function, to a scanning function that checks for suspect code or data, allowing for early detection and inhibition of malicious activity, thereby preventing shellcode execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If heap spraying technique is used to inject shellcode into memory, then the likelihood of successful code execution is improved, but the difficulty of detecting and preventing the attack increases
Solution Approach 1:
The patent applies preliminary action by intercepting and scanning data before it is decoded and executed. The scanning function analyzes the compressed or encoded data in its original form, before JavaScript engines or decoding functions transform it into executable shellcode. This prevents the attack at an early stage, countering the reliability improvement from heap spraying by detecting malicious content before it can be successfully executed.
2Adaptability or versatility
If self-decoding JavaScript is used to obfuscate shellcode, then the ability to evade signature-based scanning is improved, but the complexity of the malicious code increases
Solution Approach 1:
The patent introduces an intermediary scanning function that sits between the data reception and the decoding/execution process. This intermediary analyzes the compressed or encoded data before it is processed by JavaScript engines or decoding functions. The scanning function acts as a mediator that can detect malicious content regardless of obfuscation techniques, countering the adaptability improvement from self-decoding JavaScript by maintaining detection capability despite increased code complexity.
3Adaptability or versatility
If shellcode is injected into the heap memory using compressed or encoded data, then the ability to evade simple signature scanning is improved, but the risk of detection by advanced scanning methods increases
Solution Approach 1:
The patent applies preliminary action by performing security scanning on compressed or encoded data before it is decoded and executed. The scanning function analyzes the data in its original compressed form, identifying malicious patterns before they are transformed into executable shellcode. This approach maintains the ability to evade simple signature scanning while reducing detectability by advanced scanning methods, as the scan occurs at the earliest possible point in the attack chain.
Data Source
AI summary
A method of detecting malware present on a computer system where the computer system is running an application. The method includes redirecting a function call, made by the application to a decoding function that performs decoding of an argument provided to it by an application, to a scanning function. The scanning function is then employed to scan an argument of the function call for suspect code or data. In the event that suspect code or data is detected, the function call is inhibited, otherwise program control is returned to the called decoding function.