Autonomous Malware Analysis GUI Element Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automation solutions in sandboxes struggle to identify the correct interactive GUI elements in malware, as standard APIs fail to determine which controls can drive malware to the next step, and malware often uses obfuscation techniques to evade detection.
Innovation Solution
A computer-implemented method using multiple algorithms such as API-based, OCR-based, and entropy-based algorithms to detect and determine the likelihood of interactive GUI elements, with a voting algorithm to select the most promising element for interaction, allowing the sandbox to simulate user operations and record interactions for further analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If standard APIs are used to detect GUI elements, then the detection process is simple, but the ability to identify correct interactive elements is insufficient
Solution Approach 1:
The patent combines multiple algorithms (API-based, OCR-based, entropy-based) into a unified detection system. Each algorithm contributes different strengths: API-based provides structural information, OCR-based extracts text content, and entropy-based identifies visual patterns. By merging these algorithms and using a voting mechanism, the system achieves higher accuracy in identifying interactive GUI elements than any single algorithm could alone.
2Measurement precision
If multiple algorithms are used to detect GUI elements, then the identification accuracy improves, but the computational complexity increases
Solution Approach 1:
The system implements a voting mechanism where algorithms contribute to the identification of interactive elements based on their confidence levels. Not all algorithms need to reach full convergence or execute complete analysis for every element. The voting system allows the most promising elements to be identified through partial contributions from multiple algorithms, reducing overall computational burden while maintaining high accuracy.
3Reliability
If malware uses obfuscation techniques, then the malware can evade detection, but the automation system cannot identify correct GUI elements
Solution Approach 1:
The patent employs multiple detection parameters and perspectives to overcome obfuscation. Instead of relying on a single parameter that malware can easily manipulate, the system uses diverse parameters from different algorithms: API-based structural parameters, OCR-based text parameters, and entropy-based visual parameters. This multi-parameter approach makes it difficult for malware obfuscation techniques to evade detection, as they would need to simultaneously manipulate multiple different parameters.
4Productivity
If automation technologies without user interaction are used, then productivity in sandbox is improved, but the ability to handle sophisticated GUI malware is reduced
Solution Approach 1:
The system implements self-service automation by using the malware's own GUI elements and their inherent properties (text content, visual patterns, structural information) to guide the automation process. The algorithms automatically identify interactive elements without requiring external human input, and the system adapts to different malware types by analyzing their specific GUI characteristics. This maintains high productivity while achieving adaptability to sophisticated GUI-based malware.
Data Source
AI summary
A computer-implemented method, a device, and a non-transitory computer-readable storage medium of automatically determining an interactive GUI element in a graphic user interface (GUI) to be interacted. The method includes: detecting, by the processor, one or more candidate interactive GUI elements in the GUI based on a plurality of algorithms; determining, by the processor, a likelihood indicator for each of the one or more candidate interactive GUI elements, a likelihood indicator indicating the likelihood that a candidate interactive GUI element associated with the likelihood indicator is an interactive GUI element to be interacted; and determining, by the processor, an interactive GUI element to be interacted from the one or more candidate interactive GUI elements based on the likelihood indicators.