Malware Classification Using Vantage-Point Trees and LSH Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing malware detection methods using file hashes and vantage-point trees are insufficient for detecting variations in malicious files, being computationally intensive and lacking generality, which affects their commercial viability and detection rate.
Innovation Solution
A cyber security method and system employing locality-sensitive hashing evaluation with a vantage-point tree structure for initial malware detection, followed by machine learning-based assessment, optimized for speed and computation using non-recursive VPT search and top-K heap operations, to identify files as malicious or non-malicious with high confidence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional file hashing algorithms (MD5, SHA-1, SHA-2) are used for malware detection, then the detection process is simple and fast, but the detection rate is insufficient for detecting variations in malicious files
Solution Approach 1:
The patent segments the detection process into multiple hierarchical levels: first-level detection using traditional hashing algorithms for simple cases, second-level detection using locality-sensitive hashing with vantage-point trees for fuzzy matching, and third-level detection using machine learning models for complex variants. This segmentation allows the system to maintain high detection rates while managing complexity through progressive filtering.
Solution Approach 2:
The patent creates a composite detection system that combines multiple detection methodologies (traditional hashing, locality-sensitive hashing, vantage-point tree structures, and machine learning models) into a unified framework. Each component contributes its strengths to the overall system, achieving high reliability against various malware types while balancing computational complexity.
2Reliability
If vantage-point tree structure is used for locality-sensitive hashing evaluation, then the detection accuracy for malware variants is improved, but the computational intensity increases
Solution Approach 1:
The patent performs preliminary filtering using traditional hashing algorithms before applying the computationally intensive vantage-point tree structure. Files that match known malware hashes are identified and handled immediately, while only files requiring fuzzy matching proceed to the LSH evaluation stage. This preliminary action reduces the overall computational burden.
Solution Approach 2:
The patent applies LSH evaluation selectively rather than to all files. The system determines whether a file requires LSH processing based on initial hashing results and file characteristics, applying the computationally intensive method only when necessary to detect variants, thus avoiding excessive computation on clearly identifiable malware or benign files.
3Reliability
If comprehensive malware database is maintained to detect all malicious files, then the detection coverage is improved, but the memory requirements and search time increase
Solution Approach 1:
The patent segments the malware database into multiple hierarchical levels with different granularities. The first level contains traditional hash values for exact matching, the second level contains locality-sensitive hashes organized in vantage-point trees for fuzzy matching, and the third level contains machine learning models for pattern recognition. This segmentation enables efficient search by processing queries at appropriate levels without scanning the entire database.
Solution Approach 2:
The patent introduces locality-sensitive hashing as an intermediary between traditional exact matching and comprehensive machine learning analysis. The LSH with vantage-point trees acts as a mediator that quickly identifies potential matches and variants without requiring full database scans, reducing search time while maintaining detection coverage.
4Reliability
If recursive VPT search is used for thorough evaluation, then the search completeness is improved, but the memory usage increases
Solution Approach 1:
The patent implements a dynamic VPT search approach that adapts the search depth and scope based on file characteristics and initial evaluation results. The search begins with a shallow evaluation and dynamically extends to deeper levels only when necessary, adjusting memory allocation accordingly. This dynamic approach maintains search completeness while optimizing memory usage based on actual needs.
Solution Approach 2:
The patent changes key parameters of the VPT search process, including the threshold for initiating LSH evaluation, the depth of tree traversal, and the number of candidate files to evaluate. By dynamically adjusting these parameters based on file characteristics and system resources, the system maintains thorough search capability while controlling memory consumption.
Data Source
AI summary
A cyber security method and system for detecting malware via an anti-malware application employing a fast locality-sensitive hashing evaluation using a vantage-point tree (VPT) structure for the indication of malicious files and non-malicious files. The locality-sensitive hashing evaluation using the VPT structure can be performed prior to initiating the deeper, more computationally intensive evaluation and is used to identify with high confidence a scanned file or data object being (i) a malicious file, (ii) a non-malicious file, or a low confidence measure of the two.