Locality Sensitive Hashing for Malware Variant Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current file evaluation techniques for malware detection face challenges due to the vast number of malware variants and the difficulty in keeping indicators of compromise (IOC) or indicators of attack (IOA) up-to-date, making it complex to effectively identify and classify files for cybersecurity threats.
Innovation Solution
A system utilizing global and local locality sensitive hash (LSH) databases to generate and compare target file hashes, marking files for further evaluation when they do not match known hashes, and employing clustering algorithms to determine similarity and label files as 'good' or 'bad', thereby facilitating malware detection and threat assessment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional cryptographic hash functions (SHA-1) are used for file comparison, then exact malware matches can be detected, but malware variants and mutations cannot be effectively identified
Solution Approach 1:
The patent transforms the exact-match hash comparison parameter into a similarity-based parameter. Instead of requiring identical SHA-1 hashes, the system uses locality-sensitive hashing to compute similarity scores that can identify files with minor modifications, thereby detecting malware variants while maintaining detection accuracy for exact matches
Solution Approach 2:
The patent introduces locality-sensitive hash values as an intermediary between exact cryptographic hashes and malware variant detection. These LSH values serve as a mediator that preserves the precision of exact matching while enabling the detection of similar files through similarity comparison, bridging the gap between exact match and variant detection
2Reliability
If indicators of compromise (IOC) and indicators of attack (IOA) are continuously updated to cover all malware, then detection coverage improves, but the complexity and difficulty of maintenance increases
Solution Approach 1:
The patent implements a self-service mechanism where the system automatically computes locality-sensitive hash values and performs similarity comparisons without requiring manual updates of IOC/IOA databases. The system serves itself by continuously learning from new file samples and automatically adapting to detect variants, reducing maintenance complexity while improving detection coverage
Solution Approach 2:
The patent performs preliminary computation of locality-sensitive hash values for files before they need to be evaluated for malware. By pre-computing and storing these hash values in databases, the system prepares detection capabilities in advance, enabling rapid variant detection without requiring real-time updates of complex IOC/IOA rules
3Measurement precision
If exhaustive comparison of all known malware hashes is performed, then detection accuracy is maximized, but evaluation time and processing speed increase
Solution Approach 1:
The patent applies partial action by performing similarity-based filtering using locality-sensitive hashes before conducting full malware evaluation. This partial comparison quickly identifies potentially malicious files with variant characteristics, allowing the system to focus detailed analysis only on suspicious files rather than performing exhaustive comparisons on all files, thereby reducing evaluation time while maintaining accuracy
Data Source
AI summary
A global locality sensitive hash (LSH) database stores global locality sensitive hashes of files of different private computer networks. Each of the private computer networks has a corresponding local LSH database that stores local locality sensitive hashes of files of the private computer network. A target locality sensitive hash is generated for a target file of a private computer network. The global and local LSH databases are searched for a locality sensitive hash that is similar to the target locality sensitive hash. The target file is marked for further evaluation for malware or other cybersecurity threats when the target locality sensitive hash is not similar to any of the global and local locality sensitive hashes.