Statistical N-Gram Signatures for Obfuscated Malware Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing malware detection methods struggle to efficiently identify obfuscated malware due to the reliance on human-intelligible patterns, leading to missed non-human-intelligible elements and limited effectiveness in diverse threat databases.
Innovation Solution
A histogram model and matching rule constructor are used to derive statistically relevant n-grams from unidentified file samples, generating matching signatures without manual string discernment, enabling efficient malware detection across heterogeneous threat databases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human inspection methods are used to identify malware, then human-intelligible patterns can be detected, but non-human-intelligible elements are missed
Solution Approach 1:
The patent replaces manual human inspection with automated computational analysis. The system uses computer-executable instructions to automatically extract and compare n-grams from file samples, eliminating the limitation of human cognitive biases toward intelligible patterns while maintaining high detection accuracy through statistical analysis of both human-intelligible and non-human-intelligible elements.
Solution Approach 2:
The patent transforms the detection approach by changing the parameter of pattern recognition from human-centric intelligibility to statistical significance. By using n-gram frequency analysis and statistical thresholds, the system identifies patterns based on their discriminatory power rather than human interpretability, thereby detecting both human-intelligible and non-human-intelligible malware indicators equally effectively.
2Reliability
If manual inspection methods are used, then expertise-based identification is possible, but the process is high-cost and time-consuming
Solution Approach 1:
The system enables self-service malware detection by automatically performing tasks that previously required expert human inspection. The computer-executable instructions autonomously extract n-grams, compare them against threat databases, and identify malware without human intervention, thereby maintaining expert-level reliability while dramatically improving detection efficiency and reducing costs.
Solution Approach 2:
The patent substitutes the mechanical process of manual expert inspection with an automated computational system. The matching engine and statistical analysis algorithms replicate and enhance human expert capabilities while operating at machine speed, eliminating the trade-off between reliability and productivity by providing both expert-level accuracy and automated efficiency.
3Ease of operation
If human-intelligible character strings are used for comparison, then intuitive identification is achieved, but non-human-intelligible discriminating patterns are deemphasized
Solution Approach 1:
The patent fundamentally changes the parameter of pattern selection from human intelligibility to statistical discriminative power. By using n-gram frequency analysis and significance testing, the system automatically identifies the most discriminating patterns regardless of whether they are human-intelligible, thereby maintaining ease of automated operation while maximizing pattern discrimination accuracy.
Solution Approach 2:
The system replaces human cognitive judgment about pattern intelligibility with computational statistical analysis. The automated process objectively evaluates all patterns by their statistical significance and discriminatory power, eliminating human bias toward intelligible patterns while maintaining operational simplicity through algorithmic consistency and repeatability.
Data Source
AI summary
Methods and systems are provided for a histogram model configuring a computing system to derive an indicator of compromise signature based on a sliding window index of identified malware samples, and a matching rule constructor configuring a computing system to generate matching signatures by selecting statistically relevant n-grams of an unidentified file sample. A matching rule constructor configures the computing system to construct a matching rule including, as a signature, 32 n-grams found in the unidentified file sample which occur most frequently, and another 32 n-grams found in the unidentified file sample which occur least frequently amongst records of the threat database across 32 discrete file size ranges. These functions can configure backend operations to a sample identification operation performed by a user operating a client computing device, in a fashion that does not require a user to manually discern strings from the unidentified file sample to derive a signature for the matching engine to search against the threat database.