Entropy-Based Anomaly Detection for Zero-Day Malware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting malware are ineffective in identifying zero-day threats and generate high false positives, as they rely on static historical data and are not robust enough to detect evolving malware patterns, particularly when data is encrypted or obfuscated.
Innovation Solution
A system and method that divides data files into blocks, calculates entropy values, and uses a machine learning model, specifically a convolutional neural network, to determine the security level of files by generating obfuscation and forensic values, and comparing these with a database to identify anomalies and prevent malicious files from being stored.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If static historical data and traditional malware databases are used for detection, then known malware can be filtered effectively, but zero-day threats and evolving malware patterns cannot be detected
Solution Approach 1:
The patent transitions from static malware databases to dynamic entropy analysis that adapts to new threats. The system continuously evaluates entropy values of file blocks and compares them against evolving patterns, enabling detection of zero-day malware without relying on pre-existing signatures. This dynamic approach allows the system to adapt to changing threat landscapes while maintaining reliable detection of known malware.
Solution Approach 2:
The patent changes the detection parameter from binary malware signatures to continuous entropy values. By analyzing entropy across different file blocks and comparing distributions rather than exact matches, the system can detect novel threats that differ in their statistical properties from known malware, thereby improving adaptability while maintaining detection reliability.
2Measurement precision
If detailed manual analysis of suspicious programs is performed, then thorough detection of malicious behavior is achieved, but the process becomes extremely time consuming and inefficient
Solution Approach 1:
The patent replaces manual mechanical analysis with automated entropy-based detection. Instead of administrators manually examining file blocks and API calls, the system automatically calculates entropy values, generates distributions, and compares patterns using computational algorithms. This substitution maintains thorough detection capability while eliminating the time loss associated with manual analysis.
Solution Approach 2:
The patent creates simplified copies of malware characteristics through entropy representations. Rather than requiring detailed examination of actual malicious code behavior, the system uses entropy distributions as abstract copies that capture essential threat characteristics. This allows rapid automated comparison while preserving the detection thoroughness needed to identify malicious patterns.
3Reliability
If cryptographic hashing is used to compare file characteristics, then known malware can be identified, but slight modifications to malware allow them to slip through defenses
Solution Approach 1:
The patent applies local quality analysis by examining entropy characteristics of individual file blocks rather than treating the entire file as a single hash. By dividing files into blocks and analyzing local entropy distributions, the system can detect malware even when slight modifications occur in specific regions, as the overall entropy pattern remains characteristic of the malware family while allowing for local variations.
Solution Approach 2:
The patent creates a composite detection approach by combining multiple entropy values from different file blocks into an overall distribution pattern. This composite representation is more robust to individual block modifications than a single cryptographic hash, as it captures the aggregate statistical properties of the malware while tolerating minor variations in specific regions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This document describes a system and method for detecting anomalous data files and preventing detected anomalous data files from being stored in a data storage. In particular, the system and method detects anomalous data files by dividing each data file into blocks of data whereby entropy values are obtained for each block of data and this information is collated and subsequently used in a machine learning model to ascertain the security level of the data file.