Inline Malware Detection Using Filetype-Specific N-Gram Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing malware detection systems face challenges in efficiently identifying and mitigating newly created malware due to resource constraints and the need for real-time analysis, especially in environments where signatures are not immediately available.
Innovation Solution
Implementing machine learning models on data appliances for inline malware detection, utilizing features like n-grams and classification models, such as logistic regression and gradient boosting trees, to provide initial maliciousness verdicts while awaiting signatures from a security platform.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If signature-based malware detection is used, then detection accuracy is improved, but response time deteriorates due to waiting for signature updates
Solution Approach 1:
The system performs preliminary analysis by extracting features (n-grams, file metadata, behavioral characteristics) from suspicious files and compares them against known malware patterns before formal signature verification is complete. This preliminary action enables early detection and response while signature updates are being prepared, resolving the contradiction by providing accurate detection capability without waiting for signature updates.
2Reliability
If comprehensive malware analysis is performed, then detection capability is improved, but resource consumption increases
Solution Approach 1:
The malware detection system is segmented into multiple independent components: feature extraction module, pattern matching module, signature verification module, and response module. Each component performs a specific function with optimized resource usage. The feature extraction and pattern matching can operate independently with lower resource requirements, providing detection capability while managing resource consumption through functional segmentation.
3Speed
If real-time malware detection is implemented, then response speed is improved, but system complexity increases
Solution Approach 1:
The system introduces an intermediary layer consisting of pre-computed feature templates and pattern databases that mediate between the incoming file and the detection engine. Instead of performing complex analysis from scratch for each file, the intermediary patterns enable rapid comparison and matching, achieving real-time response speed while managing system complexity through the use of pre-prepared reference data structures.
Data Source
AI summary
Detection of malicious files is disclosed. A set comprising a plurality of sample classification models is received and stored. A determination is made that n-gram analysis should be performed on a sequence of received packets associated with a received file. Performing the n-gram analysis includes using a determined filetype associated with the sequence of received packets to select at least one stored sample classification model included in the set for use in performing the n-gram analysis. A determination is made that the received file is malicious based at least in part on the n-gram analysis of the sequence of received packets. In response to determining that the file is malicious, propagation of the received file is prevented.