Encrypted File Classification for Low-Latency Exfiltration Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data loss prevention (DLP) systems struggle to detect encrypted files, which are often used to evade detection and exfiltrate sensitive data, and also face challenges in identifying ransomware-encrypted devices without significantly impacting network performance.
Innovation Solution
A machine learning-based encrypted file classification system that uses behavior analytics and statistical tests to determine the randomness of file content, classifying files as encrypted or unencrypted, and adjusts user confidence scores based on anomalous behavior patterns to detect and block data exfiltration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional data loss prevention technologies are used to detect encrypted files, then detection capability is improved, but network performance degradation increases
Solution Approach 1:
The system segments the file inspection process into two stages: a fast first stage that analyzes metadata and file properties to identify encrypted files, and a second stage that performs deep content inspection only on files suspected of being encrypted. This segmentation allows the system to maintain high detection capability for encrypted files while minimizing the performance impact of inspection on all file types.
Solution Approach 2:
The system performs preliminary classification of files as encrypted or unencrypted using machine learning models and statistical tests before initiating full content inspection. This preliminary action filters out the majority of unencrypted files that require full inspection, allowing the system to focus computational resources only on potentially encrypted files, thus maintaining both detection accuracy and network performance.
2Measurement precision
If encryption detection accuracy is improved, then false positive rate increases
Solution Approach 1:
The system employs multiple statistical parameters and machine learning models that analyze different characteristics of file data, including entropy, distribution patterns, and metadata properties. By changing and combining multiple detection parameters, the system achieves high accuracy in identifying encrypted files while reducing false positives through sophisticated thresholding and voting mechanisms.
Solution Approach 2:
The system incorporates feedback loops where detection results are continuously refined based on observed patterns and user reports. The machine learning models are trained on feedback data to improve their accuracy over time, allowing the system to adapt to new encryption methods while maintaining low false positive rates through iterative improvement.
Data Source
AI summary
The disclosed technology facilitates User and Entity Behavior Analytics (UEBA) by classifying a file being transferred as encrypted or not. The technology involves monitoring movement of a files by a user over a wide area network, detecting file encryption for the files using a trained classifier, wherein the detecting includes processing by the classifier some or all of the following features extracted from each of the files: a chi-square randomness test; an arithmetic mean test; a serial correlation coefficient test; a Monte Carlo-Pi test; and a Shannon entropy test, counting a number of the encrypted files moved by the user in a predetermined period, comparing a predetermined maximum number of encrypted files allowed in the predetermined period to the count of the encrypted files moved by the user and detecting that the user has moved more encrypted files than the predetermined maximum number, and generating an alert.