Dictionary DGA Detection Pipeline With Filtering, Caching, and Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting dictionary DGA domain names are inefficient due to their resemblance to legitimate domain names, leading to increased difficulty in detection and higher computational costs and latency.
Innovation Solution
A low-latency and cost-efficient detection system using a trained machine learning model that filters out non-dictionary DGA domain names through passive DNS data and heuristic analysis, caching domain names for reduced processing, and load balancing to minimize model input, with GPU processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a trained machine learning model is used to detect dictionary DGA domain names, then detection accuracy is improved, but computational cost and latency increase
Solution Approach 1:
The detection system is segmented into multiple stages: a filtering stage that quickly eliminates obviously benign domain names using simple rules, and a classification stage that applies the trained machine learning model only to suspicious domain names. This segmentation allows the system to maintain high detection accuracy while reducing the computational burden and latency associated with applying the full model to all domain names.
Solution Approach 2:
The system performs preliminary filtering of domain names using passive DNS data and heuristic analysis before applying the trained machine learning model. By pre-processing and eliminating obviously benign domain names in advance, the system reduces the number of inputs to the computationally expensive model, thereby reducing latency and computational cost while maintaining detection accuracy for suspicious domains.
2Reliability
If all domain names are input into the trained model for classification, then detection completeness is improved, but computational cost increases
Solution Approach 1:
The system extracts and removes obviously benign domain names from the input stream before they reach the trained machine learning model. By taking out these non-suspicious domain names through filtering based on passive DNS data and heuristic analysis, the system maintains detection completeness for malicious domains while significantly reducing the computational cost of processing all domain names through the full model.
Solution Approach 2:
Instead of applying the full machine learning model to all domain names (excessive action), the system applies the model only to the subset of domain names that pass the filtering stage and are deemed suspicious (partial action). This approach maintains detection completeness for malicious domains while avoiding the unnecessary computational cost of processing obviously benign domain names through the expensive model.
3Productivity
If dictionary DGA domain names are detected using traditional methods, then processing speed is maintained, but detection accuracy decreases due to resemblance to legitimate domain names
Solution Approach 1:
The system introduces an intermediary filtering layer between the incoming domain names and the trained machine learning model. This intermediary layer uses passive DNS data and heuristic analysis to pre-process domain names, eliminating obviously benign ones before they reach the model. This intermediary filtering maintains processing speed by quickly eliminating easy cases while improving detection accuracy by ensuring the model focuses on suspicious domain names that require its advanced analysis capabilities.
Data Source
AI summary
A service includes a trained model comprising a classifier that predicts whether domain names are dictionary DGA generated. Using passive DNS data and/or a heuristic analysis based on natural language processing of the domain name, the service filters domain names that are not candidate (i.e., potential) dictionary DGA domain names out of the detection pipeline. There domain names are thus classified without being fed into the model for more computationally expensive processing. Domain names that are not filtered out are queued for input into an instance of the model and classification by the model, with the queued domain names processed in small batches and load balanced across model instances. Predicted domain name classes output by the model are cached for subsequent cache reads to avoid multiple runs of the model for one domain name.