DNS Log Analysis for Malware Detection via ML Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Analyzing and searching massive quantities of machine data generated in computing environments is challenging due to the vast types and formats of data from numerous components, requiring efficient data intake and query systems to process and store data for real-time insights.
Innovation Solution
A data intake and query system utilizing a flexible schema and late-binding schema to process and store machine data as events, enabling field-searchability and using machine learning models to identify domain names associated with malware, allowing for real-time operational intelligence and threat detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are applied to analyze massive quantities of machine data, then malware detection capability is improved, but data processing complexity increases
Solution Approach 1:
The patent segments the data processing pipeline into distinct components: data intake, event creation, machine learning model application, and threat detection. Each component handles specific tasks independently, allowing the system to manage complex malware detection through modular processing stages rather than monolithic analysis.
Solution Approach 2:
The patent introduces event data as an intermediary structure between raw machine data and malware detection analysis. Events serve as standardized containers that organize machine data into analyzable units with specific schemas, enabling machine learning models to process heterogeneous data sources through a unified interface and reducing overall system complexity.
2Adaptability or versatility
If all machine data is retained for later analysis, then analytical flexibility is improved, but storage requirements increase
Solution Approach 1:
The patent extracts essential characteristics from raw machine data during the event creation process, storing only the relevant features and metadata needed for analysis rather than retaining complete raw data. This extraction approach preserves analytical flexibility by maintaining key data attributes while significantly reducing storage requirements through selective data retention.
3Measurement precision
If machine learning models process diverse data formats, then detection accuracy is improved, but processing time increases
Solution Approach 1:
The patent transforms diverse machine data formats into a standardized event schema with consistent parameters and structures. By changing the parameter representation of heterogeneous data sources into a unified format, machine learning models can process various data types efficiently without sacrificing detection accuracy, as the standardized parameters enable optimized processing pipelines.
Data Source
AI summary
Techniques are described for providing users of a data intake and query system with pre-trained ML models capable of identifying malicious threats (e.g., malware, botnets, ransomware, etc.) in users' computing environments based on an analysis of Domain Name System (DNS) log data collected from DNS servers in users' environments. DNS log data is ingested by a data intake and query system and processed to obtain searchable timestamped event data. This event data can then be used as input to ML models provided by a security ML application described herein to detect potential occurrences of malicious activity within users' computing environments.