Asynchronous Hidden Markov Models for APT Metadata Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems fail to effectively detect Advanced Persistent Threats (APT) and malicious software due to their stealthy and adaptive nature, which evades conventional detection methods by using low bandwidth communication and obfuscation techniques, making it difficult for existing Network Intrusion Detection Systems (NIDS) to identify and alert on APT activity.
Innovation Solution
The implementation of Asynchronous Hidden Markov Models for Internet Metadata Analytics (AHIMA) that utilizes machine learning and AI to analyze network traffic metadata, constructing a multi-partite graph to represent network infrastructure and encode states, enabling the detection of anomalies and malicious activity through incremental signal transformation and lossy compression, and providing real-time alerting and forensic analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional NIDS techniques are used for detection, then system simplicity is maintained, but detection capability against APT fails due to stealthy and adaptive nature of APT malware
Solution Approach 1:
The system segments the detection problem into multiple components: behavioral machine analytics for detecting malicious behavior, cyber hunting analytics for facilitating investigation, and multi-partite graph representation of network infrastructure. This segmentation allows each component to specialize in specific detection tasks, improving overall reliability while managing complexity through modular architecture.
Solution Approach 2:
The system employs dynamic Bayesian networks and asynchronous hidden Markov models that continuously adapt to new network metadata and update probability distributions in real-time. This dynamic adaptation enables the system to track evolving APT behaviors and maintain high detection capability as threats change over time.
2Object-affected harmful factors
If APT malware operates infrequently with low bandwidth communication, then stealth is improved, but detection by conventional systems becomes more difficult
Solution Approach 1:
The system introduces an intermediary layer of behavioral analytics that observes network metadata patterns rather than directly inspecting malware communications. This intermediary approach detects anomalies in communication behavior patterns, making it possible to identify stealthy APT activities without requiring direct observation of the malicious traffic itself.
Solution Approach 2:
The system changes the detection parameters from looking for specific malicious signatures to monitoring behavioral parameters such as communication frequency, bandwidth usage patterns, and temporal metadata characteristics. This parameter transformation enables detection of infrequent, low-bandwidth APT communications by identifying deviations from normal behavioral patterns.
3Adaptability or versatility
If APT uses obfuscation and modification techniques, then evasiveness is improved, but signature-based detection becomes ineffective
Solution Approach 1:
Instead of trying to match known malware signatures, the system inverts the approach by establishing baseline behavioral patterns for normal network activity and detecting deviations from these baselines. This inversion makes the system immune to obfuscation and modification techniques, as it detects based on behavioral anomalies rather than signature patterns.
Solution Approach 2:
The system transforms the detection problem from signature-based pattern matching to behavioral parameter analysis, monitoring changes in communication patterns, timing, and metadata characteristics. This parameter transformation renders obfuscation and modification techniques ineffective, as the underlying behavioral patterns remain detectable even when the malware code is obscured.
4Reliability
If NIDS processes large amounts of network metadata in real-time, then detection coverage is improved, but processing speed and memory requirements increase
Solution Approach 1:
The system extracts only the relevant features and metadata from network traffic, storing compressed representations in a multi-partite graph structure. This extraction approach maintains comprehensive detection coverage by preserving essential behavioral patterns while reducing the volume of data that requires real-time processing, thereby improving processing speed.
Solution Approach 2:
The system employs a nested architecture where asynchronous hidden Markov models operate within a dynamic Bayesian network framework, which itself is embedded in a multi-partite graph representation of the network. This nested structure allows efficient processing by organizing computational tasks at multiple levels, enabling real-time analysis of large metadata volumes without overwhelming system resources.
Data Source
AI summary
Implementations described and claimed herein provide systems, methods and computer-readable media with instructions for detecting anomalies in computer network traffic online, real-time, historical, forensic, and/or playback mode. The implementations can include monitoring network traffic metadata, parsing the metadata, constructing a multi-partite graph of nodes and edges based on a long-term incremental signal transformation or a short-term concurrent snapshot, and generating streaming analytics based on the multi-partite graph representing a likelihood that network traffic associated with a specified network component is infected with malware.