Network Device Anomaly Detection with Dual Embeddings and LLM Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection systems in cybersecurity often produce high false positive detections of anomalous devices, leading to unnecessary alerts and resource wastage.
Innovation Solution
A two-stage detection method involving dual embedding generation, clustering, and verification using a language model to reduce false positives, where dual embeddings capture both character-level and sentence-level patterns, followed by clustering to identify outliers, and a pre-trained LLM verifies the anomalousness of these outliers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional anomaly detection systems are used, then detection speed is maintained, but false positive detections increase leading to resource wastage
Solution Approach 1:
The detection process is segmented into two distinct stages: a first stage that performs initial anomaly detection using clustering algorithms on device profiles, and a second stage that verifies suspected anomalies using a language model. This segmentation allows the system to maintain high detection accuracy while reducing false positives by having the second stage filter out spurious detections from the first stage.
Solution Approach 2:
A language model serves as an intermediary verification mechanism between the initial clustering-based detection and the final anomaly confirmation. The language model acts as a mediator that reviews and validates the anomalies identified in the first stage, providing a human-like understanding of device profiles to distinguish true anomalies from false positives before final alert generation.
2Measurement precision
If dual embedding generation is implemented, then measurement precision improves, but device complexity increases
Solution Approach 1:
The system merges two different embedding approaches into a unified dual embedding representation: character-level embeddings that capture n-gram patterns and sentence-level embeddings that capture semantic meaning. By combining these two perspectives through concatenation, the system achieves more precise device profile representation without requiring a single overly complex embedding model.
Solution Approach 2:
The dual embedding approach adds another dimension to the representation space by incorporating both character-level and sentence-level embeddings. This dimensional expansion allows the system to capture multiple aspects of device profiles simultaneously, improving measurement precision by representing devices in a richer, multi-faceted vector space.
Data Source
AI summary
An anomalous device detection service performs two-stage detection of anomalous devices in a network with verification of detected anomalies to reduce the incidence of false positive detections. The detection service generates a dual embedding for each device profile. The dual embedding comprises a sentence embedding and a character-based embedding that have been concatenated. The detection service clusters the dual embeddings and, from the resulting cluster(s), identifies outliers that correspond to candidate anomalous device profiles. The outliers are referred to as candidates at this stage because the detection service then verifies the verdicts of anomalousness resulting from clustering using an LLM was adapted to predict if a device profile is actually anomalous based on examples of anomalous and non-anomalous device profiles that were provided to the LLM with few-shot prompting. Devices corresponding to profiles that the LLM predicts are anomalous are flagged for further investigation and/or remediation.