Hierarchical Novelty Detection for Network Security
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing rule-based detection methods in data centers are inflexible and prone to false positives, struggling to effectively identify and prevent security threats in large-scale environments with numerous unique processes and varying network behaviors.
Innovation Solution
The approach involves observing intended states of processes, generating additional states, clustering them, and employing novelty detection techniques based on cluster sizes to identify and prevent security threats, using methods like tree-based models for large clusters and manual review for small clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If rule-based detection methods are used to identify security threats, then the detection process is simple and fast, but the detection accuracy is low and false positives are high
Solution Approach 1:
The patent segments the detection process into multiple stages: behavior collection, state generation, clustering, and hierarchical novelty detection. Each stage processes specific aspects of process behavior separately, allowing complex analysis to be broken down into manageable components that improve accuracy without overwhelming system complexity
Solution Approach 2:
The patent transitions from traditional rule-based detection to a multi-dimensional approach by generating process states from behavior sequences, clustering them in state space, and applying novelty detection across multiple levels. This dimensional transformation enables the system to capture complex behavioral patterns that simple rules cannot detect
2Adaptability or versatility
If the detection system covers all unique processes and behaviors, then the coverage is comprehensive, but the false positive rate increases
Solution Approach 1:
The patent performs preliminary clustering of process states before novelty detection. By pre-organizing behaviors into clusters representing normal variations, the system establishes a baseline of acceptable behavior patterns. This preliminary action allows the system to distinguish between normal behavioral diversity and actual anomalies, reducing false positives while maintaining comprehensive coverage
Solution Approach 2:
The patent applies different detection strategies to different clusters based on their characteristics. Novelty detection is applied locally within each cluster context, allowing the system to adapt its detection sensitivity to the specific behavior patterns of each process group, thereby maintaining high coverage without uniformly increasing false positives
3Measurement precision
If traditional novelty detection is applied uniformly to all clusters, then the implementation is simple, but the detection accuracy varies across different cluster sizes
Solution Approach 1:
The patent implements a dynamic detection approach where the novelty detection method adapts to the characteristics of each cluster, particularly its size. The system selectively applies different detection techniques based on cluster properties, optimizing accuracy for each scenario while maintaining overall system manageability through hierarchical organization
Data Source
AI summary
The disclosure provides an approach for detecting and preventing attacks in a network. Embodiments include determining a plurality of network behaviors of a process by monitoring the process. Embodiments include generating a plurality of intended states for the process based on subsets of the plurality of network behaviors. Embodiments include determining a plurality of intended state clusters by applying a clustering technique to the plurality of intended states. Embodiments include determining a state of the process. Embodiments include identifying a given cluster of the plurality of intended state clusters that corresponds to the state of the process. Embodiments include selecting a novelty detection technique based on a size of the given cluster. Embodiments include using the novelty detection technique to determine, based on the given cluster and the state of the process, whether to generate a security alert for the process.