Network Threat Detection via Dynamic Clustering and Partial Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In network security, it is challenging to detect threatful behavior in network infrastructure due to difficulties in collecting information from network entities that may be offline, damaged, busy, improperly configured, or compromised, making it hard to identify threatful behavior without complete data.
Innovation Solution
A network security monitor uses a dynamic clustering approach combined with pattern recognition techniques like k-Nearest Neighbors (k-NN) to detect threats using partial information, generating clusters from collected data and assigning new records to existing clusters based on matching processes involving fuzzy logic and weighted attributes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complete information is collected from all network entities, then threat detection accuracy is improved, but information collection reliability deteriorates due to entities being offline, damaged, busy, improperly configured, or compromised
Solution Approach 1:
The patent applies partial action by using only the subset of available information from entities that can be successfully monitored, rather than requiring complete information from all entities. The system processes partial data from available sources to detect threats, accepting that not all network entities will provide information due to various failure modes.
Solution Approach 2:
The patent introduces an intermediary information aggregation layer that collects data from multiple network entities and synthesizes available information. This intermediary layer handles the uncertainty of partial information collection by aggregating data from multiple sources and using pattern recognition to detect threats even when not all sources are available.
2Adaptability or versatility
If information is collected from multiple network entities, then threat detection capability is improved, but system complexity increases due to handling partial and incomplete data
Solution Approach 1:
The patent creates a universal information processing framework that handles multiple types of network entities and multiple failure modes through a single pattern recognition system. The k-NN classifier and clustering algorithms provide multi-functional capability to process diverse data sources and detect various threat types using the same underlying mechanism.
Solution Approach 2:
The patent uses parameter changes by adjusting the number of neighbors (k) in the k-NN algorithm and modifying cluster parameters dynamically based on the available data. The system adapts its detection parameters based on the quality and quantity of information received from network entities, allowing flexible operation under varying conditions.
3Measurement precision
If cluster-based pattern recognition is used to classify behaviors, then threat detection with partial information is improved, but computational requirements increase
Solution Approach 1:
The patent applies preliminary action by pre-computing clusters from historical data and pre-establishing patterns of normal and abnormal behavior. The k-NN classifier uses pre-computed distance metrics and cluster centroids, allowing rapid classification of new behaviors without performing complex computations in real-time.
Solution Approach 2:
The patent uses copying by creating cluster representations that capture the essential characteristics of groups of similar behaviors. Instead of comparing new behaviors against all historical data, the system copies key features into cluster prototypes and compares against these simplified representations, reducing computational load while maintaining detection accuracy.
Data Source
AI summary
Detecting threats in a network are provided. A network security monitor obtains records for entities that access a network. The records include attributes associated with the entities. The network security monitor generates clusters from the records using a matching process. The network security monitor classifies a first cluster as a threat cluster. The network security monitor receives, subsequent to generating the plurality of clusters, a record from an entity that access the network. The network security monitor assigns the record to the first cluster using the matching process. The network security monitor detects, responsive to assigning the record to the first cluster, a threat associated with the entity.