Network Endpoint Behavior Modeling via Vector Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for monitoring network behavior of multiple network endpoints require large storage capacities, becoming impractical and costly due to the proliferation of network-connected devices, such as IoT devices.
Innovation Solution
The system reduces network flow records to vectors, allowing for the storage of behavior models for millions of network endpoints using a minimal amount of storage, such as a few gigabytes, by using control circuitry to assign dedicated queues and encode data into multi-dimensional vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If network flow records are stored for all network endpoints to enable behavior monitoring, then monitoring accuracy is improved, but storage requirements increase exponentially
Solution Approach 1:
The patent extracts only the essential behavioral characteristics from complete network flow records by identifying and retaining key fields (source/destination IPs, ports, protocols, timestamps) while discarding redundant data. This extraction process reduces storage requirements while preserving the information necessary for effective behavior monitoring and anomaly detection.
Solution Approach 2:
Instead of storing complete network flow records and then analyzing them, the patent inverts the approach by directly computing behavior models from incoming records in real-time. The system processes records through dedicated queues and generates condensed behavior representations on-the-fly, eliminating the need for massive historical storage while maintaining monitoring capability.
2Adaptability or versatility
If behavior models for millions of network endpoints are stored, then monitoring coverage is improved, but storage infrastructure costs increase
Solution Approach 1:
The patent segments the monitoring system into independent components: dedicated queues for different network endpoints, modular behavior model generators, and separate storage units for condensed models. This segmentation allows the system to handle millions of endpoints by distributing processing across multiple queues and generators, reducing the complexity of any single component while maintaining comprehensive coverage.
Solution Approach 2:
The patent changes the parameters of stored data from complete network flow records (high dimensionality, large size) to condensed behavior models (reduced dimensionality, small size). By transforming the data representation and storing only essential behavioral parameters rather than raw records, the system achieves wide monitoring coverage with minimal storage infrastructure.
3Difficulty of detecting and measuring
If complete network flow records are retained for analysis, then diagnostic capability is improved, but data processing time increases
Solution Approach 1:
The patent applies preliminary action by pre-processing network flow records as they arrive, immediately extracting key fields and computing behavior model updates in real-time. Dedicated queues and behavior model generators process records continuously, maintaining up-to-date behavior models without requiring post-collection processing of massive historical datasets, thus reducing diagnostic time while preserving capability.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Systems and methods are disclosed herein for reducing storage space used in tracking behavior of a plurality of network endpoints by modeling the behavior with a behavior model. To this end, control circuitry may determine a respective network endpoint, of a plurality of network endpoints, to which each respective record of a plurality of received records corresponds. The control circuitry then may assign a dedicated queue for each respective network endpoint, and transmit, to each dedicated queue, each record that corresponds to the respective network endpoint to which the respective dedicated queue is assigned. The control circuitry may then determine, for each respective network endpoint, a respective behavior model, and may store each respective behavior model to memory.