Archetypal Analysis Reconstructs Sampled Network Activity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Monitoring and analyzing large volumes of network traffic in cloud-computing platforms is challenging due to the vast number of entities, making it difficult to detect network anomalies and maintain security effectively, as existing solutions require significant resources and expert knowledge, leaving network activity unexplained and potentially leading to missed attacks or vulnerabilities.
Innovation Solution
A network activity monitor reconstructs network activity by determining vertices in a multidimensional space, assigning usage patterns to each vertex, and representing entities as a weighted combination of these patterns, enabling detailed analytics and anomaly detection without requiring ad-hoc solutions or labeled data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If packet sampling is implemented to reduce data volume, then resource consumption is reduced, but the ability to accurately reconstruct network usage deteriorates
Solution Approach 1:
The patent introduces an intermediary reconstruction system that uses machine learning models to bridge the gap between sampled data and actual network usage. The system employs intermediate representations (embeddings) that capture essential network flow characteristics, allowing accurate reconstruction of network usage patterns from limited sampled data without requiring full packet capture
Solution Approach 2:
The patent creates simplified copies of network traffic data through sampling, then uses machine learning models to generate reconstructed copies of the original network usage patterns. These reconstructed copies preserve the essential characteristics and patterns of actual network traffic while requiring minimal storage and processing resources
2Measurement precision
If ad-hoc machine learning classifiers are constructed for each network usage type, then specific usage detection capability is improved, but device complexity and time investment increase
Solution Approach 1:
The patent implements a universal machine learning framework that can detect and classify multiple types of network usage patterns simultaneously. The system uses a single trained model that processes various network flow types (web browsing, file transfers, video streaming, etc.) through a unified approach, eliminating the need to develop and maintain separate classifiers for each usage type
Solution Approach 2:
The patent transforms network flow data into standardized parameter representations (embeddings) that capture essential characteristics across different usage types. By changing the representation parameters to a unified feature space, the system can apply a single classification model to diverse network activities, reducing complexity while maintaining detection accuracy
3Measurement precision
If labelled data is required for training classifiers, then detection accuracy for known patterns is improved, but adaptability to unknown network activities deteriorates
Solution Approach 1:
The patent implements self-service through unsupervised and semi-supervised learning approaches where the system automatically learns network usage patterns from raw data without requiring extensive manual labelling. The machine learning models autonomously identify patterns, cluster similar activities, and adapt to new usage types as they appear in the network traffic, enabling both high accuracy for known patterns and adaptability to unknown activities
Data Source
AI summary
Methods, systems, apparatuses, and computer program products are provided for reconstructing network activity. A network activity monitor is configured to monitor network activity for various network entities. Based on the monitoring, a set of features may be obtained for each network entity. A determination may be made for a number of vertices suitable for describing the sets of features in a multidimensional space. In some implementations, the vertices may define a convex hull in the multidimensional space. Each of the vertices may be assigned a different usage pattern that represents a certain type of network usage types. Reconstructed network activity for a particular network entity may be represented as a weighted combination of the usage patterns. Based on the reconstruction, a network anomaly may be detected, a network may be modified, and/or an alert may be generated.