Hierarchical Flow Statistics Aggregation for Scalable Network Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for collecting and storing network traffic statistics from a large number of network appliances in a communication network result in unmanageable data volumes, leading to significant network traffic burdens and prolonged query times due to the large amount of data being processed and analyzed.
Innovation Solution
A system and method for aggregating select network traffic statistics by building hierarchical strings of flow attributes, extracting network metrics, and transmitting condensed information to a network information collector, reducing the amount of data transmitted and stored.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all network traffic statistics are collected and stored for each network appliance, then complete network monitoring capability is achieved, but data volume becomes unmanageable and network burden increases
Solution Approach 1:
The patent extracts only the most relevant and frequently queried network traffic statistics from the complete set of flow attributes. Instead of collecting all possible flow data, the system identifies and collects only those attributes that are most useful for network monitoring and analysis, thereby reducing data volume while maintaining essential monitoring capabilities
Solution Approach 2:
The patent segments network traffic statistics into hierarchical groups based on attribute relationships. By organizing flow attributes into hierarchical structures and selecting representative statistics from each segment, the system reduces the overall data volume while preserving the ability to analyze different aspects of network traffic independently
2Loss of information
If all network traffic statistics are transmitted to the collection engine, then complete data availability is achieved, but network bandwidth consumption increases
Solution Approach 1:
The patent extracts only the essential network traffic statistics that are most commonly needed for network monitoring and analysis. By transmitting only these extracted attributes rather than complete flow data, the system maintains data availability for critical monitoring functions while significantly reducing network bandwidth consumption
3Adaptability or versatility
If complete network traffic data is stored in the database, then comprehensive query capability is achieved, but query processing time increases
Solution Approach 1:
The patent extracts and stores only the network traffic statistics that are most frequently queried and most relevant to network monitoring needs. By maintaining a database of these extracted attributes rather than complete flow data, the system achieves comprehensive query capability for essential monitoring functions while reducing query processing time due to the smaller data set
Solution Approach 2:
The patent organizes stored network traffic statistics into hierarchical segments that enable efficient query processing. By segmenting the data structure to match common query patterns and organizing attributes hierarchically, the system maintains versatile query capability while improving query performance through optimized data access paths
4Measurement precision
If detailed flow attributes are collected for each network appliance, then granular network analysis is achieved, but data management complexity increases
Solution Approach 1:
The patent segments flow attributes into hierarchical groups and selects representative statistics from each segment. This segmentation approach maintains granular analysis capability by preserving important attribute relationships while reducing data management complexity through organized, hierarchical data structures that are easier to process and maintain
Data Source
AI summary
Disclosed herein are systems and methods for the collection, aggregation, and processing of network traffic statistics for a plurality of network appliances in a wide area network. Select network traffic statistics can be collected and associated with a hierarchical string, and aggregated over time. In this way, only information that is likely to be relevant is gathered and maintained, allowing for the maintenance of select network traffic statistics for large-scale operations.