Heap-Ordered Packet Flow Representation for Network Traffic Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network analysis devices face challenges in real-time analysis and reporting of traffic flow statistics due to continuously updating data and high network traffic volumes, especially when handling multiple concurrent queries from users.
Innovation Solution
Implementing a network device with traffic flow accounting functionality that uses heap tables to maintain packet flow identifiers, allowing for real-time query resolution by cloning the heap table for processing while continuing to update the original statistics, and applying algorithms like truncated heap sort to efficiently retrieve the required statistics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If network analysis devices continuously update traffic flow statistics in real-time, then the data remains current and useful for monitoring, but it becomes difficult to analyze and respond to user queries efficiently
Solution Approach 1:
The patent divides the continuously updating traffic flow data into discrete packet flow identifiers that can be independently managed. Each identifier represents a specific flow and can be processed separately, allowing the system to handle updates and queries without processing the entire data set at once.
Solution Approach 2:
The patent pre-processes and organizes packet flow identifiers into a heap-ordered data structure before queries arrive. This preliminary organization allows the system to quickly retrieve top-N flows by maintaining the heap property, where the highest priority flows are always at the root, enabling O(1) access to top flows without full sorting when queries arrive.
2Loss of information
If network devices process large volumes of traffic data to provide comprehensive statistics, then the analysis coverage is complete, but the time required to resolve user queries increases significantly
Solution Approach 1:
The patent extracts only the essential packet flow identifiers and their associated statistics from the complete traffic data, storing them in a compact heap-ordered structure. This extraction allows the system to maintain complete traffic analysis capability while using minimal space and time for query processing, as only the identifier and key statistics need to be stored and processed.
Solution Approach 2:
The patent changes the organizational parameter of the data from unstructured or fully sorted formats to a heap-ordered structure based on flow priority or volume. This parameter change enables efficient retrieval of top-N flows by maintaining the heap property, where the highest priority flows are always at the root, allowing O(1) access to top flows without full sorting when queries arrive.
3Productivity
If multiple concurrent queries are handled simultaneously, then user service capacity increases, but the complexity of managing and processing the data increases
Solution Approach 1:
The patent creates copies of the heap-ordered packet flow identifier structure for each concurrent query operation. Each query receives its own isolated heap copy, allowing multiple queries to process simultaneously without interfering with each other or with the main updating process. This copying mechanism enables high concurrent query capacity while keeping the data management logic simple and uniform.
Data Source
AI summary
Techniques are described for collecting and reporting statistics for packet flows within a network. A network device includes traffic flow accounting functionality to generate traffic flow statistics for the network. To aid the analysis of the flow statistics and reporting of the flow statistics to clients, the network device maintains packet flow identifiers in the form of one or more heap tables, i.e., in heap-ordered fashion. Each packet flow identifier, e.g., index or pointer, within the heap table corresponds to one of the packet flows, and may be used to retrieve the traffic flow statistics associated with the respective packet flow. The network device updates the flow statistics and the heap table in real-time. Upon receiving client queries, the network device creates clones of the heap table and applies a truncated heap sort algorithm to quickly respond to the queries.