Communication Data Statistical Apparatus for High-Speed Network Flow Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for taking statistics of communication data in high-speed networks face challenges with increased line speeds, as they require frequent memory access and storage, leading to difficulties in recording statistical information effectively.
Innovation Solution
A communication data statistical apparatus that classifies packets into aggregate flows and updates statistical information based on aggregate data, reducing the frequency of memory access and enhancing cache hit rates, allowing for efficient statistics collection even in high-speed networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If header information of every packet is analyzed and statistical information is stored in memory for each flow, then flow analysis capability is improved, but memory access speed becomes insufficient at line speeds of 40 Gbps or higher
Solution Approach 1:
The patent segments the statistical processing into two distinct stages: (1) aggregation processing that combines multiple packet flows into aggregate flows with aggregated statistical information, and (2) statistical processing that performs detailed flow classification and statistics update. This segmentation allows the system to reduce memory access frequency by first aggregating data in a speed-optimized manner before performing detailed statistical analysis, thereby resolving the contradiction between precise flow analysis and memory access speed requirements at high line speeds.
Solution Approach 2:
The patent performs preliminary aggregation of packet flows into aggregate flows before performing detailed statistical processing. By pre-aggregating multiple packet flows and their statistical information in advance, the system reduces the volume of data that requires detailed processing and memory storage, thereby improving overall processing speed while maintaining the capability for precise flow analysis when needed.
2Productivity
If a buffer is used to accumulate statistical information for collective update, then updating overhead is reduced, but buffer capacity is quickly exhausted by diverse flows
Solution Approach 1:
The patent merges multiple packet flows into aggregate flows by combining flows that share common characteristics. This merging reduces the total number of separate flows that need to be tracked individually, thereby reducing the buffer capacity required while still maintaining the ability to perform collective updates. The aggregation process combines statistical information from multiple flows, reducing the overall data volume that must be stored and processed.
Solution Approach 2:
The patent changes the parameter representation by transitioning from individual flow parameters to aggregate flow parameters. Instead of maintaining separate statistical information for each individual flow, the system transforms the data into aggregated statistical information that represents multiple flows collectively. This parameter transformation reduces the quantity of data requiring buffer storage while preserving the essential statistical characteristics needed for network analysis.
Data Source
AI summary
An apparatus has a receiver module configured to receive packets including multiple different identifiers, an aggregation module configured to classify each packet, which is received by the receiver module into a certain aggregate flow and generate aggregate statistical information including number of packets belonging to the certain aggregate flow, and a statistical processing module configured to perform an updating process arranged to classify the certain aggregate flow into a specific statistical flow and to update statistical information including number of packets belonging to the specific statistical flow based on corresponding aggregate statistical information, the statistical processing module repeating the updating process for multiple aggregate flows, into which packets are classified by the aggregation module, so as to update the statistical information in each of the multiple different statistical conditions.