Distributed Flow Analysis Hierarchical Processor Array
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional network testing systems face challenges in real-time analysis of network traffic due to the high volume of test data, where statistics are accumulated faster than they can be communicated to the analysis processor, leading to post-analysis of test results only after completion of testing.
Innovation Solution
A hierarchical processor array is implemented, where network cards, unit processors, and client computers work together to filter, aggregate, and rank flow statistics in a distributed manner, allowing for near-real-time analysis by segregating packets into flows using packet group identification and performing analysis across multiple tiers of processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all test statistics are transferred to a common processor for analysis, then comprehensive analysis can be performed, but the analysis processor cannot receive all test statistics in real-time due to communication bandwidth limitations
Solution Approach 1:
The system segments the centralized analysis function by distributing processing across multiple processors at different hierarchical levels. Network cards perform initial flow statistics accumulation, unit processors aggregate data from multiple network cards, and client computers perform final analysis. This segmentation enables parallel processing and eliminates the single-point bottleneck, allowing real-time analysis without sacrificing comprehensiveness.
Solution Approach 2:
The system introduces a hierarchical dimension to the processing architecture, organizing processors in multiple levels (network cards → unit processors → client computers). This dimensional transformation allows data to flow through multiple processing stages simultaneously, enabling comprehensive analysis at each level while maintaining overall real-time performance that a single processor cannot achieve.
2Reliability
If test data is accumulated at high rate, then complete test coverage is achieved, but the volume of data precludes real-time analysis
Solution Approach 1:
The system extracts and processes only the essential flow statistics data needed for analysis, rather than handling all raw packet data. By identifying and extracting key flow characteristics at the network card level, the system maintains complete test coverage while reducing data volume to manageable levels for real-time processing at higher hierarchical levels.
Solution Approach 2:
The processing workload is segmented across the hierarchical processor array, with each level handling specific processing tasks. Network cards segment data into flows and accumulate basic statistics, unit processors segment and aggregate data from multiple sources, and client computers perform final analysis. This segmentation enables the system to handle high-volume test data while maintaining real-time analysis capability.
3Loss of time
If a hierarchical processor array is implemented for distributed analysis, then near-real-time analysis is achieved, but system complexity increases
Solution Approach 1:
The system merges multiple processing functions into a unified hierarchical architecture where network cards, unit processors, and client computers work together as an integrated system. Each component performs its designated function while contributing to the overall analysis goal, reducing the need for complex inter-component communication protocols and simplifying the overall system design despite the distributed nature.
Solution Approach 2:
The hierarchical processor array uses universal processing units that can perform multiple functions at different hierarchical levels. The same basic processor architecture can operate as a network card, unit processor, or client computer depending on its position and configuration in the hierarchy, reducing the need for specialized hardware and simplifying system implementation.
Data Source
AI summary
There are disclosed apparatus, methods, and a storage medium for testing a network. A plurality of flows may be received and performance statistics for the flows may be measured and stored. At least a portion of the performance statistics may be analyzed in near real-time using a hierarchical array of processors.