Traffic Matrix Estimation Using Recursive QSP-MLE for Large Flows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for directly measuring traffic matrices in large communication networks are costly, processor-intensive, and complex, while indirect estimation methods suffer from underdetermination and inaccuracies due to heavy-tailed distributions and outliers, particularly affecting smaller flows.
Innovation Solution
The QSP-MLE model and recursive improvement strategy, which includes quasi-stationary peaking maximum-likelihood estimation and tomocentrality, are used to estimate traffic matrices by filtering out anomalies, recursively improving large flow estimation, and combining with ensemble models to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If direct measurement methods (NetFlow/IPFIX) are deployed on all core routers, then traffic matrix measurement accuracy is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent introduces an external measurement device as an intermediary that collects link utilization data from routers without requiring software deployment on the routers themselves. This mediator approach enables accurate traffic matrix measurement while avoiding the complexity of deploying measurement software on each core router.
Solution Approach 2:
The patent replaces the mechanical/software-based approach of deploying NetFlow/IPFIX agents on each router with a statistical estimation model that uses readily available link utilization data. This substitution eliminates the need for complex software deployment while achieving measurement goals through mathematical modeling.
2Measurement precision
If direct measurement is implemented on all core routers, then traffic matrix data accuracy is improved, but processor demand on routers increases
Solution Approach 1:
The patent uses an external measurement device as a mediator that performs the computational work of collecting and processing link utilization data, thereby eliminating the processor demand on core routers while maintaining measurement accuracy.
Solution Approach 2:
The patent leverages existing router capabilities to export link utilization data without requiring additional processing power. The routers simply provide the data they already have, while the computationally intensive estimation work is performed externally by the measurement device using statistical models.
3Device complexity
If statistical estimation methods are used to estimate traffic matrix, then device complexity is reduced, but measurement precision deteriorates due to underdetermination
Solution Approach 1:
The patent transforms the underdetermined traffic matrix estimation problem into a determined problem by changing parameters: it uses time-varying link utilization measurements across multiple time points and incorporates the known network topology structure as additional constraints, thereby enabling accurate estimation without increasing device complexity.
Solution Approach 2:
The patent employs iterative statistical estimation methods that use feedback from link utilization measurements to continuously refine traffic matrix estimates. The model incorporates feedback loops where initial estimates are refined using additional measurement data and constraints until convergence is achieved, improving precision while maintaining low complexity.
4Device complexity
If conventional statistical approaches are used for traffic matrix estimation, then device complexity remains low, but measurement precision is reduced due to heavy-tailed distributions and outliers
Solution Approach 1:
The patent segments the traffic flow estimation process into two stages: first estimating larger flows that dominate network traffic, then estimating smaller flows. This segmentation allows the model to focus computational resources on the most significant flows while handling outliers and heavy-tailed distributions more effectively, improving overall precision without significantly increasing complexity.
Solution Approach 2:
The patent changes the statistical parameters and assumptions of the estimation model to better fit real network traffic characteristics. It uses time-varying parameters that adapt to changing traffic conditions and incorporates constraints based on network topology and flow conservation laws, thereby improving accuracy despite heavy-tailed distributions and outliers.
Data Source
AI summary
Systems and methods include receiving network data from a network; estimating large flows from the network data utilizing a first statistical approach; responsive to the network being large flow dominant, recursively estimating flows in the network utilizing the first statistical approach until an exit condition is reached; and combining the recursively estimated flows and forming a traffic matrix based thereon. The recursively estimating flows can include estimating a set of large flows utilizing the first statistical approach and freezing a resulting estimate while leaving smaller flows unsolved; repeating the estimating for a next set of large flows from the unsolved smaller flows; and continuing the repeating until the exit condition is reached.