We formulate the network-wide traffic measurement / analysis problem as a series of set-cardinality-determination (SCD) problems. By leveraging recent advances in probabilistic distinct sample counting techniques, the set-cardinalities, and thus, the network-wide traffic measurements of interest can be computed in a distributed manner via the exchange of
extremely light-weight traffic digests (TD's) amongst the network nodes, i.e. the routers. A TD for N packets only requires O(loglog N) bits of memory storage. The computation of such O(loglog N)-sized TD is also amenable for efficient hardware implementation at wire-speed of 10 Gbps and beyond. Given the small size of the TD's, it is possible to distribute
nodal TD's to all routers within
a domain by piggybacking them as opaque
data objects inside existing control messages, such as OSPF link-state packets (LSPs) or I-BGP control messages. Once the required TD's are received, a
router can estimate the traffic measurements of interest for each of its local link by solving a series of set-cardinality-determination problems. The traffic measurements of interest are typically in form of per-link, per-traffic-aggregate packet counts (or flow counts) where an aggregate is defined by the group of packets sharing the same originating and / or destination nodes (or links) and / or some intermediate nodes (or links). The local measurement results are then distributed within the domain so that each
router can construct a network-wide view of routes / flow patterns of different traffic commodities where a commodity is defined as a group of packets sharing the same
origination and / or termination nodes or links. After the initial network-wide traffic measurements are received, each
router can further reduce the associated measurement /
estimation errors by locally conducting a minimum
square error (MSE) optimization based on network-wide commodity-flow conservation constraints.