Probabilistic Filters Iterative Switching Network Flow Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Bloom filters in network elements experience increased false positivity and memory inefficiency due to stale members, especially as they grow with new hot flows, making them error-prone for maintaining state in forwarding or middlebox service operations.
Innovation Solution
Implementing an iterative switching mechanism between two probabilistic filters, where one acts as an active filter and the other as a backup, with the backup storing a smaller subset of recent and high-frequency flows, allowing for refreshing the active filter without the need for time-based record removal, thereby reducing false positives and improving memory efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a bloom filter is used to track data message flows, then memory efficiency is improved, but false positivity increases as the filter grows with new hot flows
Solution Approach 1:
The patent divides the bloom filter into two separate filters: a first bloom filter for tracking all data message flows and a second bloom filter for tracking only hot flows. This segmentation allows each filter to serve a specific purpose, with the second filter enabling efficient identification of hot flows without the first filter experiencing increased false positivity from storing all flows.
Solution Approach 2:
The patent applies different tracking qualities to different flow types by storing all flows in the first bloom filter with basic tracking, while selectively storing only hot flows in the second bloom filter with enhanced tracking. This local quality differentiation optimizes memory usage while maintaining high reliability for hot flow detection.
2Measurement precision
If connection tracking storage is used to maintain exact flow records, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent extracts the hot flow tracking function from the main connection tracking storage by implementing a separate second bloom filter. This extraction allows the primary tracking mechanism to remain simple while adding sophisticated hot flow detection capabilities through the specialized second filter.
Solution Approach 2:
The patent creates a simplified copy of the connection tracking functionality specifically for hot flows using the second bloom filter. This copy focuses only on the essential tracking aspect for hot flows, avoiding the complexity of maintaining complete flow records while still providing accurate hot flow identification.
Data Source
AI summary
Some embodiments of the invention provide novel methods for using probabilistic filters to keep track of data message flows that are processed at an element (e.g., forwarding element or middlebox service element) of a network. In some embodiments, the method iteratively switches between two probabilistic filters as the active and backup filters as a way of maintaining and refreshing its active probabilistic filter without the need for maintaining time values for removing outdated records from its active filter.