Packet Gateway Bloom Filter Policy Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current threat intelligence gateways (TIGs) face performance challenges in filtering large volumes of packets against millions of packet filtering rules without significantly impacting network performance, due to increasing policy sizes and network speeds.
Innovation Solution
The use of probabilistic data structures like Bloom filters to efficiently determine if a packet matches any CTI-derived packet filtering rule, reducing unnecessary searches and optimizing memory usage by associating multiple identical rules with a single rule.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional packet filtering methods are used to filter packets against millions of rules, then network security protection is provided, but TIG performance degrades and latency increases
Solution Approach 1:
The patent introduces Bloom filters as an intermediary data structure between the packet filtering system and the rule database. The Bloom filter serves as a mediator that quickly determines whether a packet's network address matches any threat indicator, avoiding the need to search through millions of actual filtering rules. This intermediary layer maintains security protection while dramatically improving processing speed.
Solution Approach 2:
The patent creates a simplified copy of the threat intelligence data in the form of Bloom filters. Instead of storing and searching the complete set of millions of packet filtering rules, the system creates a probabilistic data structure copy that contains only the essential membership information. This copy enables fast lookups while using significantly less memory, resolving the contradiction between comprehensive security coverage and processing performance.
2Measurement precision
If complete packet filtering rules are stored in memory for accurate matching, then security accuracy is maintained, but memory usage increases significantly
Solution Approach 1:
The patent creates a simplified copy of the threat intelligence data in the form of Bloom filters. Instead of storing and searching the complete set of millions of packet filtering rules, the system creates a probabilistic data structure copy that contains only the essential membership information. This copy enables fast lookups while using significantly less memory, resolving the contradiction between comprehensive security coverage and processing performance.
Solution Approach 2:
The patent changes the parameter representation from storing complete filtering rules with all metadata to storing only hash-based membership information in Bloom filters. This parameter transformation reduces the memory footprint from millions of detailed rules to a compact bit array, while maintaining the ability to accurately determine whether a packet address matches any threat indicator through probabilistic membership testing.
3Quantity of substance
If probabilistic data structures like Bloom filters are used to reduce memory usage, then memory efficiency improves, but false positive rate increases
Solution Approach 1:
The patent changes the parameter representation from storing complete filtering rules with all metadata to storing only hash-based membership information in Bloom filters. This parameter transformation reduces the memory footprint from millions of detailed rules to a compact bit array, while maintaining the ability to accurately determine whether a packet address matches any threat indicator through probabilistic membership testing.
4Measurement precision
If all packet filtering rules are searched for each packet, then accurate matching is achieved, but search time increases
Solution Approach 1:
The patent extracts the essential membership information from the complete set of packet filtering rules and stores it in Bloom filters. By taking out only the critical information needed for matching (the presence or absence of a network address in the threat intelligence database) and discarding the redundant rule details, the system achieves fast filtering without sacrificing matching accuracy.
Solution Approach 2:
The patent creates a simplified copy of the threat intelligence data in the form of Bloom filters. Instead of storing and searching the complete set of millions of packet filtering rules, the system creates a probabilistic data structure copy that contains only the essential membership information. This copy enables fast lookups while using significantly less memory, resolving the contradiction between comprehensive security coverage and processing performance.
Data Source
AI summary
A packet gateway may protect TCP/IP networks by enforcing security policies on in-transit packets that are crossing network boundaries. The policies may include packet filtering rules derived from cyber threat intelligence (CTI). The rapid growth in the volume of CTI and in the size of associated CTI-derived policies, coupled with ever-increasing network link speeds and network traffic volume, may cause the costs of sufficient computational resources to be prohibitive. To efficiently process packets, a packet gateway may be provided with at least one probabilistic data structure, such as a Bloom filter, for testing packets to determine if packet data may match a packet filtering rule. Packet filtering rules may be grouped into subsets of rules, and a data structure may be provided for determining a matching subset of rules associated with a particular packet.