Adaptive Hybrid DFA for Network Traffic Pattern Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network security devices face inefficiencies in pattern matching during Deep Packet Inspection (DPI) due to high resource consumption and redundant memory usage in traditional Deterministic Finite Automata (DFA) implementations, which hinder effective detection of malicious traffic patterns.
Innovation Solution
Implementing a hybrid DFA with adaptive reconfiguration, combining a computationally efficient array implementation and a memory-efficient bitmap implementation, allowing dynamic promotion and demotion of states based on usage, to optimize memory usage and improve packet throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a traditional Deterministic Finite Automata (DFA) implementation is used for pattern matching in Deep Packet Inspection, then pattern matching capability is achieved, but system resources are consumed significantly and memory usage is inefficient
Solution Approach 1:
The DFA is segmented into two distinct portions: a first portion implemented with array-based states for frequently accessed patterns, and a second portion implemented with bitmap-based states for less frequently accessed patterns. This segmentation allows the system to optimize for both speed and memory efficiency by matching packets against the appropriate portion based on access frequency and resource constraints.
Solution Approach 2:
The system dynamically reconfigures the DFA by monitoring state traversal frequencies and automatically promoting frequently accessed bitmap states to array states, or demoting infrequently accessed array states to bitmap states. This dynamic adaptation enables the DFA to optimize its resource allocation based on actual traffic patterns, improving both throughput and efficiency over time.
2Speed
If a high-throughput DFA implementation is used, then packet processing speed is improved, but memory consumption increases due to uncompressed state storage
Solution Approach 1:
Different portions of the DFA are assigned different implementation qualities: the first portion uses array-based states with higher memory consumption but faster access speeds, while the second portion uses bitmap-based states with lower memory consumption but slightly slower access. This local differentiation allows the system to optimize overall performance while managing memory constraints effectively.
Solution Approach 2:
The system changes the implementation parameter of DFA states from a static choice to a dynamic one, where states can transition between array-based and bitmap-based implementations based on their access frequency. This parameter change allows the system to adapt memory allocation to actual usage patterns, reducing overall memory consumption while maintaining necessary processing speed.
3Quantity of substance
If compressed DFA implementation is used, then memory usage is reduced, but packet throughput decreases due to compression overhead
Solution Approach 1:
The system performs preliminary action by pre-loading frequently accessed DFA states into the array-based first portion before packet processing begins. This preliminary organization ensures that high-frequency patterns are immediately accessible in an optimized format, eliminating compression overhead for the most critical patterns and maximizing throughput for common traffic types.
Solution Approach 2:
The array-based first portion acts as an intermediary layer between the compressed bitmap states and the packet processing engine. Frequently accessed patterns are promoted to this intermediary layer, which provides faster access without requiring full decompression of the compressed bitmap format, thus maintaining memory efficiency while improving throughput for common patterns.
4Adaptability or versatility
If the same DFA states are stored in both compressed and uncompressed formats, then flexibility is improved, but redundant memory usage increases
Solution Approach 1:
The system merges the compressed and uncompressed DFA implementations into a unified hybrid structure where both portions coexist and share the same underlying state definitions. This merging eliminates redundant storage by allowing the system to switch between compressed and uncompressed representations of the same states based on access frequency, rather than maintaining separate complete copies of all states in both formats.
Solution Approach 2:
The system discards the uncompressed version of states when they are promoted to the array-based first portion, and recovers them by promoting bitmap states to array states when needed. This dynamic discarding and recovering mechanism maintains implementation flexibility while eliminating redundant memory storage, as each state exists in only one format at any given time.
Data Source
AI summary
In general, techniques are described for network traffic pattern matching using adaptive deterministic finite automata (DFA). A network device may implement the techniques to promote pattern matching. The network device comprises a control unit that stores first and second data defining first and second portions of a DFA, respectively. The first data defines first states of the DFA in an uncompressed format. The second data defines second states of the DFA in a compressed format. The network device also includes an interface that receives network packets. The control unit processes the network packets to traverse the first and second states. The control unit then compares a number of times the first and second states have been traversed. Based on the comparison, the control unit dynamically reallocates the first states of the DFA in the uncompressed format and the second states of the DFA in the compressed format.