Compressed DFA Pattern Matching for High-Speed Network Traffic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-pattern matching algorithms, such as the Aho-Corasick algorithm, are unable to keep up with high network speeds due to the need for a memory lookup operation for each byte of the input stream, making them inefficient for high-speed applications like virus/worm detection in data networks.
Innovation Solution
A method and system for multi-character multi-pattern pattern matching using a compressed deterministic finite state automaton (DFA) that transitions between multiple states based on multiple characters, combined with a Ternary Content-Addressable Memory (TCAM) and Static Random Access Memory (SRAM) to enhance pattern matching speeds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional multi-pattern matching algorithms (Aho-Corasick) are used, then pattern matching accuracy is maintained, but processing speed is insufficient for high-speed network applications
Solution Approach 1:
The patent segments the input stream into fixed-size windows (e.g., 4 bytes at a time) and processes multiple characters simultaneously through parallel DFA transitions. This segmentation allows the system to evaluate multiple potential pattern matches in parallel within each window, dramatically increasing processing speed while maintaining accuracy through systematic evaluation of all segments.
Solution Approach 2:
The patent transitions from processing one character at a time (1D sequential processing) to processing multiple characters simultaneously in parallel (2D/3D state space exploration). By constructing a multi-dimensional state transition table that captures all possible transitions within a window, the system evaluates multiple paths through the DFA state space in parallel, achieving exponential speedup relative to sequential processing.
2Measurement precision
If memory lookup is performed for each byte of input stream, then accurate pattern detection is achieved, but the number of memory operations becomes too high for high-speed processing
Solution Approach 1:
The patent pre-computes and stores all possible DFA transitions for every combination of states and character windows in a transition table before processing begins. This preliminary action creates a comprehensive lookup structure that captures all valid transitions, allowing the system to perform accurate pattern detection through simple table lookups rather than complex runtime computations, thereby maintaining precision while reducing operational complexity.
Solution Approach 2:
The patent merges multiple sequential memory lookup operations into a single parallel lookup operation. By combining the state information and character window into a composite key, the system performs one memory access that retrieves all relevant transition information for the entire window, replacing what would otherwise require multiple separate lookups for each character in the window.
3Speed
If multiple memory lookups are performed per character to achieve high-speed matching, then processing speed improves, but memory access overhead increases
Solution Approach 1:
The patent merges multiple character processing operations into a single memory access event. By constructing composite keys that encode both the current DFA state and the character window, the system retrieves all necessary transition information in one memory operation, eliminating the need for separate memory accesses for each character and thereby reducing energy consumption proportional to the window size.
Solution Approach 2:
The patent creates a compressed representation of the DFA transition table that fits in faster, smaller memory (such as L1 cache or SRAM). By copying only the essential transition information in a compact format, the system reduces memory access latency and energy consumption, allowing frequent lookups to occur in high-speed memory rather than slower main memory.
Data Source
AI summary
Disclosed is a method and system for multi-character multi-pattern pattern matching. In the multi-character multi-pattern pattern matching method, patterns in an input stream are detected by transitioning between states of a “compressed deterministic finite state automaton (DFA)”, with each transition based on multiple characters of the input stream. The compressed DFA is created by compressing an original DFA, such as an Aho-Corasick DFA, such that each state of the compressed DFA represents multiple consecutive states of the original DFA and each transition between the states of the compressed DFA is a combination of all of the transitions between the multiple consecutive states of the original DFA. This method can be implemented using a Ternary Content-Addressable Memory (TCAM) to store the transitions of the compressed DFA and compares the transitions with multiple characters of an input stream at a time to detect patterns in the input stream.