Multi-Packet Pattern Matching via Hash Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pattern matching techniques for network packet streams, such as finite state automaton (FSA) and hash-based schemes, face inefficiencies in high-speed operations due to memory usage and false positives, making them unsuitable for gigaspeed routers and unable to detect long patterns spread across multiple packets.
Innovation Solution
A pattern detection system utilizing content addressable memories (CAMs) to store hash values representing patterns and data blocks, allowing for quick detection of patterns across multiple packets by calculating and comparing hash values and block counts, enabling efficient detection at line-rate speeds without excessive memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If FSA-based functions (Knuth-Morris-Pratt, Aho-Corasick) are used for pattern matching, then search speed is linear to data size and independent of number of target signatures, but system memory usage becomes large and pattern matching rate reduces due to frequent memory fetch operations
Solution Approach 1:
The patent replaces traditional FSA-based mechanical pattern matching systems with a hash-based computational approach. Instead of using finite state automata that require extensive memory tables and sequential state transitions, the invention uses hash functions to directly compute pattern matches, substituting the mechanical FSA operation with a more memory-efficient hashing mechanism that achieves linear time complexity without the memory overhead.
Solution Approach 2:
The patent changes the fundamental parameter of pattern matching from state-based transitions in FSA to hash value comparisons. By transforming the matching criterion from following predetermined state transitions to comparing computed hash values, the system achieves both linear time complexity and reduced memory requirements, as hash values can be computed and compared with constant space overhead.
2Quantity of substance
If hash-based schemes are used to store signature strings, then memory usage is reduced, but false positives occur and long patterns spread across multiple packets cannot be detected
Solution Approach 1:
The patent implements nested hashing where multiple hash values are computed for different segments or perspectives of the same data. Instead of relying on a single hash function that may produce collisions, the system nests multiple hash computations, where each hash provides an additional layer of verification. This nested approach maintains low memory usage while significantly reducing false positives through multi-layered hash validation.
Solution Approach 2:
The patent introduces hash values as intermediary representations between the original data and the matching decision. Rather than directly comparing raw data or using single-hash schemes, the system uses multiple hash values as intermediaries that mediate the matching process. These intermediary hash values provide a buffer that reduces direct collisions while maintaining detection accuracy, especially for patterns spanning multiple packets.
3Speed
If hardware alternatives using FPGAs are used for high speed operations, then pattern matching speed increases, but flexibility decreases when target patterns frequently change and hardware costs increase
Solution Approach 1:
The patent implements a dynamic pattern matching system where hash functions and matching criteria can be rapidly reconfigured without hardware redesign. Unlike fixed FPGAs that require physical reconfiguration, the invention uses software-based hash computation that can adapt to changing patterns through algorithmic adjustments. This dynamic approach maintains high speed performance while providing flexibility to frequently changing target patterns, eliminating the need for expensive hardware reconfiguration.
Data Source
AI summary
Pattern matching for a network flow includes storing a representation of the pattern. The representation of the pattern includes hash values representing blocks of data in the pattern. Hash values are calculated for blocks of data in the network flow. The blocks of data are spread across multiple packets in the network flow. The calculated hash values for the blocks of data in the network flow are compared to the hash values in the representation of the pattern to detect the pattern in the network flow.