Pattern Matching for High Throughput Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current pattern matching techniques for high-throughput network processing struggle to maintain efficient performance and allow for non-interruptive rule updates, especially in devices like routers and intrusion detection systems, which require handling vast network traffic and frequent rule set changes.
Innovation Solution
A novel pattern matching technique involving an array of tiny finite state machines that search for subsets of rules and bits, utilizing bit-level partitioning and Aho-Corasick state machines to operate in parallel, enabling efficient hardware or software implementation and non-interruptive rule updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional pattern matching algorithms (e.g., Boyer-Moore, Aho-Corasick) are used in software, then implementation flexibility is maintained, but processing throughput cannot keep up with tens of gigabits per second network traffic speeds
Solution Approach 1:
The patent divides the pattern matching task into multiple parallel finite state machines, each handling a subset of patterns. This segmentation enables the system to process network traffic at tens of gigabits per second by distributing the computational load across multiple specialized processing units, thereby achieving high throughput while maintaining manageable complexity through modular architecture
Solution Approach 2:
The patent replaces traditional software-based pattern matching with hardware-implemented finite state machines. This substitution of mechanical/software processing with dedicated hardware circuits enables the system to achieve the required processing throughput of tens of gigabits per second, as hardware circuits can operate at much higher speeds than software algorithms
2Adaptability or versatility
If rule sets are updated frequently in intrusion detection systems, then detection accuracy improves, but system operation is interrupted during updates
Solution Approach 1:
The patent implements a dual-buffer architecture where one buffer contains the currently active rule set while another buffer is pre-loaded with updated rules. This preliminary preparation of backup rule sets allows the system to switch between buffers atomically, enabling rule updates without interrupting the continuous operation of the intrusion detection system
Solution Approach 2:
The patent employs dynamic rule set switching through a selector mechanism that can rapidly transition between different rule set buffers. This dynamic architecture allows the system to adapt to new threats by loading updated rules into a standby buffer and then seamlessly switching to it, maintaining both adaptability and continuous operation
3Productivity
If bit-level partitioning is applied to create parallel finite state machines, then processing efficiency increases by an order of magnitude, but storage requirements for rule sets increase
Solution Approach 1:
The patent merges multiple finite state machines into a single integrated hardware circuit that processes bits in parallel. By combining the functionality of multiple partitioned state machines into one unified structure that operates on bit-level parallelism, the system achieves order-of-magnitude processing efficiency improvements while avoiding the storage overhead of duplicating rule sets across multiple separate machines
Data Source
AI summary
A pattern matching technique for high throughput network processing includes a simple yet powerful special purpose architecture and a set of novel string matching algorithms that can work in unison. The novel set of algorithms allow for bit-level partitioning of rules such that may be more easily implemented in hardware or software. The result is a device that maintains tight worst case bounds on performance, can be updated with new rules without interrupting operation, compiles in seconds instead of hours, and is ten times more efficient than the existing best known solutions in this area.