Two-Dimensional Compression for DFA State Transitions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep packet inspection (DPI) in distributed networks faces challenges due to the high computational complexity of signature matching, particularly because deterministic finite automata (DFA) require large memory for storing next state transitions, leading to memory bandwidth issues that hinder high throughput.
Innovation Solution
A two-dimensional compression algorithm is employed to compress state transitions and group states, reducing memory requirements by identifying redundant transitions and using bitmapping and conquer step grouping to create a compressed DFA table for efficient DPI.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deterministic finite automata (DFA) are used for signature matching, then matching accuracy and reliability are improved, but memory usage increases prohibitively
Solution Approach 1:
The patent divides the DFA state transitions into multiple groups or tables, organizing states into subsets that can be processed separately. This segmentation allows the system to load only relevant state transition groups into memory at any given time, reducing overall memory requirements while preserving the complete matching capability across multiple segments.
Solution Approach 2:
The patent introduces a new dimension of organization by grouping states based on transition patterns and creating multiple state transition tables. Instead of storing all transitions in a single flat structure, the system uses a hierarchical or multi-dimensional arrangement where states are organized into groups, enabling more efficient memory utilization through structured access patterns.
2Measurement precision
If complete DFA state transitions are stored, then transition accuracy is improved, but memory bandwidth consumption increases
Solution Approach 1:
The patent extracts and stores only the essential or frequently accessed state transition information in memory, while less critical transitions are either computed on-demand or stored in slower memory tiers. This selective extraction reduces memory bandwidth consumption by minimizing the volume of data that needs to be frequently accessed during signature matching operations.
Solution Approach 2:
The patent performs preliminary grouping and organization of state transitions into structured tables before actual signature matching occurs. By pre-processing and organizing the transition data into optimized groups during initialization or idle periods, the system reduces the computational and memory bandwidth requirements during active signature matching operations.
3Quantity of substance
If DFA size is reduced through compression, then memory usage is improved, but device complexity increases
Solution Approach 1:
The patent changes the organizational parameters of the DFA representation by introducing grouping criteria based on transition patterns and state characteristics. By reorganizing states into groups according to specific parameters (such as transition destinations, source states, or frequency of access), the system achieves compression through more efficient data structures without requiring complex compression algorithms.
Data Source
AI summary
A hardware system for signature matching in a distributed network is disclosed. The hardware system comprises a network processor and a memory. The network processor is configured to perform horizontal compression on a state table using bitmaps, wherein the state table has a plurality of states and state transitions. The processor is also configured to perform a first grouping of states of the state table using the bitmaps to generate a first one or more sets of states, perform a second grouping of states of the state table based on the first one or more sets of states and a transition threshold to generate a second one or more sets of states, perform a conquer step grouping of the states of the state table based on the second one or more sets of states and conquer criteria to generate third one or more sets of states, and generate a two dimensioned compressed state table based on the third one or more sets of states. The memory circuit is configured to store the two dimensioned compressed state table.