Compressing NFA Automata via Common Prefix Grouping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network Intrusion Detection Systems (NIDS) face significant challenges in efficiently matching regular expressions due to the large size of matching automata, which can lead to resource exhaustion and vulnerability to Denial of Service (DOS) attacks, especially when dealing with deep matches against a large rule set.
Innovation Solution
A method for compressing Non-deterministic Finite Automata (NFA) by identifying and utilizing common prefixes among regular expressions, reducing the number of states and redundant paths, thereby improving throughput and reducing the size of the matching automaton through a series of processes including transition sequence conversion, grouping, subgrouping, and merging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If regular expressions are used for pattern matching in NIDS, then matching specificity and dynamic capability are improved, but automaton size and resource consumption increase
Solution Approach 1:
The patent segments the automaton construction process into multiple passes, where common prefixes are identified and extracted first, then the remaining patterns are processed separately. This segmentation allows the system to handle large rule sets by breaking them into manageable components that can be processed and stored more efficiently.
Solution Approach 2:
The patent extracts common prefixes from multiple regular expressions and creates separate transition sequences for these shared portions. By taking out the commonalities, the system avoids redundant path creation in the automaton, significantly reducing the overall size while maintaining the ability to match all original patterns.
2Adaptability or versatility
If the number of regular expressions is increased to improve detection coverage, then detection capability is improved, but automaton size grows and memory requirements exceed available resources
Solution Approach 1:
The patent merges transition sequences by identifying and combining common prefixes across multiple regular expressions. Instead of creating separate paths for each pattern, the system merges shared portions into a single automaton structure, allowing thousands of patterns to be represented within limited memory resources.
Solution Approach 2:
The patent creates a universal automaton structure where a single set of transitions can serve multiple detection purposes. By designing the automaton to handle common prefixes that appear across many different patterns, the system achieves multi-functionality, allowing one automaton to detect numerous different threat types simultaneously.
3Adaptability or versatility
If complex regular expressions with repetition and counting are used, then pattern matching capability is improved, but redundant paths in NFA increase and minimization becomes computationally hard
Solution Approach 1:
The patent performs preliminary processing to identify common prefixes before the main automaton construction. By preparing and extracting shared patterns in advance, the system reduces the complexity of subsequent minimization operations, making the overall process computationally feasible even for large rule sets with complex patterns.
Data Source
AI summary
The present disclosure provides a means for compressing Non-deterministic Finite Automata (NFA) for faster matching during Deep Packet Inspection (DPI) when a Network Intrusion Detection System (NIDS) is evaluating traffic to find suspicious network traffic. The present disclosure accomplishes this through four primary components. First, it provides a time-efficient method for accurately comparing two regular expressions so that common prefixes can be identified. Second, it provides a time-efficient method for grouping regular expressions by their common prefixes. Third, it provides a method for subgrouping within groups by longest common prefixes in order to maximize compression. Finally, it provides a method for building a compressed NFA using heuristics derived from the length of the common prefix to a subgroup.