Incremental DFA Compilation for RegEx Accelerator Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
General-purpose processors are inefficient in handling high-capacity network and storage workloads, particularly in processing packet streams due to their design limitations.
Innovation Solution
A highly programmable data processing unit with specialized hardware-based accelerators, such as a RegEx accelerator, is used to offload tasks like regular expression evaluation, utilizing incremental compilation techniques to efficiently update and combine deterministic finite automata graphs for improved performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors are used to process packet streams, then device versatility is maintained, but processing speed and efficiency deteriorate
Solution Approach 1:
The system segments processing tasks by separating general-purpose packet handling from specialized regular expression matching. The RegEx accelerator operates as an independent hardware module that offloads specific pattern-matching workloads from the main processor, allowing each component to be optimized for its specific function while maintaining overall system versatility.
Solution Approach 2:
The RegEx accelerator acts as an intermediary hardware component between the processor and packet stream data. It receives packet data from the processor, performs specialized regular expression evaluation in hardware, and returns results to the processor, thereby improving overall processing speed without requiring the main processor to be redesigned for high-speed packet handling.
2Reliability
If entire DFA graphs are recompiled when rules are added or modified, then evaluation accuracy is maintained, but compilation time increases
Solution Approach 1:
The DFA graph is segmented into multiple independent sub-graphs, each corresponding to a specific regular expression rule. When a rule is added or modified, only the corresponding sub-graph needs to be recompiled and updated in the DFA cache, rather than recompiling the entire DFA graph. This segmentation maintains evaluation accuracy while dramatically reducing compilation time for rule updates.
Solution Approach 2:
The system pre-allocates and pre-structures the DFA graph as a collection of independent sub-graphs during initial compilation, organizing them in a way that facilitates selective updates. This preliminary organization enables efficient incremental compilation by allowing individual sub-graphs to be independently modified without affecting the overall graph structure or requiring full recompilation.
Data Source
AI summary
A compiler/loader unit for a RegEx accelerator is described that receives a first set of regular expression rules for implementing the RegEx accelerator, generates, based on the first set of regular expression rules, an initial deterministic finite automata (DFA) graph, and generates, an initial memory map for allocating the initial DFA graph to a memory of the RegEx accelerator. The compiler/loader unit receives receive, a second set of one or more new or modified regular expression rules for implementing the RegEx accelerator and in response performs incremental compilation of the second set of regular expressions. The compiler/loader unit generates, based on the second set of one or more regular expression rules, a supplemental DFA graph and reconciles the initial DFA graph with the supplemental DFA graph to generate an updated memory map for allocating the initial DFA graph and the supplemental DFA graph to the memory of the RegEx accelerator.