Compressed DFA Graph Arc Configuration for Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing DFA graph search systems face a trade-off between memory usage and processing time, with compressed graphs requiring multiple memory accesses and larger graphs requiring significant memory, limiting efficient content searching in network security applications.
Innovation Solution
A compressed DFA graph is implemented, reducing memory accesses and processing time by pruning arcs that point back to the root node and using bit maps for efficient node traversal, allowing for reduced memory usage and single memory access per input character.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a compressed DFA graph is used, then memory requirement is reduced, but multiple memory accesses are required per input character
Solution Approach 1:
The DFA graph is segmented into multiple levels or tiers, with frequently accessed nodes stored in faster memory (e.g., L1 cache) and less frequently accessed nodes in slower memory (e.g., L2 cache or main memory). This hierarchical segmentation allows the system to reduce overall memory usage while maintaining single-access performance for the most critical path nodes.
Solution Approach 2:
The patent implements a nested memory structure where smaller DFA sub-graphs are embedded within larger graph structures. Each nested level contains only the arcs and nodes necessary for that portion of the search, allowing the system to access complete sub-graphs as single units and reduce the number of memory accesses required.
2Productivity
If a larger DFA graph is used, then single memory access per character is achieved, but memory requirement increases significantly
Solution Approach 1:
Different portions of the DFA graph are stored with different quality levels in terms of memory speed and accessibility. Hot paths (frequently executed arcs) are stored in high-speed memory with full accessibility, while cold paths are stored in lower-speed memory. This local quality differentiation allows single-access performance for critical paths without the memory cost of storing the entire graph in high-speed memory.
Solution Approach 2:
The system pre-computes and stores arc configuration information (such as bit maps indicating valid arcs) in the DFA graph structure itself. This preliminary action allows the traversal algorithm to determine the next state with a single memory access by reading pre-computed configuration data, eliminating the need for multiple sequential accesses that would otherwise be required.
3Quantity of substance
If arc configuration information is stored in the DFA graph, then graph size is reduced, but complexity of traversal increases
Solution Approach 1:
Instead of storing complete arc configuration data structures, the patent stores compact bit-map representations where each bit indicates the presence or absence of a valid arc from a given state. This copying of essential information in a compressed format reduces graph size while the bit-map structure itself provides an efficient, constant-time interface for determining valid transitions, avoiding the need for complex traversal logic.
Data Source
AI summary
An apparatus, and corresponding method, for generating a graph used in performing a search for a match of at least one expression in an input stream is presented. The graph includes a number of interconnected nodes connected solely by valid arcs. A valid arc may also include a nodal bit map including structural information of a node to which the valid arc points to. A walker process may utilize the nodal bit map to determine if a memory access is necessary. The nodal bit map reduces the number of external memory access and therefore reduces system run time.