Perfect Hash Table Update for High-Speed String Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network intrusion detection systems face challenges in efficiently implementing multi-string matching algorithms, particularly in avoiding hash collisions, optimizing memory usage, and ensuring fast hash index generation, which affects the system's performance and vulnerability to attacks.
Innovation Solution
The development of progressive perfect hashing techniques, such as P2-Hashing and 2D P2-Hashing, which involve dividing transitions into independent sets, renaming states to avoid collisions, and using bipartite graph decomposition to construct perfect hash tables, allowing for efficient insertion and updating of rules without additional memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional hash tables are used for multi-string matching, then memory usage is reduced, but hash collisions occur which slow down matching speed
Solution Approach 1:
The patent segments the hash table into multiple dimensions (e.g., primary hash table and secondary hash tables). Each dimension handles a portion of the hashing, allowing the system to achieve perfect hashing (no collisions) by distributing transitions across multiple segmented structures rather than relying on a single hash table.
Solution Approach 2:
The patent transitions from traditional one-dimensional hash tables to multi-dimensional perfect hash structures. By adding dimensional layers (primary table + secondary tables), the system resolves hash collisions that would occur in a single dimension, achieving both high speed and collision-free matching.
2Speed
If perfect hash tables are constructed to avoid collisions, then matching speed is improved, but construction complexity and memory usage increase
Solution Approach 1:
The patent performs preliminary actions during the hash table construction phase by pre-calculating and organizing transitions into independent sets using bipartite graph decomposition. This preliminary structuring simplifies the ongoing maintenance and updating operations, as the foundation is already optimized for perfect hashing.
Solution Approach 2:
The patent implements dynamic updating mechanisms that allow the perfect hash table to be modified efficiently. By maintaining independence between transition sets and using systematic replacement strategies, the system can update rules without reconstructing the entire hash structure, keeping complexity manageable despite the perfect hashing requirement.
3Adaptability or versatility
If rules are updated in traditional hash tables, then new patterns are added, but existing transitions may be disrupted causing collisions
Solution Approach 1:
The patent segments transitions into independent sets that can be updated separately. When a rule needs updating, only the specific transition set affected is modified, while other sets remain unchanged. This segmentation isolates the impact of updates and prevents disruptions to the overall hash table perfection.
Solution Approach 2:
The patent employs systematic parameter changes during rule updates, such as renumbering states or adjusting transition mappings in a controlled manner. By changing parameters systematically rather than arbitrarily, the system maintains the perfect hashing property while adapting to new rules.
4Loss of time
If multiple memory accesses are allowed for hash operations, then more complex data structures can be used, but processing time increases
Solution Approach 1:
The patent performs preliminary organization of transitions into independent sets during the construction phase. This preliminary action enables single-memory-access retrieval during matching operations, as the structure is pre-optimized to provide direct access without requiring multiple sequential memory reads.
Solution Approach 2:
By organizing the hash structure into multiple dimensions with specific access patterns, the patent enables the system to provide fast single-access retrieval. The multi-dimensional organization allows the relevant transition information to be located and retrieved in a single memory operation, avoiding the need for multiple sequential accesses.
Data Source
AI summary
A representation of a new rule, defined as a set of a new transition(s), is inserted into a perfect hash table which includes previously placed transitions to generate an updated perfect hash table. This may be done by, for each new transition: (a) hashing the new transition; and (b) if there is no conflict, inserting the hashed new transition into the table. If, however, the hashed new transition conflicts with any of the previously placed transitions, either (A) any transitions of the state associated with the conflicting transition are removed from the table, the hashed new transition is placed into the table, and the removed transitions are re-placed into the table, or (B) any previously placed transitions of the state associated with the new transition are removed, and the transitions of the state associated with the new transition are re-placed into the table.