Anchored Match Algorithm for Large URL Pattern Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-pattern string matching algorithms, such as Aho-Corasick and Wu-Manber, exhibit poor performance when dealing with large numbers of patterns, leading to increased computing time and resource consumption, especially when matching against numerous URL patterns.
Innovation Solution
A map and chain data structure is employed to index Uniform Resource Locators (URLs) by their significant hostname components, using hash values to efficiently match new URLs against large sets of patterns, allowing for a significant increase in the number of patterns that can be processed without degrading performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If current multi-pattern string matching algorithms (Aho-Corasick, Wu-Manber) are used to match large numbers of patterns, then the algorithms can handle multiple patterns, but performance degrades significantly with greater than 10^4 patterns due to increased computing time and resource consumption
Solution Approach 1:
The patent segments the pattern matching problem by dividing patterns into groups based on their first substrings (anchors). Instead of treating all patterns uniformly, it creates anchor-to-pattern mappings that organize patterns into manageable segments, allowing efficient retrieval without processing the entire pattern set for each match operation.
Solution Approach 2:
The patent performs preliminary actions by pre-processing patterns to extract anchors and build anchor-to-pattern mappings before actual matching occurs. This pre-computation of the mapping structure enables rapid matching operations later, as the heavy lifting of organizing patterns is done in advance rather than during each matching operation.
2Reliability
If Aho-Corasick algorithm constructs a finite state machine with each node representing a single character, then all pattern strings can be recognized, but the complexity of constructing and searching through the dictionary data structure increases linearly with pattern size
Solution Approach 1:
The patent extracts the first substring (anchor) from each pattern and uses it as a key for organizing patterns. By taking out this critical identifying feature and using it to group patterns, the algorithm avoids constructing the complete finite state machine while still enabling efficient pattern recognition through the anchor-to-pattern mapping structure.
3Ease of operation
If Wu-Manber algorithm creates three table data structures (shift table, hash table, prefix table) to catalogue strings, then string matching can be performed, but the shift table specifies a maximum number of shifts calculated from a first subset of characters which limits the number of string patterns that can be catalogued
Solution Approach 1:
The patent changes the parameter used for organizing patterns from the Wu-Manber approach of using shift calculations based on character subsets to using the first substring (anchor) as the organizing parameter. This parameter change allows the system to handle a much larger number of patterns by creating direct anchor-to-pattern mappings rather than being constrained by shift table limitations.
Data Source
AI summary
Described embodiments may provide systems and methods for matching with indexed Uniform Resource Locator (URL) pattern sets. A device may establish a hash map of patterns. Each pattern may include a suffix of a URL and indexed by a hash of a host prefix of the URL. The hash map of patterns may include a first hash of a first host prefix of a first URL. The device may identify a second URL to match. The second URL may include a second host prefix and a second suffix. The device may calculate a second hash of the second host prefix. The device may determine that the second hash matches an index of the first hash of the hash map. The device may identify, responsive to the determination, a match between a first suffix of the first URL in the hash map and the second suffix of the second URL.