Mass-scale mail address matching method

Abstract

The invention discloses a mass-scale mail address matching method. A Hash model with the excellent performances in mass-scale Hash time consumption and Hash conflict processing is selected as a Hash function of the matching method. Because the repetition probability of the domain names of mail addresses is high, a Bloom filter is used for the storing and matching of the mail addresses, so as to reduce the memory consumption of the method and improve the matching efficiency of the method. A red-black tree is used for string a mode string after the Hash conflict, thereby improving the full textmatching performance of the algorithm. The method achieves the optimization of a storage structure and a matching flow of a WM matching method. A high-efficiency Hash model (BKDRHash) is employed forreducing the Hash conflict. The Bloom filter is used for the storing and matching of the domain names of the mail addresses, thereby avoiding the repeated storage of the domain names of the mail addresses, and reducing the memory consumption of the matching method. The red-black tree is used for processing the elements generating Hash conflicts, thereby reducing the full-text matching time consumption.

Description

technical field [0001] The invention relates to a Wu-Manber multi-pattern matching method. Background technique [0002] Wu-Manber algorithm based on hash filtering idea, referred to as WM algorithm. The WM algorithm combines the "bad character" idea of ​​the BM algorithm, and is a typical algorithm based on suffix scanning, but in practical applications, the WM algorithm uses character blocks. This means that after the matching fails, the jumping distance of the matching pointer becomes larger, which improves the matching performance of the algorithm. When the WM algorithm is matching, it uses a hash table to select a subset of the pattern string set to match the full text of the current text, reducing unnecessary matching operations. The optimal time complexity of WM algorithm can reach O(B n / m) (B is the length of the character block, m is the shortest pattern length). The execution time of the WM algorithm will not increase proportionally with the increase of the patt...

AI Technical Summary

Problems solved by technology

[0004] 1) The method is optimized for the characteristics of the email address, and the memory consumption of the method needs to be optimized;

[0005] 2) In massive scale matching scenarios, hash collisions are serious

[0006] 3) Full-text matching takes a lot of time in massive scale matching scenarios

The pattern strings that generate hash conflicts are stored in the linked list. This is the traditional processing method. When the full text is accurately matched, it is necessary to traverse the linked list one by one to check whether the match is successful. Although the PREFIX table can reduce the number of exact matches and improve the matching Efficiency, but this sequential search method, for a large collection of pattern strings, the performance improvement effect of the PREFIX table is not obvious

Images