Large-scale and multi-key word matching method for text or network content analysis

Abstract

A large-capacity multikey-word matching method used on content analysis of text or network includes setting up jump table and key word table, calculating jump value of each jump table item and correlating key word with key word table item, carrying out Hash operation on data block in window, indexing jump value in jump table and moving window as per said value if jump value is not zero or otherwise making Hash operation again data block, indexing key word table and comparing key word correlated with said table item with field in text in sequence for confirming whether they are matched with each other or not.

Description

technical field [0001] The invention relates to a large-scale multi-keyword matching method for text or network content analysis, in particular to a text or network content processing technology under the background of a large-scale keyword set, and belongs to the technical field of computer data processing. Background technique [0002] Multi-keyword matching is one of the fundamental problems in the field of computer science. The problem it solves is to quickly and accurately judge whether a certain text or data block contains one or some keywords in a given keyword set. Multi-keyword matching technology has now been widely used in various fields of network security such as firewalls, virus detection, intrusion detection and defense, and content filtering. It can also be extended to other disciplines, such as information management systems, network search engines, and bioinformatics. Gene sequence detection in etc. Therefore, the research and improvement of multi-keyword...

AI Technical Summary

Problems solved by technology

[0007] However, through the analysis of the WM algorithm process, it can be known that after the number of keywords increases, the number of items with a jump value of zero in the jump table of the WM method will increase, and the average jump value will decrease accordingly, so that the matching process cannot be effectively avoided. A large number of unnecessary character comparisons, resulting in a slowdown in the matching speed of text or network content

The experimental results show that when the number of keywords reaches 100,000, the matching speed of the WM method drops significantly, and the extremely low throughput can hardly meet the practical requirements of text or network content processing.

Images