Content filtering method and device

Abstract

Embodiments of the present invention provide a content filtration method and device. The method comprises: respectively extracting a keyword from entered rule conditions; dividing the rule conditions into one or more groups according to the extracted keyword, and pre-compiling a group matching dataset for the extracted keyword; respectively pre-compiling a precise matching dataset for the rule conditions of the groups corresponding to the extracted keyword; obtaining to-be-filtered content; using the group matching dataset to perform keyword matching on the to-be-filtered content; using the precise matching dataset of the rule conditions of the groups corresponding to the matched keyword to perform precise matching of the rule conditions on the to-be-filtered content; and executing a corresponding filtration policy according to a matching result of the precise matching. The present invention performs group pre-filtration on the rule conditions; therefore the number of the rule conditions in each group is small, and occupied memory is reduced. However, the precise matching based on the rule conditions after the group pre-filtration has a higher matching accuracy.

Description

technical field [0001] Embodiments of the present invention relate to data processing technologies, and in particular, to a content filtering method and device. Background technique [0002] As the largest information center in the world, the Internet is growing at an astonishing speed, but the information in it is uneven, and there are many bad websites and bad resources. In addition, there are also some suspicious websites that contain malicious software, which can threaten the user's personal privacy and even damage the user's computer. [0003] In order to avoid the harm of bad information, the prior art adopts the content filtering technology based on the application layer protocol to filter the webpage. For example, for an enterprise network gateway, filtering policies can be configured to filter webpages with certain types of content, so as to restrict the prohibited behaviors of internal users of the enterprise network, such as prohibiting access to inappropriate we...

AI Technical Summary

Problems solved by technology

[0006] However, this content filtering technology in the prior art has relatively large defects.

The rule condition matching method used for URL address content filtering is carried out by using the DFA graph. When the number of rule conditions is too large or complex rule condition configuration is required, for example, regular expressions including wildcards, such as ".* / abc .* / news", ".*\.www\.domain.*\.com", etc., you will encounter the problem of consuming a lot of memory

This is the main shortcoming of the DFA algorithm. The existing technology can use a compressed DFA, such as the D2FA (Delayed DFA) algorithm instead of the standard DFA for matching, but it will cause low matching performance, because the time efficiency of the D2FA algorithm is several times lower than that of the standard DFA.

Images