Text filtering method and system based on keyword weight value

Abstract

The invention provides a text filtering method based on a keyword weight value. The method comprises the following steps that the weight value of a keyword is calculated; a text is filtered based on the calculated weight value of the keyword; the process of calculating the weight value of the keyword comprises the steps of judging whether the keyword is a brand-new keyword or not, calculating the number of accurate judgment data and the number of wrong judgment data in historical judgment data and the number of accurate judgment data including the keyword and the number of wrong judgment data including the keyword if the keyword is the brand-new keyword, and calculating the weight value of the keyword. The invention further provides a text filtering system based on the keyword weight value.

Description

technical field [0001] The present application relates to the fields of security and data services, and in particular to a text filtering method and system based on keyword weights. Background technique [0002] Text information is the most disseminated content in mobile Internet information, including: web pages, short messages, multimedia messages, instant messaging tools, etc. Information filtering (such as politics, pornography, gambling...) for sensitive content in text transmission is an important technology in the Internet. In general, the system categorizes text as "OK" and "Needs filtering." [0003] From the perspective of information volume, the amount of text data accessed by users on each link (10G) is as many as hundreds of millions per day, and the entire network has hundreds of billions of data, and the proportion of information that needs to be filtered is very small, generally less than 1%, so it is difficult to accurately capture the information to be fi...

AI Technical Summary

Problems solved by technology

[0020] In order to solve the problem that the system judges the text with low accuracy, the application provides a text filtering method based on keyword weight, which includes the following steps: calculating the weight of the keyword; and based on the calculated keyword The weight of the text is filtered; wherein the step of calculating the keyword weight includes: judging whether the keyword is a new keyword, if not, calculating the number M of correct judgment data and the number M of wrong judgment data in the historical judgment data Number N, and the number M1 of correct judgment data containing keywords and the number N1 of wrong judgment data; and calculating the keyword weight Value 0 = VL + M 1 / M M 1 / M + N 1 / N ( VH - VL ) . Among them, VL is the minimum weight value of the keyword set by the user, and VH is the maximum weight value of the keyword set by the user.

Images