A fine-grained semantic detection method for bad text content on the Internet

Abstract

The invention belongs to the technical field of text content filtration, and particularly relates to a fine-grained semantic detection method of harmful text contents in network. Aiming at an introduced harmful information scene, the method comprises the steps of: constructing a train text set in which independent sentences are used as basic units, thereby establishing a mathematic description of the scene by using a probability topic model; performing information content extraction to a Web page to be detected; performing sentence identification to the text information; calculating a condition probability of each sentence under the model based on the established probability topic model; and accomplishing the fine-grained semantic detection under the set content detection sensitivity. According to the invention, the model construction is hardly affected by the number of the topics, and probability calculation on the sentence and word level is carried out effectively, so that the method is applicable for various application circumstances requiring harmful text content detection; furthermore fine-grained detection to harmful words and sentences of the text content is supported, so that the method improves the detection rate and reduces the misinformation rate effectively, and is beneficial to improving the practicability of text content filtration.

Description

technical field [0001] The invention belongs to the technical field of text content filtering, and in particular relates to a detection method for network bad text information content. Background technique [0002] At present, the Internet has become a main way and space for creating and sharing information. With the continuous emergence of various online forums and social interactive media, a large amount of text information is generated every day, such as various news reports, product introductions, etc. , various online comments and so on. Among them, a large amount of bad text information content is full of various network spaces. The emergence of pornographic information, violent information, online abuse and other harmful information has caused great harm to the healthy growth of young people, and for office workers, continuous browsing of such information also leads to low work efficiency. Therefore, the detection of bad text information content has become an import...

AI Technical Summary

Problems solved by technology

However, these two types of text sets are not easy to construct in practical applications, resulting in poor detection performance

[0006]3. Although the detection method using simple semantic analysis technology such as LSA (Latent Semantic Analysis) [1] takes semantic recognition into consideration, there are difficult problems in the establishment of semantic space and the process of semantic extraction, such as the setting of the space dimension. The problem makes the performance not high in the face of flexible and changeable text content detection

Existing methods have deficiencies in vocabulary setting, training text setting, and semantic space construction, and it is still difficult to meet the requirements for detection and filtering of bad semantic text content

Images