Text filtering system and method

Abstract

The invention discloses a text filtering system and a text filtering method. The system at least comprises an ontology base construction module, an adaptive learning module and a text filtering module, wherein the ontology base construction module is used for constructing an ontology base according to the filtering requirements of a user; the adaptive learning module dynamically regulates the ontology base constructed by the ontology base construction module by performing training and learning on a group of filtering samples to make the ontology base gradually meet the filtering requirements of the user; and the text filtering module performs preprocessing, characteristic word set extraction and similarity matching on a text to be filtered to obtain relevance between the text to be filtered and an ontology, and filters the text to be filtered according to the relevance. By the system and the method, a filtering model for the user can be accurately expressed; and in the filtration, the filtering model expressed by the ontology for the user can be regulated by automatic learning, and a filtering threshold value can be dynamically regulated to achieve a good filtering effect.

Description

technical field [0001] The invention relates to a text filtering system and method, in particular to an ontology-based adaptive text filtering system and method. Background technique [0002] In the field of information retrieval and filtering, text filtering has always been a research hotspot. At present, many domestic and foreign literatures have adopted different methods to realize text filtering. [0003] The current text filtering methods mainly include the fuzzy clustering text filtering method based on genetic algorithm, the text filtering method using improved classification algorithm, the text filtering method using adaptive learning filtering algorithm and the text filtering method only using ontology. Among them, the fuzzy clustering method based on genetic algorithm is used to directly cluster each individual in the population with a fuzzy similarity matrix, and then use the proposed fitness function to evaluate the fitness of the population according to the clu...

AI Technical Summary

Problems solved by technology

Among them, the fuzzy clustering method based on genetic algorithm is used to directly cluster each individual in the population with a fuzzy similarity matrix, and then use the proposed fitness function to evaluate the fitness of the population according to the clustering results. However, this The filtering accuracy of this text filtering method depends on the effect of clustering, and it cannot express the user's filtering needs well; the text filtering method using an improved classification algorithm filters bad text information, and improves the traditional one from the perspective of the data layer. The disadvantage of the KNN algorithm is also that it is not accurate enough to express the user's needs; the text filtering method using the adaptive learning filtering algorithm can perform adaptive learning by training the sample set, and can adjust the filtering model, but its filtering effect on the user The expression of requirements is also not precise enough; only the ontology text filtering method is used, and the filtering accuracy depends on the establishment of the ontology. If the ontology library is not created accurately, it will greatly affect the accuracy of text filtering

[0004] To sum up, it can be seen that in the text filtering method of the prior art, there is a problem that the expression of the user's needs is not accurate enough or the creation of the ontology database is not precise enough to affect the accuracy of the text filtering. Therefore, it is necessary to propose an improved technical means to solve this problem.

Images