Text filtering system and method

Abstract

The invention discloses a text filtering system and a text filtering method. The method comprises the following steps of: establishing a filtering model according to the filtering requirement of a user; training a group of filtering samples to form a body library which is close to the filtering requirement of the user; and extracting characteristic words of a text to be filtered, identifying entities in the characteristic words, extracting an entity relation to form an entity relation vector of the text to be filtered, calculating the similarity of the filtering model and the text to be filtered, and filtering the text which is higher than a similarity threshold value. The characteristics of filtered texts are expressed accurately through extraction of entity relations according to the established filtering model of the user, so that the filtering accuracy can be increased.

Description

technical field [0001] The present invention relates to a text filtering system and method, in particular to a text filtering system and method based on entity relationship extraction. Background technique [0002] Text filtering has received more attention for many years, and has a good application prospect in the fields of information retrieval and filtering. Among the current text filtering methods, some use the fuzzy clustering method based on genetic algorithm to directly cluster each individual in the population with a fuzzy similarity matrix, and then use the proposed fitness function to evaluate the population according to the clustering results. However, the filtering accuracy of this method depends on the effect of clustering, and it cannot express the user's filtering needs well. Some use improved classification algorithms to filter bad text information, and improve the traditional KNN algorithm from the perspective of the data layer, which is also not accurate e...

AI Technical Summary

Problems solved by technology

Among the current text filtering methods, some use the fuzzy clustering method based on genetic algorithm to directly cluster each individual in the population with a fuzzy similarity matrix, and then use the proposed fitness function to evaluate the population according to the clustering results. However, the filtering accuracy of this method depends on the effect of clustering, and it cannot express the user's filtering needs well.

Some use improved classification algorithms to filter bad text information, and improve the traditional KNN algorithm from the perspective of the data layer, which is also not accurate enough to express the needs of users

Some filtering methods also use ontology to express the user's filtering needs, but the establishment method of the ontology database expressing the user's filtering needs is not accurate enough, which will greatly affect the filtering accuracy of the text

Some filtering algorithms use adaptive learning text filtering. Although the user's filtering template can be adaptively learned and the filtering model can be adjusted, the method of using feature vectors cannot accurately express the user's filtering needs.

Images