Classifier construction method based on semantic computation, and classifier

Abstract

The invention belongs to the technical field of information retrieval and database structure, discloses a classifier construction method based on semantic computation, and a classifier. A neural network model is used to train word vectors on Wikipedia data to obtain distributed representation of words; vector representation of a classification is obtained through a classification label, and a weighted average method is used to obtain vector representation of a text; and by calculating the semantic relationship between a classification vector and a text vector, the most likely classification towhich the text belongs is obtained. The unsupervised learning phase does not need to data labeling, and only needs a user to define a small number of feature words to complete the creation of the classifier, the online speed is fast, and there is no need to wait for long label data accumulation; the unsupervised learning stage can make full use of the existing limited label data, and guide and improve unsupervised classification by extracting valid feature words.

Description

technical field [0001] The invention belongs to the technical field of information retrieval and database structure thereof, and in particular relates to a semantic calculation-based classifier construction method and a classifier. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: With the continuous deepening of the globalization process and the rapid development of the Internet, text data is showing explosive growth, but the data sources and forms are diverse, which brings great challenges to the management and use of documents. Text classification technology uses machine learning methods to automatically classify and mark text sets according to a certain classification system or standard, so as to realize the classification, archiving and fast query and retrieval of massive data. At present, text classification technology is relatively mature and has been widely used in many fields. The most primitive metho...

AI Technical Summary

Problems solved by technology

[0004] (1) Manual labeling of data often requires heavy manual labor and requires high labeling accuracy, which often requires three people to label the same text, resulting in long labeling work cycles, high labor costs, and slow production speed;

[0005] (2) In the absence of corpus, the model trained by the classifier is poor, which often cannot meet the effectiveness requirements of the system application

Images