Method for improving vector distance classifying quality

Abstract

The invention discloses a method for improving the vector distance classifying quality. The method comprises the following steps that the characteristic weighting process based on VSM is carried out, wherein the characteristic weighting process comprises the characteristic item word frequency weighting based on the word meaning and the characteristic item frequency weighting based on a document structure; word stems of English searching are extracted; a user query log is analyzed; a training corpus is corrected and expanded. The main defect of VSM formed by contradiction between the characteristic entry mutual independence requirement and the natural language diversity is overcome by constructing a segmentation dictionary, the classifying quality and the algorithm efficiency are improved while the classifying algorithm processing is simplified through the word stem processing technology, the query requirement in which a user takes great interest is obtained by analyzing the query log of the user and is used for guiding and amending the terminological dictionary, then the corpus can be dynamically changed along with the improvement of various special technologies through the amending and expanding of the training corpus, and a Robot program is guided to collect the latest special technical data.

Description

Technical field [0001] The invention relates to a technology for improving the quality of automatic classification, in particular to a method for improving the quality of vector distance classification. Background technique [0002] The key problem of automatic document classification is how to construct a classification function or classification model (also called a classifier), and use this classification model to map unknown documents to a given category space. There are many construction algorithms for classifiers, mainly including probability and statistics algorithms, machine learning algorithms, and neural network algorithms. The probability and statistics algorithm uses a relatively simple mechanism. It has achieved satisfactory results in processing large-scale real documents. [0003] The idea of ​​the simple vector distance algorithm is very simple. According to the arithmetic average, it generates a center vector for each type of document set through sample training;...

AI Technical Summary

Problems solved by technology

[0006] Although these algorithms have achieved good classification results in some fields or in some specific occasions, their classification quality and classification technology still need to be further improved.

Images