KNN text classification method based on improved K-Medoids

Abstract

The invention provides a KNN (K-Nearest-Neighbor) text classification method based on improved K-Medoids and relates to the field of computer text data processing. The method comprises the following steps: pre-processing a training text set and a testing text set, wherein preprocessing comprising removal of participles and stop words, DF feature selection and vector representation, so as to obtain a training text vector space and a testing text vector space; carrying out training sample clipping on the basis of an improved K-Medoids method, namely, optimizing from the points of initial center point selection and replacement of center point search strategy, and applying optimization to the training sample clipping so as to obtain a new training text space; and finally, carrying out KNN classification, defining a representative function and applying the representative function to class attribute functions for KNN classification so as to obtain a final result. Experimental results show that compared with a conventional KNN method and a KNN method based on the K-Medoids, the KNN text classification method provided by the invention has higher classification accuracy and classification efficiency.

Description

technical field [0001] The invention relates to the field of computer text data processing, in particular to a K-nearest-neighbor (K-Nearest-Neighbor, KNN) text classification method based on improved K-Medoids. Background technique [0002] With the development of the Internet, the Internet of Things, and cloud computing, data is growing exponentially, leading us into the era of big data. The Internet Data Center (IDC) of the United States pointed out that the data on the Internet is increasing at a rate of 50% every year, and more than 90% of the data in the world is generated in recent years. At present, the amount of global data has reached the ZB level, and with the generation of a large amount of data, there is also great potential value contained in it. [0003] In today's era of big data, it is very important to mine the potential value of data. As a technology to discover the potential value of data, data mining has attracted great attention. Text data accounts f...

AI Technical Summary

Problems solved by technology

[0004] As one of the classic classification methods, KNN has the advantages of simple implementation and high robustness; but there are also many shortcomings that make it unsuitable for many practical applications.

The shortcomings of KNN mainly include the following two aspects: First, the classification process consumes a lot of time due to the huge amount of similarity calculations, resulting in low classification efficiency.

Second, the classification performance is easily affected by the training samples. When the data is severely unevenly distributed, the performance of the classifier may be seriously affected, or even become extremely poor.

Images