A knn text classification method based on improved k-medoids

Abstract

A KNN text classification method based on improved K‑Medoids, which involves the field of computer text data processing; first, the training text set and test text set will be preprocessed, including word segmentation, stop word removal, DF feature selection and vector Indicates that the training text vector space and the test text vector space are obtained; then the training sample clipping based on the improved K-Medoids method is carried out, that is, optimization is carried out from the perspective of the initial center point selection and the replacement center point search strategy, and it is applied to the training The sample is clipped to obtain a new training text space; finally, KNN classification is performed, and the representativeness function is defined, which is applied to the category attribute function for KNN classification to obtain the final result. Experimental results show that, compared with traditional KNN methods and K-Medoids-based KNN methods, the present invention has higher classification accuracy and classification efficiency.

Description

technical field [0001] The invention relates to the field of computer text data processing, in particular to a K-nearest-neighbor (K-Nearest-Neighbor, KNN) text classification method based on improved K-Medoids. Background technique [0002] With the development of the Internet, the Internet of Things, and cloud computing, data is growing exponentially, leading us into the era of big data. The Internet Data Center (IDC) of the United States pointed out that the data on the Internet is increasing at a rate of 50% every year, and more than 90% of the data in the world is generated in recent years. At present, the amount of global data has reached the ZB level, and with the generation of a large amount of data, there is also great potential value contained in it. [0003] In today's era of big data, it is very important to mine the potential value of data. As a technology to discover the potential value of data, data mining has attracted great attention. Text data accounts f...

AI Technical Summary

Problems solved by technology

[0004] As one of the classic classification methods, KNN has the advantages of simple implementation and high robustness; but there are also many shortcomings that make it unsuitable for many practical applications.

The shortcomings of KNN mainly include the following two aspects: First, the classification process consumes a lot of time due to the huge amount of similarity calculations, resulting in low classification efficiency.

Second, the classification performance is easily affected by the training samples. When the data is severely unevenly distributed, the performance of the classifier may be seriously affected, or even become extremely poor.

Images