Document classification method and device

Abstract

The embodiment of the invention provides a document classification method and device. The document classification method comprises the steps that each participle in documents to be classified is converted into a vector by training a deep neural network language model; vectors are clustered to generate a similar participle set; the documents to be classified are converted into a characteristic frequency inverse document matrix according to the characteristics of the set; the characteristic frequency inverse document matrix is converted into a hierarchical clustering tree by calculating the similarity of the vectors of any two documents to be classified; the hierarchical clustering tree is dynamically cut at different heights based on preset end conditions to obtain classified documents. According to the method, context information of participles in specific language environments is considered during document classification, and accordingly each type of documents is higher in semantic comprehension and semantic recognition. In addition, the hierarchical clustering tree is dynamically cut at different heights based on the preset end conditions, the problem that all types of documents are large in document number difference is avoided, and document classification is more reasonable.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a document classification method and a document classification device. Background technique [0002] The explosive growth of information in the Internet has brought inconvenience to the management and use of information. In order to reveal the potentially valuable information or structure hidden behind Web data, Web data mining technology has achieved rapid development and wide application in recent years. Document clustering is one of the most important tools in the field of Web data mining. Among them, document clustering methods in the prior art mainly include K-means, hierarchical clustering methods, and the like. [0003] However, the document clustering methods in the prior art still have the following problems: the context information of the words in the document in a specific context is not considered when classifying the document, so the resulting classified document...

AI Technical Summary

Problems solved by technology

[0005] The technical problem to be solved by the embodiments of the present invention is to provide a document classification method and device to solve the common problem of low semantic understanding and low semantic recognition when document clustering methods in the prior art classify documents And the problem of unreasonable document classification

Images