Label extraction method based on short text clustering technology

Abstract

The invention discloses a label extraction method based on short text clustering. The label extraction method comprises: S1, extracting and obtaining all useful vocabularies of a short text; S2, calculating the similarity among the short texts by utilizing word2vec according to the text characteristics of the useful vocabularies of each short text; S3, assuming that each short text is a cluster center, calculating the similarity between each cluster center and all other short texts, and if the similarity is greater than a preset threshold T1, classifying the corresponding short text into the cluster center; S4, trimming each cluster formed in the S3 for the first time; S5, combining all the clusters trimmed for the first time; S6, performing secondary pruning on the clusters merged in theS5; and S7, extracting the cluster center of each cluster after the second trimming as a label of each cluster. Labels which can represent meanings of all types of short texts and have complete meanings are generated while the short texts are clustered, and the problems that an existing clustering technology depends on center point selection and an existing label extraction technology is incomplete in semantics, does not accord with logic and the like are solved.

Description

technical field [0001] The invention relates to the technical field of information processing, in particular to a label extraction method based on short text clustering. Background technique [0002] With the development of the Internet and information technology, various network information shows an exponential growth trend, especially the rise of network platforms such as Weibo, which makes short text information explode again. Short text data information is sparse but focused, and cannot be eliminated as spam. How to obtain effective information from a large amount of short text data requires an effective method to improve the effect of short text clustering and hotspot discovery. At present, many platforms on the Internet use manual methods for label planning, which is not only time-consuming and laborious, but also has great limitations. For example, the coverage of manually customized labels is limited and can only contain text with fixed meanings. The text needs to ...

AI Technical Summary

Problems solved by technology

[0009] Another purpose of the present invention is to provide a label extraction method based on short text clustering, which can represent all categories of short text meanings while short text clustering The label with complete meaning solves the problem that the existing clustering technology relies on the center point selection and the semantics of the existing label extraction technology is not perfect and illogical.

Images