Multi-subject extraction method based on concept vector model

Abstract

The invention provides a multi-subject extraction method based on a concept vector model. The method includes the following steps that firstly, a document is preprocessed through a traditional method and then vectors formed by feature words are preliminarily acquired; then synonyms are merged through the corresponding relation between word meanings and concepts in Hownet, disambiguation is conducted on polysemes through correlation between semantic classes and contexts, and the concept vector model is established to represent the document; concept similarity is calculated through related semantic information of the concepts in Hownet, a K-means algorithm is improved through a 'preset seed' method for clustering of the concepts, and then a plurality of subject concept clusters are formed; eventually, according to the corresponding relation between the concepts and words, a plurality of sub subject term sets are acquired. According to the method, semantic information is taken into consideration, the defects of sensitivity of the K-means algorithm to an initial center, space-time cost instability and the like are overcome, and the quality of extracted subjects is improved.

Description

technical field [0001] The invention relates to the technical field of text information extraction, in particular to a multi-subject extraction method based on a concept vector model. Background technique [0002] Since human society entered the information age, various electronic texts have emerged in large numbers. Among these massive texts, there are a large number of multi-theme texts, which contain rich and diverse subject information. For example: a report on Premier Li Keqiang’s visit to Europe is a political news, but also economic news. With the development of science and technology, the degree of integration between disciplines is getting higher and higher, most of the research spans multiple disciplines, and many scientific and technological texts contain multiple topics from different aspects, such as a text on biological gene information mining , covering both computer science and biomedical domain topics. Therefore, there are a large number of multi-theme tex...

AI Technical Summary

Problems solved by technology

[0006] The technical problem to be solved by the present invention is that in order to solve the problem that traditional text processing technology is based on word frequency statistics, a text can only propose a single topic. In order to solve the problems of low algorithm efficiency and low quality of extracted subject words caused by context information, a multi-topic extraction method based on concept vector model is provided. Carry out one-to-one mapping, express the text as a concept model, and automatically merge synonyms into the same concept during the mapping process, realizing vector dimensionality reduction; disambiguate polysemy words appearing in the text according to the correlation between semantic classes and context

Images