Doc2vec-based similar entity mining method

Abstract

The invention belongs to similar document mining problems in natural language processing, relates to the technical field of word embedded expression, document keyword extraction, document embedded expression and nearest neighbor rapid calculation in high-dimensional space, and discloses a Doc2vec-based similar entity mining method. According to the method, similar entities are effectively mined through description documents of the entities by using Word2vec word embedded expression, TFIDF document keyword extraction, converting the description documents of the entities into continuous and dense vectors by using Doc2vec, and using Balltree data structures.

Description

technical field [0001] The invention belongs to the problem of mining similar documents in natural language processing, and relates to technical fields such as word embedding expression, document keyword extraction, document embedding expression, nearest neighbor fast calculation in high-dimensional space and the like. Background technique [0002] In many fields such as search, machine reading comprehension, user portraits, and recommendation systems, similar word mining, similar document mining, and more specifically similar APP or similar public account mining play a key role. For similarity mining, one of the most direct methods needs to map words or documents into a high-dimensional space, that is, word embedding or document embedding. [0003] At present, the most mainstream and successful method of word embeddings is Word2Vec technology. The technique is a neural probabilistic language model, which was first proposed by Bengio Y et al. The neural probabilistic langu...

AI Technical Summary

Problems solved by technology

This makes the tree more complex to construct, but very efficient at computing nearest neighbors, even in very high dimensions

Images