Rapid hierarchical document querying method

Abstract

The invention discloses a rapid hierarchical document querying method. The rapid hierarchical document querying method comprises the following steps: establishing data models for documents of document sets, and carrying out formatting treatment on the documents to obtain a document centroid vector and a document label; taking the generated document centroid vector as a point in high-dimensional vector space, and constructing a hash index structure in a memory for every document set by using a locality sensitive hash method; acquiring a candidate document set in the hash index structure by using a querying method based on locality sensitive hash thinking according to the document centroid vector of a queried text; and acquiring a nearest neighbor document under word movement distance measurement in a candidate document set by using a filtering-thinning hierarchical frame according to the document label of the queried text. When the designed hierarchical querying method is applied to document classification and retrieval, the efficiency and the effect are balanced well, and a target document is acquired rapidly under the accuracy can be guaranteed when the user queries documents under word movement distance measurement.

Description

technical field [0001] The invention relates to a fast hierarchical document query method, in particular to a Word2Vec model in the field of machine learning, a locally sensitive hash method in the field of databases, and a filtering-refining framework under bulldozer distance measurement. Background technique [0002] With the development of information technology, people's ability to produce, collect and store information has been continuously enhanced. One of the main information carriers is documents. Accurately expressing the similarity between two documents has a wide range of applications in document retrieval, document classification and document clustering. The latent semantic analysis method is to extract low-dimensional semantic information through matrix decomposition, and the topic model is a method to model the hidden topics in the text. Recently, with the development of deep learning, the Word2Vec model and the Doc2Vec model have been proposed one after anoth...

AI Technical Summary

Problems solved by technology

Since bulldozer distance is usually formalized as a linear optimization problem and can be modeled as the minimum cost flow of bipartite networks, it requires a large time complexity

Images