Short text query expansion and indexing method based on word vector

Abstract

The invention discloses a short text query expansion and indexing method based on a word vector. The short text query expansion and indexing method particularly comprises: A, pretreatment of corpus information of a short text; B, expression of every word in a corpus dictionary by the word vector through a training model; C, query extension; D, obtaining of a text candidate set through a query expansion word set and a BM25 index model; E, extraction of subject of the short text; F, calculation of the text vector of the short text; G, re-sequencing of the short text returned by a traditional indexing model. The short text query expansion and indexing method can more exactly and effectively satisfy the indexing demand of a user; moreover, the query expansion module can find out words capable of expressing user's intension according to the existing data so as to perform the query expansion.

Description

technical field [0001] The invention relates to the technical fields of data mining and search engines, in particular to a short text query expansion and retrieval method based on word vectors. Background technique [0002] With the rapid development of computer and Internet, it becomes more and more difficult to accurately obtain information from massive information resources. A large part of the massive information exists in the form of short text, and short text is also an indispensable data form in people's daily life. Short text information mainly includes blog messages, microblog information, short messages, chat records, etc., and is characterized by short message length, flexible language form, huge data scale, strong timeliness, and fast update speed. Traditional search engines are not very accurate in these short text retrievals, and cannot meet people's needs for accurate information acquisition. Therefore, the present invention designs and implements a search en...

AI Technical Summary

Problems solved by technology

The disadvantage of this type of method is: when the user gives a search term, the search engine can only return documents containing the search term, but cannot return other documents that are semantically related but expressed in different words

The disadvantage of this type of method is: when the user gives a search term, the search engine will introduce a lot of noise information, although the recall rate of the retrieval system is improved to a certain extent, but it also introduces a large amount of irrelevant text, reducing the search accuracy

[0016] These methods only enrich the representation of query words semantically, but they do not attempt to understand the user's query intent, but find words that are similar to each word for query expansion, which can easily lead to problems such as topic deviation and introduction of noise.

Images