A Query Word Rewriting Method Fused with Word Vector Model and Naive Bayesian

Abstract

The invention provides an inquiring term rewriting method merging a term vector model and naive Bayes and relates to information processing technologies. The method includes the steps that word2vec is used for training the vector model, first N terms most similar to an inquiring term are calculated, and an initial relevant lexicon is formed; then, relevancy calculation and analysis are conducted, candidate terms for inquiring and rewriting the lexicon are filtered, and terms with high relevancy are reserved. The method can effectively improve the accuracy and recall degree of inquiring results and effectively solve the problem that search inquiring is in vain or few valid results are sent back.

Description

Technical field [0001] The invention relates to the technical field of computer information processing, in particular to a query word rewriting method in data mining technology. Background technique [0002] The word vector model is a technology that uses neural networks to map each word in a high-dimensional discrete space (the number of words in the dictionary) into a real number vector in a low-dimensional continuous space (that is, word embedding). In natural language processing tasks, word embedding provides a better semantic-level word distributed feature representation, which brings many conveniences to text processing tasks. The goal of word embedding representation is to learn the vector representation of each word and use this vector representation for different text processing tasks. The learned word vector can be used as a complete word feature input into the supervised learning algorithm of some specific tasks, or it can be used as a beneficial extension that depend...

AI Technical Summary

Problems solved by technology

[0004] The Skip-Gram model of the Hierarchical Softmax algorithm is used for query rewriting. The rewritten words are only calculated from the semantic relevance, without considering the connection between the query word and the context of the search recall results, and there are not many semantically related words. , requires a lot of anticipation to be effectively mined; while the query rewriting method based on Naive Bayes mines rewritten words from the co-occurrence probability between the query word and the context of the search recall result, although the context between , but the semantic correlation between the query term and the rephrased term is weak

Images