Synonym mining method and device for question and answer retrieval system

Abstract

The invention relates to a synonym mining method and device for a question and answer retrieval system. According to the invention, question and answer corpora are classified; keyword extraction is carried out by category to obtain a set of keywords to be processed, meanwhile, word vector training is carried out on large corpora in the vertical field; cosine similarity of the word vectors is calculated to obtain a generalized related word set of the keywords of the current category; part-of-speech screening is performed to obtain a set of thumbnail-related words, an Euclidean distance is calculated in the thumbnail related word set; synonym pairs are obtained, the co-occurrence frequency of the synonym pairs is counted; the replacement probability of the synonyms is calculated, finally, feedback correction is conducted on the synonym pairs which do not meet the retrieval recall threshold value according to the retrieval recall result obtained after replacement of the synonyms, the semantic deformation problem obtained after replacement of the synonyms is well solved, and the accuracy of synonym mining and the accuracy of question and answer pair retrieval results are improved.

Description

technical field [0001] The invention relates to the field of information retrieval, in particular to a synonym mining method and device for a question-and-answer retrieval system. Background technique [0002] With the rapid development of Internet technology, in the face of massive information and resources, traditional search engines cannot well meet people's comprehensive, fast, and accurate knowledge acquisition needs. Intelligent question-and-answer retrieval systems based on accurate knowledge acquisition will become the future development direction. In recent years, with the continuous rise of artificial intelligence and the development and progress of technology, the application of question and answer retrieval system has penetrated into all walks of life, and has gradually become a very practical and popular way of knowledge acquisition. [0003] Synonym replacement, as an important technology of question-answer retrieval system, is the basic and necessary work in ...

AI Technical Summary

Problems solved by technology

[0010] In order to overcome the above-mentioned deficiencies, the present invention aims to provide a synonym mining method and device for a question-and-answer retrieval system. The present invention classifies the question-and-answer corpus and extracts keywords by category to obtain a keyword set to be processed. The large corpus in the field conducts word vector training, and calculates the cosine similarity of the word vector to obtain the generalized related word set of the current category keywords, and then performs part-of-speech screening to obtain the abbreviated related word set, and then calculates the abbreviated related word set The Euclidean distance of the synonym pairs is obtained, and the co-occurrence frequency of the synonym pairs is counted, and the replacement probability of the synonyms is calculated. Finally, according to the search and recall results after the replacement of the synonym pairs, the synonym pairs that do not meet the retrieval and recall threshold are used. Feedback correction is better. It solves the problem of semantic deformation after synonym replacement, improves the accuracy of synonym mining and the accuracy of question-and-answer retrieval results

Images