A Calculation Method of Question Similarity Based on Multiple Features

Abstract

The invention discloses a problem similarity calculation method based on a plurality of features, includes steps: For the input new question sentence, Compared with the stored historical questions andcorresponding answers, the similarities between the new questions and the historical questions are calculated based on character features, semantic features of words, semantic features of sentences,implied topic features of sentences and semantic features of answers. The final similarity is the product of the above five similarities and their corresponding weights, which are trained by linear regression method. The invention adopts a plurality of features to increase the diversity of sample attributes, and improves the generalization ability of the model. At that same time, the soft cosine distance is utilized to convert the TF-IDF is fused with editing distance, word semantics and other information, which overcomes the semantic gap between words and improves the accuracy of similarity calculation.

Description

technical field [0001] The invention relates to the research field of computer natural language processing and automatic question answering system, in particular to a method for calculating question similarity based on multiple features. Background technique [0002] With the rapid increase of digital information, it is more difficult for people to obtain the information resources they need from the Internet. How to accurately and quickly find the required information for users in the massive digital information has brought severe challenges to natural language processing (NLP) technology and information retrieval technology. Therefore, in order to provide users with real-time and high-precision information acquisition channels, research institutions and related technology companies have begun to study automatic question answering systems (QA). In the automatic question answering system, users only need to enter a question to get the corresponding answer directly. It is no ...

AI Technical Summary

Problems solved by technology

However, this method relies too much on the semantic dictionary, and the accuracy of similarity calculation is affected by the completeness and correctness of the semantic dictionary; similarly, the similarity calculation method based on semantics is less effective in dealing with long sentence patterns with complex syntax

[0010] At the same time, most of the methods in the prior art are based on a single type of information to extract text representation features, focusing on a single type of feature, without considering that the meaning of text representation is composed of multi-faceted and multi-level information, so the accuracy of calculating similarity is not high. high

Images