Mixed multi-feature sentence similarity calculation method and system, and storage medium

Abstract

The invention requests to protect a mixed multi-feature sentence similarity calculation method and system, and a storage medium. The mixed multi-feature sentence similarity calculation method comprises the following steps: obtaining a test set and a training set for sentence similarity calculation, and obtaining a word vector corresponding to each word through a word vector model; calculating sentence word vector similarity through a computer by utilizing weighted sum to remove non-information noise from the word vectors based on a smooth inverse frequency algorithm; based on a word dependencytriple structure, respectively calculating the similarity between the test sentence and the sentence dependency syntax with the top ten screened similarities; and based on the sentence mixing similarity calculated by the two obtained sentence vectors, adjusting an optimization coefficient beta by adopting a P@N and MRR (mean sorting reciprocal) parameter determination method to obtain a sentencewith the maximum sentence similarity with the sentence in the training set. According to the mixed multi-feature sentence similarity calculation method, the characteristics of keywords, word vectors,syntactic structures and the like in sentences are considered, so that the deep meanings of the sentences are expressed more accurately, and the similarity of sentence contents is judged correctly.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to a sentence similarity calculation method. Background technique [0002] Sentence similarity calculation is based on the computer being endowed with rich meaning vocabulary, and constructs a sentence similarity calculation model through the features between sentences, so that the computer can quickly match the most similar sentences in the system. Sentence similarity calculation has a wide range of applications in various fields of natural language processing. For example, in the automatic question answering system, how to search the frequently asked questions database, how to find the corresponding answer in the knowledge base according to the user's question, and solve it by calculating the similarity between the sentence in question and the corresponding sentence in the knowledge base. In the information filtering technology, through the calcula...

AI Technical Summary

Problems solved by technology

For example, the TF-IDF method extracts keywords through the IDF algorithm, but the simple structure of the algorithm is difficult to reflect the importance of words; there are also some documents that measure the similarity of sentences by extracting common keywords, but this method misses Contribution of words other than key words to sentence meaning

Images