Context similarity calculation-based word sense disambiguation method

Abstract

The invention relates to a context similarity calculation-based word sense disambiguation method. The method comprises the steps of processing training corpora, and training a model by using a part-of-speech tagging version of ukWaC; screening parts of speech, and only reserving notional words including nouns, adjectives, adverbs and verbs; training a bidirectional LSTM model by using the corporasubjected to part-of-speech screening; inputting example sentences of to-be-disambiguated words to the bidirectional LSTM model to obtain context vectors; inputting contexts of the to-be-disambiguatedwords to the bidirectional LSTM model to obtain context vectors of the to-be-disambiguated words; and calculating cosine similarity for the context vectors of the to-be-disambiguated words and the context vectors of the example sentences, and further selecting semanteme of the to-be-disambiguated words by utilizing a k-neighbor method according to an obtained similarity result. According to the method, the semanteme is better modeled; the words and the parts of speech are combined by using an underline behind the words directly; obtained word vectors well distinguish different parts of speechof the same word; and the disambiguation accuracy is improved by 0.5% on an experimental basis of baselines.

Description

technical field [0001] The invention relates to a natural language translation technology, in particular to a word meaning disambiguation method based on context similarity calculation. Background technique [0002] Word sense disambiguation, or WSD for short, is a problem with a long history and a wide range of applications. At present, it can be divided into three categories: supervised methods, unsupervised methods and knowledge-based methods. Although published supervised word sense disambiguation systems perform well when given large-scale training corpora with specific semantics, the lack of large-scale annotated corpora is the main problem. Using pre-trained word vectors can solve this problem to some extent. Because the word vectors pre-trained on a large-scale corpus contain more semantic and grammatical information, using it to train a supervised system will improve performance. In order to infer the meaning of words in a sentence, both the target word and the c...

AI Technical Summary

Problems solved by technology

[0004] Aiming at the deficiencies in the word sense disambiguation in the prior art that regard different parts of speech of a word as one point for modeling, resulting in poor disambiguation accuracy, the problem to be solved by the present invention is to provide a method that can use different parts of speech of the same word A word sense disambiguation method based on contextual similarity calculation that distinguishes well and improves disambiguation accuracy

Images