A Word Vector Generation Method Based on Gaussian Distribution

Abstract

The invention discloses a method for generating word vectors based on Gaussian distribution. Firstly, the corpus is preprocessed; secondly, punctuation marks are used to divide the context of the corpus; Optimize the objective function to obtain word vectors. The innovations and beneficial effects of the technical solution of the present invention are as follows: 1. Words are represented based on Gaussian distribution, which avoids the point estimation characteristics of traditional word vectors, and can bring probability quality to word vectors, and richer information such as word meaning implication and inclusion relationship . 2. Using multiple Gaussian distributions to represent words can cope with the polysemy of a word in natural language. 3. The similarity between Gaussian distributions is defined based on the Hellinger distance, and the parameter update and word sense discrimination are combined to adaptively infer the number of word meanings, which solves the problem of the assumption that the number of word senses in the prior art model is fixed.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to a word vector generation method based on Gaussian distribution. Background technique [0002] Word vector is a method for mathematical modeling of words in Natural Language Processing (NLP). The earliest word vector representation is one-hot code (One-Hot). The one-hot code represents each word as a high-dimensional vector, and the position of 1 in the vector is the index of the word in the dictionary. One-hot codes have disadvantages such as high dimensionality, sparseness, and ignoring semantic and syntactic information. With the development of deep learning, methods of using neural networks to train word vectors have emerged. The neural network-based word vector model captures the syntactic and semantic information of the context for the co-occurrence of words, and represents the word as a low-dimensional, dense real-valued vector. Word vectors are often used as f...

AI Technical Summary

Problems solved by technology

[0010] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provides a method for generating word vectors based on Gaussian distribution, which uses Gaussian distribution to represent words, so as to overcome the shortcomings that traditional word vector models cannot represent probability distributions based on point estimation , while solving the problem that the prior art model assumes a fixed number of word senses

Images