Word vector generation method based on Gaussian distribution

Abstract

The present invention discloses a word vector generation method based on the Gaussian distribution. The method comprises: firstly, preprocessing the corpus; secondly, using the punctuation to performtext division on the corpus; then combining the local and global information to infer the word meaning, and determining the mapping relationship between the word and the word meaning; and finally, obtaining a word vector by optimizing the objective function. The innovations and beneficial effects of the technical scheme of the present invention are as follows that: 1, words are represented based on the Gaussian distribution, point estimation characteristics of traditional word vectors are avoided, and more abundant information such as probabilistic quality, meaning connotation, an inclusion relationship, and the like can be brought to the word vectors; 2, multiple Gaussian distributions are used to represent the words, so that the linguistic characteristics of a word in the natural language can be coopered with; and 3, the similarity between the Gaussian distributions is defined based on the Hellinger distance, and by combining parameter updating and word meaning discrimination, the number of word meanings can be inferred adaptively, and the problem that the number of hypothetical word meanings of the model in the prior art is fixed is solved.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to a method for generating word vectors based on Gaussian distribution. Background technique [0002] Word vector is a method for mathematical modeling of words in Natural Language Processing (NLP). The earliest word vector representation is one-hot code (One-Hot), which represents each word as a high-dimensional vector, and the position of 1 in the vector is the index of the word in the dictionary. One-hot encoding has the disadvantages of high dimensionality, sparseness, and neglect of semantic and syntactic information. With the development of deep learning, there has been a method of using neural networks to train word vectors. The word vector model based on neural network captures the grammatical and semantic information of the context for the co-occurrence of words, and represents words as low-dimensional, dense real-valued vectors. Word vectors are often used as ...

AI Technical Summary

Problems solved by technology

[0010] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provides a method for generating word vectors based on Gaussian distribution, which uses Gaussian distribution to represent words, so as to overcome the shortcomings that traditional word vector models cannot represent probability distributions based on point estimation , while solving the problem that the prior art model assumes a fixed number of word senses

Images