Coding method for word vectorization by using dot mutual information between words

Abstract

The invention discloses a coding method for performing word vectorization by utilizing dot mutual information among words, which comprises the following steps of: S0, traversing corpora, and performing word segmentation on all texts to obtain a word set; the method comprises the following steps: S1, initializing word embedding parameters and counting the word frequency of each word; s2, traversing the corpus, and extracting positive and negative sample pairs for training word embedding parameters from the corpus; s3, calculating a loss function of the positive and negative sample pairs, solving a gradient of a corresponding word embedding parameter, and updating the gradient; and S4, after traversing the corpus for a plurality of times, storing a context word vector of each word, and embedding the context word vector as a solved word. According to the invention, the point-like mutual information between words can be fitted more accurately.

Description

technical field [0001] The invention relates to the field of artificial intelligence, in particular to a method for vectorizing and encoding words (including English words or Chinese words). Background technique [0002] Word quantization is a very critical step in natural language processing. The goal of word quantization is to convert symbolic information such as natural language into digital information in vector form, and use a continuous vector to represent each word to facilitate subsequent information processing. This process is also known as word encoding or word embedding. The current common word quantification methods are divided into two categories: one-hot representation model and distribution representation model. The former uses a sparse vector to represent a word. The length of the vector is the size N of the dictionary. Each vector has only one dimension which is 1, indicating the position of the word in the dictionary, and the other dimensions are all 0. ...

AI Technical Summary

Problems solved by technology

In order to obtain the word embedding of words, the traditional method represented by Word2Vec mainly uses the co-occurrence matrix that can describe the co-occurrence frequency between words to estimate the word vector, but there is a problem of poor interpretability of the coding model, and there is still room for improvement in performance.

Images