Method and device for training word vector embedding model

Abstract

The embodiment of this specification provides a method for training a word vector embedding model, which includes multiple iterative updates, any one of which includes: first determining the central word and the corresponding multiple context words from the word sequence corresponding to the training sentence; then , according to the first word vector matrix, determine the center word vector corresponding to the center word, and according to the second word vector matrix, determine the multiple context word vectors corresponding to the multiple context words; then, based on the multiple context word vectors The similarity between them determines the corresponding multiple attention weights; and then uses the multiple attention weights to carry out weighted summation on the multiple context word vectors to obtain the context representation vector of the central word; then, calculate the The first similarity between the central word vector and the context representation vector; finally, at least aiming at increasing the first similarity, update the first word vector matrix and the second word vector matrix.

Description

technical field [0001] The embodiments of this specification relate to the application of machine learning technology to the field of text processing, and in particular, relate to a method and device for training a word vector embedding model. Background technique [0002] Word vector technology solves the problem that computers are difficult to understand the semantics of human language by mapping words into real number vectors. For example, humans can easily judge that "cat" and "cat" are two words with very close semantics, but it is difficult for a computer to describe the semantic similarity between these two words. In this regard, the word vector algorithm can be used to generate a word vector for "cat" and "cat", and then by calculating the similarity between word vectors, the semantic similarity between "cat" and "cat" can be determined. Therefore, the accuracy of the word vector algorithm determines the semantic understanding ability of the computer. [0003] Howe...

AI Technical Summary

Problems solved by technology

[0003] However, the current word embedding algorithm is relatively simple, and it is difficult to meet various needs. For example, while quickly generating word embeddings for a large number of words, it is necessary to ensure that the determined embeddings have high accuracy.

Images