Word vector generating method and device based on incremental learning

Abstract

The embodiment of the invention provides a word vector generating method and device based on incremental learning. The word vector generating method based on incremental learning includes the steps that a word co-occurrence matrix of an original corpus, a word co-occurrence matrix of a newly added corpus and a training result parameter of the original corpus are acquired, wherein the training result parameter comprises a gradient value and a first matrix decomposition result; the training result parameter of the original corpus serves as an initial training parameter of the newly added corpus; a total objective function is iteratively optimized through a gradient descent algorithm through the initial training parameter of the newly added corpus, the word co-occurrence matrix of the original corpus and the word co-occurrence matrix of the newly added corpus, and a second matrix decomposition result is obtained, wherein the second matrix decomposition result is a solution enabling the total objective function to achieve minimization; multiple word vectors are obtained according to the second matrix decomposition result. The duration consumed in the word vector generating process can be effectively shortened.

Description

technical field [0001] Embodiments of the present invention relate to computer technology, and in particular to a method and device for generating word vectors based on incremental learning. Background technique [0002] The word vector is to use a vector to represent a word, so as to mathematicize the natural language symbols so that the computer can process the natural language. [0003] The GloVe algorithm is a new word vector generation method, which comprehensively uses the global statistical information and local statistical information of words to generate a language model and a vectorized representation of words. The GloVe algorithm combines the advantages of the traditional statistical-based word vector model and prediction-based word vector model. The training process is simpler and more efficient, and the generated word vectors can better reflect the linear relationship between words. [0004] However, the GloVe algorithm does not consider the case of incremental...

AI Technical Summary

Problems solved by technology

[0004] However, the GloVe algorithm does not consider the case of incremental learning. When the corpus changes incrementally, the global co-occurrence matrix changes.

The GloVe algorithm can only merge the original corpus and the incremental part of the corpus to obtain the merged corpus, and then retrain the entire corpus from the initial state, which will cause the process of generating word vectors to take a long time

Images