Two-stage semantic word vector generation method

Abstract

The invention provides a two-stage semantic word vector generation method. The two-stage semantic word vector generation method comprises the following five steps of: performing text matrixizing; constructing a feature extractor; performing semantic recognition; constructing a neural language model; and generating a semantic term vector. According to the method, the corresponding word vectors aregenerated for different semantics of the polysemy by using the plurality of neural networks, the defect that the polysemy only corresponds to one word vector in a traditional word-level embedded modeis overcome, and the size of the used corpus is within an acceptable range; meanwhile, a mode of combining a convolutional neural network (CNN) and a support vector machine (SVM) is adopted, on one hand, the feature extraction capability of the convolutional neural network is utilized, and on the other hand, the generalization and robustness of the SVM are utilized, so that the word meaning recognition effect is better, and the generated semantic word vector quality is higher.

Description

technical field [0001] The invention belongs to the field of neural networks, and in particular relates to a two-stage semantic word vector generation method. Background technique [0002] Word representation is one of the key issues in natural language processing. Whether the word representation method is appropriate directly affects the modeling methods of tasks such as syntactic analysis, semantic representation and text understanding, and also affects the accuracy and robustness of application systems such as information retrieval and question answering systems. [0003] At present, the representation strategies of Chinese words can be summarized into three types: traditional 0-1 representation, distributed representation based on latent semantic information and distributed representation based on neural network language model. The traditional 0-1 representation has two problems: on the one hand, the 0-1 representation causes data sparsity, which makes the word vectors ...

Images