Chinese word vector modeling method

Abstract

The invention discloses a Chinese word vector modeling method. A Chinese word vector modeling method of the present invention includes: using the BPE algorithm to adaptively combine Chinese strokes, constructing Chinese character sub-blocks and using the attention mechanism to complete the combination and representation of the internal structure of Chinese characters; Chinese character representation for information extraction uses Highway network for fine-grained information enhancement; considering the complexity of Chinese grammar, a bidirectional LSTM structure is constructed for semantic encoding in the semantic extraction stage, and a Chinese character adaptive combination layer, morphological information extraction layer, fine-grained An end-to-end deep neural language model with granular information enhancement layer and semantic information extraction layer as the basic components. Beneficial effects of the present invention: the present invention creatively constructs an input form different from n-gram, and performs adaptive fusion of Chinese strokes and radicals to form sub-blocks of Chinese characters.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to a Chinese word vector modeling method. Background technique [0002] Word embeddings have become an essential part of any deep learning-based natural language processing system. Natural language processing systems encode words and sentences in fixed-length dense vectors, resulting in vastly improved processing of text data by neural networks. In recent years, a large number of word embedding methods have been proposed. The most commonly used models are Word2VEC and GloVe, both of which are unsupervised methods based on distributional assumptions and are available in various languages. Considering the complexity of Chinese character morphology, more and more scholars have begun to study the modeling methods of Chinese word vectors. Scholars at the Hong Kong Polytechnic University first proposed to use Chinese radical information as components of CBOW and Skip-Gram to ...

AI Technical Summary

Problems solved by technology

The present invention mainly solves the problem of unregistered words from the perspective of word vector training, and disassembles the words into strokes that have been included in the user dictionary, thereby characterizing all Chinese characters

Images