A Word Embedding Representation Method Based on Internal Semantic Hierarchy

Abstract

The invention relates to a word embedding representation method based on an internal semantic hierarchical structure and relates to natural language processing based on deep learning. The internal hierarchical structure of each word in an input text is acquired by an open-source tool; the structure is similar to a conventional constituency tree structure, the difference is that a character is taken as a basic unit, and the hierarchical position and classification information of each character in the structure are labeled; the structure is subjected to serialization operation according to the principle that the hierarchical structure is invariant, and a text sequence keeping the hierarchical position and the classification information in a word is obtained; a bidirectional GRU network acts on the sequence for embedding representation of codes, then, two embedding representation vectors obtained by forward and backward GRU networks are spliced and finally subjected to non-linear transformation and ReLU operation, and a final embedding representation vector of the word is obtained. The frame structure is clear and concise, and the method is direct and favorable to learning of word embedding representation with richer meaning and better serves follow-up natural language processing tasks.

Description

technical field [0001] The present invention relates to natural language processing based on deep learning, in particular to a word embedding representation method based on internal semantic hierarchy. Background technique [0002] Natural language processing, as an intersection branch of computer science and linguistics, is a very popular subject right now. It mainly discusses various theories and methods of processing and using natural language to enable effective communication with computer systems. In recent years, the research on natural language processing based on deep learning has become the main trend in the development of this discipline. [0003] Word embedding refers to the use of distributed vectors to represent the semantic information of words. By mapping the words in natural language into low-dimensional and dense vectors, the words are in the same vector space, and the concept of "distance" is introduced to measure the semantic similarity between words, wh...

Images