Semantic coding method of long-short-term memory network based on attention distraction

Abstract

The invention discloses a semantic coding method of a long short-term memory network based on attention distraction, and belongs to the field of natural language processing and generation. Aiming at the problems of semantic deviation, gradient disappearance, gradient explosion, incomplete contextual information fusion and the like in the prior art, a neural network used by the method adds a parameter sharing unit on the basis of BiLSTM, and the capability of obtaining and fusing bidirectional feature information of a model is enhanced; an activation function in an improved deep learning model is adopted, so that the probability of occurrence of a gradient problem is reduced; for an input layer and a hidden layer, a model is constructed in an interactive space and extended LSTM mode, so that the capability of fusing context information of the model is enhanced; an attention distraction mechanism of statement structure information variables is introduced, and semantic generation is limited, so that high semantic accuracy is improved. The method is suitable for natural language generation applications such as automatic news or title writing, robot customer service, conference or diagnosis report generation and the like.

Description

technical field [0001] The invention relates to a semantic coding method based on a distracted long-short-term memory network, which belongs to the field of natural language processing generation. Background technique [0002] Natural language generation is mainly used in the fields of human-computer dialogue, abstracts, picture and video descriptions, etc. It is the core technology for applications such as automatic news or headline writing, robot customer service, meeting or diagnostic report generation. The correctness of semantics is the key to generate language. [0003] Natural language generation mostly adopts the sequence conversion form of encoding to decoding, which converts a sequence of information into another corresponding sequence of text. The process of hierarchical encoding is divided into four steps: sentence semantic encoding, discourse information encoding, decoding, and sentence probability calculation. In the text generation step, sentence semantic en...

AI Technical Summary

Problems solved by technology

Although BiLSTM improves the problems of gradient explosion and gradient disappearance, the above two problems still exist when the sequence information is too long. At the same time, although BiLSTM can obtain the bidirectional feature information of the sequence, it only connects the bidirectional output without establishing a perfect Linking mechanism for fusing contextual information

Images