A Natural Language Generation Method Based on Temporal Topic Model

Abstract

The invention discloses a natural language generation method based on a time series topic model, comprising: obtaining a context word bag vector of each sentence in a document; using a time series topic model to generate a topic distribution vector of each sentence in the document; Each word of each sentence and the corresponding topic distribution vector are input into the time-series language model, and the hidden variables of each layer corresponding to each word are obtained; the hidden variables of each layer are spliced ​​together, and the normalized exponential function is used to predict the current sentence. Next word; use stochastic gradient descent to update the encoder parameters in the temporal language model and the temporal topic model; sample and update the decoder parameters in the temporal topic model. This method combines a multi-layer topic model with a multi-layer language model to extract hierarchical semantic features and hierarchical timing information in text topics.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to a natural language generation method based on a time series topic model. Background technique [0002] In the field of natural language processing, topic models and language models are widely used text analysis methods. The topic model analyzes the bag-of-words form of the text, only considering the number of words in the document and ignoring the temporal relationship between words in the text. The multi-layer topic model can greatly improve the ability to model text and obtain hidden variables with more semantic information. [0003] The language model performs temporal modeling on the text, which can capture the temporal relationship between words in the text, so as to realize various tasks in natural language processing, such as text summarization, machine translation, image annotation, etc. The language model usually gives the previous word,...

AI Technical Summary

Problems solved by technology

This method inputs topic semantic information into the question answering system, which makes up for the lack of exogenous knowledge in the question answering model and increases the richness and diversity of answers. However, the single-layer topic model is not as complete as the semantic information extracted by the multi-layer topic model, and it cannot capture sentences. Timing features between sentences

Images