Natural language generation method based on time sequence topic model

Abstract

The invention discloses a natural language generation method based on a time sequence topic model. The natural language generation method comprises the steps of obtaining a context word bag vector ofeach sentence in a document; generating a topic distribution vector of each sentence in the document by utilizing a time sequence topic model; inputting each word of each sentence and the corresponding topic distribution vector into a sequential language model to obtain each layer of hidden variable corresponding to each word; splicing the hidden variables of each layer together, and predicting the next word in the current sentence through a normalized exponential function; utilizing a stochastic gradient descent method to update encoder parameters in the time sequence language model and the time sequence theme model; and sampling and updating decoder parameters in the time sequence topic model. According to the method, the multi-layer topic model and the multi-layer language model are combined, the hierarchical semantic features and the hierarchical time sequence information in the text topic are extracted, the semantic range of the low-layer features is small, and the semantic rangeof the high-layer features is wider.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to a natural language generation method based on a time series topic model. Background technique [0002] In the field of natural language processing, topic models and language models are widely used text analysis methods. The topic model analyzes the bag-of-words form of the text, only considering the number of words in the document and ignoring the temporal relationship between words in the text. The multi-layer topic model can greatly improve the ability to model text and obtain hidden variables with more semantic information. [0003] The language model performs temporal modeling on the text, which can capture the temporal relationship between words in the text, so as to realize various tasks in natural language processing, such as text summarization, machine translation, image annotation, etc. The language model usually gives the previous word,...

AI Technical Summary

Problems solved by technology

This method inputs topic semantic information into the question answering system, which makes up for the lack of exogenous knowledge in the question answering model and increases the richness and diversity of answers. However, the single-layer topic model is not as complete as the semantic information extracted by the multi-layer topic model, and it cannot capture sentences. Timing features between sentences

Images