Pre-training text abstract generation method based on neural topic memory

Abstract

The invention discloses a pre-training text abstract generation method based on neural topic memory, and the method comprises the steps: carrying out the coding of a sequence according to a complete input sequence through employing the context modeling capability of BERT pre-training, and completing the text embedding; encoding potential topic representations by using a topic memory network according to the output sequence representations; performing matching according to theme representation, reasoning and the coded sequence representation to form a final coded representation, and then usinga decoder for generating a preliminary output sequence; and masking each word in the output sequence and feeding into the BERT, and in combination with the input sequence, a decoder based on transformer and LSTM is used to predict the refined word of each mask position to realize fine adjustment. According to the method, the BERT layer and the LSTM layer of the bidirectional context are used for deeply capturing the features, the masked abstract is fed, the parameters are finely adjusted, and each abstract word is refined, so that the text abstract which is smoother and high in information amount is generated.

Description

technical field [0001] The invention belongs to the field of natural language processing, and in particular relates to a method for generating a pre-training text summary based on neural theme memory. Background technique [0002] Due to the rapid development of social media, the information on the Internet is increasing day by day. The huge amount of information makes information retrieval more and more difficult, and information summary plays an important role for information publishers, users and search engines. Concise text summaries can help users quickly find the information they need, and providing them to search engines can improve retrieval speed. For false title parties, text summarization can also play a very good supervisory role. Due to the huge workload of manual extraction of summaries, automatic summarization technology has been widely valued and researched. [0003] The text summarization process basically includes the text analysis process: analyze and pr...

AI Technical Summary

Problems solved by technology

The three main issues include: identification and processing of redundant information in documents; summary and identification of important content and key information; readability and coherence of generated abstracts

[0005] For summary summaries, existing technologies tend to understand documents one-way, and there is no way to accurately understand inverted sentences, clauses, etc.

Images