Hybrid text abstract generation method and system, terminal and storage medium

Abstract

The invention provides a hybrid text abstract generation method and system, and the method comprises the steps: carrying out the sentence segmentation and word segmentation of an input text, and recording the index of a sentence where each word is located; successively representing words and sentences for the clause and word segmentation results to obtain sentence vectors; copying each sentence vector, and respectively marking copied and rewritten vector marks on the two vectors; extracting important sentence vectors, and making a copy or rewrite decision according to vector marks; editing andmodifying sentences needing to be rewritten to obtain text abstracts; and training the adopted neural network to complete gradient updating of the parameters. Meanwhile, the invention provides a corresponding terminal and a storage medium. According to the method, the extracted sentences and the abstract sentences are mixed in the abstract for the first time, and the sentences directly used for the abstract and the rewritten sentences are distinguished through a copying or rewriting mechanism; according to the hierarchical reinforcement learning method training method, the extracted sentencesare used as tasks from managers to workers, and the collaboration between two networks is improved.

Description

technical field [0001] The present invention relates to the technical field of natural language processing, in particular to a hybrid text summarization method, system, terminal and storage medium based on layered reinforcement learning. Background technique [0002] The goal of a summary is to rewrite a long essay into a short, fluent version while keeping the most salient bits. With the successful application of neural networks in natural language processing (NLP) tasks, two data-driven branch extraction and abstract summarization stand out from various methods. The extraction method generally selects the most prominent sentence from the source article as the abstract, the content selection is accurate, and the result has a large amount of information, but because the sentence is not rewritten, the redundancy is high. On the contrary, abstract methods can generate more concise summaries by compressing and paraphrasing, but existing models are weak in content selection and...

AI Technical Summary

Problems solved by technology

However, since all sentences are indiscriminately compressed and truncated, they suffer from information loss during the abstraction stage

When the whole sentence is crucial, some important content will be deleted by mistake, causing serious information loss

Furthermore, the training methods for these techniques are not end-to-end due to the lack of an effective reinforcement learning framework to connect the two modules

[0003] At present, there is no description or report of the similar technology of the present invention, and no similar data at home and abroad have been collected yet.

Images