Method, system, terminal and storage medium for generating hybrid text abstract

Abstract

The invention provides a method and system for generating a hybrid text abstract. The input text is segmented and segmented, and the index of the sentence where each word is located is recorded; the results of the segmented and segmented words are successively characterized by words and sentences to obtain sentences. Vector; copy each sentence vector, and mark the two vectors with the vector labels of copy and rewrite respectively; extract important sentence vectors, and make decisions about copying or rewriting according to the vector labels; for sentences that need to be rewritten Edit and modify to obtain text summaries; train the adopted neural network to complete the gradient update of parameters. At the same time, the corresponding terminal and storage medium are provided. The present invention mixes the extracted sentences and abstract sentences in the abstract for the first time, and distinguishes the sentences directly used for the abstract from the rewritten sentences through the copying or rewriting mechanism; the training method of the hierarchical reinforcement learning method uses the extracted sentences as the manager To the tasks of the workers, the collaboration between the two networks is improved.

Description

technical field [0001] The present invention relates to the technical field of natural language processing, and in particular, to a method, system, terminal and storage medium for generating a hybrid text abstract based on hierarchical reinforcement learning. Background technique [0002] The goal of the abstract is to rewrite a long article into a short and fluid version, while keeping the most salient content. With the successful application of neural networks in natural language processing (NLP) tasks, two data-driven branch extraction and abstract summarization stand out from various methods. The extraction method generally selects the most prominent sentence from the source article as the abstract, the content selection is accurate, and the result is informative, but because the sentence is not rewritten, the redundancy is high. On the contrary, abstract methods can generate more concise summaries through compression and paraphrase, but existing models are weak in cont...

AI Technical Summary

Problems solved by technology

However, since all sentences are indiscriminately compressed and truncated, they suffer from information loss during the abstraction stage

When the whole sentence is crucial, some important content will be deleted by mistake, causing serious information loss

Furthermore, the training methods for these techniques are not end-to-end due to the lack of an effective reinforcement learning framework to connect the two modules

[0003] At present, there is no description or report of the similar technology of the present invention, and no similar data at home and abroad have been collected yet.

Images