A sentence sorting method based on depth learning self-attention mechanism

Abstract

The invention discloses a sentence sorting method based on depth learning self-attention mechanism. After a piece of text is input, firstly, each sentence in the text is translated into a distributedvector by using a long-short-term memory network to obtain syntactic information of the sentence. Secondly, the self-attention mechanism is used to learn the semantic association between sentences, and the potential logical structure is explored to preserve important information to form high-level paragraph vectors. The paragraph vector is then input into the pointer network to produce a new sentence order. The method of the invention is characterized in that the method is not influenced by the input sentence order, avoids the problem that the long-term and short-term memory network adds the wrong timing information in the process of generating the paragraph vector, and can effectively analyze the relationship between all sentences. Compared with the existing sentence sorting technology, the method provided by the invention has great improvement in accuracy, and has good practical value.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and relates to a sentence sorting method based on a deep learning self-attention mechanism. Background technique [0002] Sentence sorting task is to reorder some sentences whose order is unknown or disturbed by analyzing the semantic relationship and logical structure between sentences, so that they can form a smooth and coherent text, which can be applied in the field of natural language generation, such as text generation , multi-document extraction automatic summarization and retrieval-based question answering systems. Incorrect sentence order can lead to ambiguity in the content of the text, reduce readability, and bring confusion to readers. [0003] Existing sentence ranking research methods are mainly divided into three categories. The first type mainly relies on feature engineering, which is to artificially define some representative features to capture the semantic...

AI Technical Summary

Problems solved by technology

However, such methods are susceptible to the order of input sentences

Since the sentence order of the model input text is unknown or even disrupted, and in the process of constructing the paragraph vector, the recursive neural unit in the long short-term memory network cannot recognize the correct sentence order and can only read in the wrong order Sentence information, which leads to the confusion of logical and semantic information between sentences. The paragraph vector obtained by this method contains wrong text content and is unreliable, which makes it difficult for the pointer network to identify the correct sentence order.

Images