Automatic text summarization method based on enhanced semantics

Abstract

The invention discloses an automatic text summarization method based on enhanced semantics. The method comprises the following steps of: preprocessing a text, arranging words from high to low according to the word frequency information, and converting the words to id; using a single-layer bi-directional LSTM to encode the input sequence and extracting text information features; using a single-layer unidirectional LSTM to decode the encoded text semantic vector to obtain the hidden layer state; calculating a context vector to extract the information, most useful the current output, from the input sequence; after decoding, obtaining the probability distribution of the size of a word list, and adopting a strategy to select summarization words; in the training phase, fusing the semantic similarity between the generated summarization and the source text to calculate the loss, so as to improve the semantic similarity between the summarization and the source text. The invention utilizes the LSTM depth learning model to characterize the text, integrates the semantic relation of the context, enhances the semantic relation between the summarization and the source text, and generates the summarization which is more suitable for the subject idea of the text, and has a wide application prospect.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to an automatic text summarization method based on enhanced semantics. Background technique [0002] With the rapid development of technology and the Internet, and the advent of the era of big data, the overwhelming amount of network information is increasing day by day. Among them, the explosive growth of representative text information, such as news, blogs, chats, reports, microblogs, etc., makes the information overload, and the huge information makes people spend a lot of time browsing and reading. Therefore, how to quickly extract key content from a large amount of text information and solve the problem of information overload has become an urgent need, and automatic text summarization technology has emerged as the times require. [0003] Automatic text summarization technology can be divided into extractive summarization and generative summarization accord...

AI Technical Summary

Problems solved by technology

Traditional extractive summarization usually causes a lot of information loss, especially in long texts. Therefore, in-depth research on generative automatic summarization is of great significance for truly solving information overload.

Images