Deep learning-based blog text abstract generation method

Abstract

The invention discloses a deep learning-based blog text abstract generation method. The method comprises the following steps of: crawling blog data; preprocessing the crawled blog data and selecting blog text data; converting the selected blog text data into vector matrix data according to a Chinese word vector dictionary; constructing a deep learning encoder-decoder model, separately training an encoder and a decoder of the model, and connecting the encoder and the decoder for use after the training is completed; and repeating the steps S01 to S03 to obtain generated data, and generating a predicted abstract from the generated data through the trained model. According to the method, text abstracts of blogs are automatically generated on the basis of a deep learning frame encoder-decoder, and deeper semantic relation of the blogs can be obtained at the same time. The generated text abstracts can visually display the main content of the current blog, so that the text abstracts have wide application prospect.

Description

technical field [0001] The present invention relates to a method for generating text abstracts, in particular to a method for generating blog text abstracts based on deep learning. Background technique [0002] Natural Language Processing (NLP) is a particularly important part of artificial intelligence. It includes multiple sub-tasks such as text classification, sentiment analysis, machine translation, and reading comprehension. Almost one sub-task is a very important professional research field. are independent and interrelated. [0003] Deep learning is a new type of end-to-end learning method proposed in recent years. In ordinary processing tasks such as classification, the effect of ordinary neural networks may be almost the same, but in the process of high-dimensional data calculation and feature extraction, depth Learning to use deep networks to fit shows its powerful computational capabilities. At present, deep learning has been applied to many fields-image process...

AI Technical Summary

Problems solved by technology

[0005] The generation of text summaries in natural language is mainly divided into two methods: the first extraction type, text summaries based on rules and statistics, has been proved by a large number of practical applications; the second is abstract type, based on deep learning model summarization generation, It has been greatly improved in 2014, from mechanical text summarization to comprehensible text summarization generation, currently using the encoder-decoder framework and embedding a recurrent neural network to achieve it, the application in Chinese is not obvious

Images