Unsupervised Chinese multi-document extraction type abstract method

Abstract

The unsupervised Chinese multi-document extraction type abstract method is realized through a method in the field of network security. The method is composed of three modules: a sentence vector acquisition module, an abstract sentence selection module and an abstract sentence sorting module. The sentence vector acquisition module is composed of six parts including text preprocessing, long text processing, an Embedding layer, an SPT task, an SRT task and multi-task learning, and sentence vector representation more suitable for an extraction type abstract task is obtained; the abstract sentence selection module generates a final abstract set; and the abstract sentence sorting module outputs a final abstract sequence. According to the method provided by the invention, automatic abstract extraction and generation based on a plurality of Chinese documents fused with deep learning and a rule statistics method are realized.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to an unsupervised Chinese multi-document extractive summarization method. Background technique [0002] The development and changes of Internet technology are changing with each passing day, and Internet applications have permeated all aspects of industry and social life. According to the 45th China Internet Development Report, as of March 2020, the number of Internet users in my country has reached 904 million, and the Internet penetration rate has reached 64.5%. Among them, the number of online news users has reached 731 million, accounting for 80.9% of the total Internet users. It can be seen that with the continuous development and updating of information technology, the traditional manual text is gradually being replaced by electronic text, and the Internet has become a part closely connected with people's life, and it is also an important channel for obtai...

AI Technical Summary

Problems solved by technology

[0011] (2) Selection of summary sentences

[0013] (3) The sorting problem of summary sentences

Among them, the traditional method based on statistical rules can obtain rules for judging the sequence of sentences in some specific scenarios through manual settings, but cannot dig out the rules of deep logical relationships between sentences.

The method based on deep learning can learn the deep-level logical relationship between sentences through training, but in specific scenarios, especially when the logical relationship between sentences is relatively vague, its effect is often not good.

Images