Multi-task chapter-level event extraction method based on multi-headed self-attention mechanism

Abstract

The invention provides a multi-task chapter-level event extraction method based on a multi-headed self-attention mechanism. The method comprises the following steps: converting single sentence-level event extraction into chapter-level event extraction of a packaged sentence set; carrying out the word embedding representation by utilizing a pre-trained language model BERT; taking all word embedding and position embedding in a single sentence as input, employing a convolutional neural network model for coding, and capturing the most valuable features in the sentence in combination with a segmented maximum pool strategy; utilizing a multi-head self-attention model to obtain chapter representation and attention weight fused with full-text semantic information; utilizing a classifier to obtain a predicted event type; taking event types as prior information, linking the event types to an input sequence for event element extraction, and extracting all related elements in the sequence by using a pre-training model in combination with a machine reading understanding method. The method can be used for text-level event extraction tasks, and the breakthrough of converting a sequence labeling problem into a machine reading understanding problem is achieved.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to a multi-task article-level event extraction method based on a multi-head self-attention mechanism. Background technique [0002] In today's era, data information is growing exponentially at a geometric level. Relying on the development of Internet technology, a large amount of data is generated every moment, news data is increasing rapidly, entertainment data is increasing rapidly, advertising data is rapidly increasing, and technological data is dramatically increasing. Rapid growth... Now, we have fully entered the era of big data. Such a large amount of data information is in various forms, intricate, difficult to mine and process, and difficult to use and analyze. In order to extract more valuable information from news data, the key is to extract the entities, relationships and events contained in the news text, analyze and predict the relationship betwee...

AI Technical Summary

Problems solved by technology

[0004] The present invention solves the problem that most of the existing event extraction technologies stay in the stage of single-sentence event extraction, cannot capture detailed features across clauses, and do not fully consider the relationship between contexts in chapters, and event extraction based on pre-trained models is only applicable to sentences and paragraphs It provides a multi-task chapter-level event extraction method based on the multi-head self-attention mechanism, which solves the problem that most of the existing sentence-level event extraction stays in the single event extraction of the document, ignoring the multiple trigger words in the sentence , ignoring the relationship between different sentences, ignoring the fact that event elements and arguments may exist in different sentences, achieving a breakthrough in transforming sequence labeling problems into machine reading comprehension problems

Images