An end-to-end chapter event extraction method and system based on deep learning
Through the end-to-end method of deep learning, the Attention mechanism and Transformer model are used to solve the problem of event elements dispersion and multi-event prediction in chapter event extraction, improve the extraction performance, realize efficient event extraction and multi-event prediction, and significantly improve the F1 value.
Patent Information
- Application Number
- CN202210125508.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-10
- Publication Date
- 2024-12-10
- Estimated Expiration
- 2042-02-10
AI Technical Summary
The prior art is difficult to effectively solve the problems of event elements dispersion and multi-event prediction in chapter event extraction, resulting in low performance, F1 value is less than 80%, and it cannot meet the commercial standards.
Using an end-to-end method based on deep learning, through candidate entity recognition, document-level encoding, candidate entity relationship encoding and event decoding, the Attention mechanism and Transformer model are used to generate event trees for decoding, solving the problems of event elements dispersed and multi-event prediction. .
The performance of chapter event extraction is improved, effective event factor extraction and multi-event prediction are achieved, and the final event extraction performance is improved, and the F1 value is significantly improved.
Smart Images
Figure CN114462386B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing in computer data processing technology, and in particular to an end-to-end chapter event extraction method and system based on deep learning. Background Art
[0002] With the continuous development of modern information technology, a large amount of information data has been accumulated in all walks of life, and the world has entered the era of big data. It is of great significance to obtain useful information from massive data in real time and accurately. Event extraction aims to extract structured events from unstructured text. It is widely used in intelligence work in the fields of business and military. Event extraction technology makes it possible to automatically extract structured event information from massive data. There are two specific challenges in extracting the events and their elements mentioned in plain text. One is the dispersion of event elements. The elements that constitute an event are scattered in multiple sentences. The other is multiple events. Multiple events may be mentioned in a document. A large number of previous studies focus on event extraction at the sentence level (event elements all exist in one sentence), and the two challenges of paragraph event extraction have not been solved. Shun Zheng et al. in the document Doc2EDAG: An End-to-End Document-level Framework for Chinese Financial Event Extraction, and Hang Yang et al. in the document Document-level Event Extraction via Parallel Prediction Networks. Proceedings of the 59 th Annual Meeting of the Association for Computational Linguisticsand the 11 th International Joint Conference on Natural Language Processing, both attempted to extract chapter events. The former used path extension to decode events, which made it impossible to extract events in parallel. The latter used a parallel multi-event decoder, which required the number of events to be extracted to be specified in advance, which could easily lead to problems such as event loss and waste of computing resources. At the same time, the performance of the chapter event extraction methods currently under research is low, with F1 values below 80%, which cannot reach commercial level. Summary of the invention
[0003] The present invention provides an end-to-end chapter event extraction method and system based on deep learning to address the deficiencies of the above-mentioned prior art. Through an end-to-end deep learning model, feature extraction and encoding are performed on the chapter semantics of plain text and the event elements mentioned therein. Event element relationship tables and event trees are used for decoding to address the problems of event element dispersion and multiple events in chapter event extraction, and to improve the performance of chapter event extraction, thereby having strong practicality.
[0004] In order to achieve the purpose of the present invention, the following technologies are proposed:
[0005] An end-to-end chapter event extraction method based on deep learning is provided, including:
[0006] S10, candidate entity recognition: obtaining document data, encoding each sentence in the document, and identifying word embedding tensor representations of multiple candidate entities;
[0007] S20, document-level encoding:
[0008] S21, since a candidate entity has multiple word embedding tensors, the Attention mechanism is used to merge multiple word embedding tensors of the same candidate entity into one embedding tensor to form the embedding tensor representation of the candidate entity. There are multiple word embedding tensors corresponding to multiple candidate entities.
[0009] S22, adding event role information to each obtained candidate entity embedding tensor, and forming multiple candidate entity tensor representations with event role information;
[0010] S23, sending all candidate entity representations with event role information and sentence representations with added position encoding information to the second Transformer model, and performing document-level feature extraction to form document-level perceived candidate entity representations and sentence representations;
[0011] S30, candidate entity relationship encoding: The improved Attention mechanism is used to encode the candidate entity representation tensor, and it is calculated whether any two candidate entities have a relationship, and a candidate entity relationship table is generated. The candidate entity relationship table is used to describe whether any two candidate entities appear in the same event. If they appear in the same event, it means that the two candidate entities are related, otherwise they are unrelated.
[0012] S40, event decoding: generating a tree with candidate entities as nodes according to the candidate entity relationship table, wherein each node on the path from the root node to each leaf node of the tree constitutes an event;
[0013] Decode each path from the root node to each leaf node to obtain the set of events predicted by the model;
[0014] The event set is deduplicated and predicted events are generated.
[0015] Furthermore, the steps of candidate entity identification in S10 are as follows:
[0016] At the sentence level, a first Transformer is used to identify candidate entities in the document sentence;
[0017] The first Transformer encodes the word embedding of each sentence in the document and obtains the contextual representation of each word embedding;
[0018] And each sentence tensor in the document is recorded as: Among them, d w is the size of word embedding, l is the length of the sentence, is a real number;
[0019] The context encoding of each word embedding is denoted as h i , and h i = Transformer-1(S i ), it is concluded that
[0020] During training, we use BIO to annotate candidate entities and use softmax to encode the context of each word embedding h i Classify and obtain each sentence tensor S i Candidate entities Among them, N a is the number of candidate entities identified, a i is the i-th candidate entity, and i is the candidate entity serial number subscript.
[0021] Furthermore, the steps of using the Attention mechanism in S21 to form multiple candidate entities are:
[0022] For a candidate entity a consisting of n word embedding tensors i , the candidate entity a i In the sentence tensor S i The subscript range in is from j to k;
[0023] Use the Attention mechanism to convert [h i,j ,…,h i,k ] is merged into candidate entity a i The embedding tensor of Among them, d w is the size of word embedding;
[0024] At the same time, the word embedding tensor h marked by the [cls] sentence header is used 0 As a sentence S i The embedding tensor of
[0025] And get the embedding tensor set of all candidate entities and a collection of sentence embedding tensors Among them, N a is the number of candidate entities identified, Embedding tensor for the i-th candidate entity, is the embedding tensor of the i-th sentence, where i is the embedding vector index.
[0026] Furthermore, the step of adding event role information to each obtained candidate entity in S22 is:
[0027] For a given event type T, it has N t Event roles, use the Embedding method to get the embedding tensor of each event role Then the event role embedding tensor set of event type T is in, is the embedding tensor of the i-th event role, i is the event role index, d w is the size of word embedding;
[0028] The embedding tensor set E of each candidate entity a The set of tensors E that embed each event role of event type T t Add together to get a set of candidate entity embedding tensors with event type T event role information in, is the candidate entity embedding tensor with event role information added, N a is the number of candidate entity embedding tensors, N t The number of event role embedding tensors, i is the index subscript of the candidate entity embedding tensor with event role information added.
[0029] Furthermore, the steps of forming document-level perceived candidate entity representations and sentence representations in S23 are:
[0030] The position encoding information is added to the sentence embedding tensor;
[0031] The second Transformer is used to encode the embedding tensors of all candidate entities and sentences, and obtain document-aware candidate entities and sentences [H at ,H s ]=Transformer-2([E at ,E s ]).
[0032] Furthermore, the candidate entity relationship encoding step in S30 is:
[0033] First, the document-aware candidate entities are embedded into the tensor H through the improved Attention mechanism. at Encoded as a candidate entity relationship table;
[0034] The improved Attention mechanism uses the candidate entity embedding tensor H at Use them as Query tensor Q and Key tensor K respectively, discard the Value tensor, and change the softmax function of the Attention mechanism to the sigmoid function;
[0035] The candidate entity relationship encoding expression is obtained as follows: Among them, Q tensor is the candidate entity query tensor, K tensor is the candidate entity keyword tensor, d k is the size of the Q and K tensors, and T represents the transpose operation of the K tensor.
[0036] Furthermore, the event decoding step in S40 is:
[0037] For candidate entity a i ;
[0038] Step 1: Use a i As the root node h of the tree;
[0039] Step 2: Traverse a i In the row and column of the candidate entity relationship table T, let the row number be the i-th row and the column number be the j-th column, then T i，j Represents a i with a j whether it is relevant;
[0040] Step 3: If the h node is j If the node h is related to a, then traverse the child nodes with the node h as the parent node; if the node h is related to a j If it is irrelevant, then do not j Insert into tree;
[0041] Step 4: Take the child node as the new h node. If the h node is the same as a j If the node h is related to a, then traverse the child nodes with the node h as the parent node; if the node h is related to a j If a j Insert into the tree as a child of h node;
[0042] Step5: Repeat Step4 until a j Insert into tree;
[0043] Step 6: Take a iAs the root node h of the tree, repeat Step 2 to Step 5 until the i-th row is traversed;
[0044] Among them, the trees generated by all candidate entities constitute a forest. All nodes on the path from the root node to a leaf node of each tree constitute an event. All trees in the forest are parsed into event lists, and events with the same elements are deduplicated, and finally all events of the document are obtained.
[0045] In addition, an end-to-end chapter event system based on deep learning is provided, including:
[0046] A candidate entity recognition module is used to obtain document data, encode each sentence in the document, and identify word embedding tensor representations of multiple candidate entities;
[0047] The document-level encoding module uses the Attention mechanism to merge multiple word embedding tensors of the same category into an embedding tensor of a candidate entity and form a candidate entity. There are multiple categories of word embedding tensors and multiple candidate entities corresponding to them.
[0048] and adding event role information to each obtained candidate entity, and forming multiple candidate entity representations with event role information;
[0049] All candidate entity representations with event role information and sentence representations with added position encoding information are sent to the second Transformer model, and document-level feature extraction is performed to form document-level perceived candidate entity representations and sentence representations;
[0050] The candidate entity relationship encoder uses the improved Attention mechanism to encode the candidate entity representation tensor, calculates whether any two candidate entities have a relationship, and generates a candidate entity relationship table. The candidate entity relationship table is used to describe whether any two candidate entities appear in the same event. If they appear in the same event, it means that the two candidate entities are related, otherwise they are unrelated.
[0051] An event decoder generates a tree with candidate entities as nodes according to the candidate entity relationship table, wherein each node on the path from the root node to each leaf node of the tree constitutes an event;
[0052] Decode each path from the root node to each leaf node to obtain the set of events predicted by the model;
[0053] The event set is deduplicated and predicted events are generated.
[0054] The advantages of the above technical solution are:
[0055] The end-to-end chapter event extraction technology proposed in the present invention can effectively solve the problem of scattered event elements in chapter documents, and can flexibly and parallelly perform multi-event prediction. It adopts the attention mechanism to embed candidate words, and uses the word embedding marked by the [CLS] sentence header as the sentence embedding, which better preserves the semantic representation of word embedding and sentence embedding, thereby improving the final event extraction performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.
[0057] Figure 1 The process of one embodiment is shown Figure 1 .
[0058] Figure 2 The process of one embodiment is shown Figure 2 . DETAILED DESCRIPTION
[0059] like Figure 1-2 As shown, an end-to-end chapter event extraction method based on deep learning is provided, including:
[0060] S10, candidate entity identification: Each element of an event is composed of entities in the document, such as time, place, name, action, etc. In this embodiment, the first Transformer model is used to encode each sentence of the document, identify candidate entities in the sentence range, and obtain word embedding tensor representations of multiple candidate entities.
[0061] S20, document-level encoding: S21, using the Attention mechanism to merge multiple word embedding tensors of the same category into an embedding tensor of a candidate entity, and form a candidate entity, and there are multiple categories of word embedding tensors corresponding to multiple candidate entities; S22, adding event role information to each obtained candidate entity, and forming multiple candidate entity representations with event role information; S23, sending all candidate entity representations with event role information and sentence representations with added position encoding information to the second Transformer model, and performing document-level feature extraction to form document-level perceived candidate entity representations and sentence representations;
[0062] Since the candidate entity representation is obtained based on sentence context encoding, it only has the semantic representation ability of the sentence range, and does not have the context representation ability between sentences. Therefore, this embodiment designs a document-level encoder to perform document-level encoding on the obtained candidate entity and sentence representation tensors to obtain document-aware candidate entity and sentence representation tensors. A candidate entity is often composed of multiple word embedding tensors, so the document-level encoder uses the Attention mechanism to merge multiple word embedding tensors into an embedding tensor representing the candidate entity. Then, an event role encoder is used to add event role information to the candidate entity representation, and multiple candidate entity representations with event role information are formed. That is, each candidate entity representation and each event role generate a candidate entity representation with event role information. Finally, all candidate entity representations with event role information and sentence representations with added position encoding information are sent to the second Transformer model for document-level feature extraction to form document-level perceived candidate entity representations and sentence representations.
[0063] S30, candidate entity relationship encoding: the improved Attention mechanism is used to encode the candidate entity representation tensor, and it is calculated whether any two candidate entities have a relationship, and a candidate entity relationship table is generated. The candidate entity relationship table is used to describe whether any two candidate entities appear in the same event. If they appear in the same event, it means that the two candidate entities are related, otherwise they are unrelated. After obtaining the document-aware candidate entity representation and sentence representation, this embodiment uses a candidate entity relationship encoder to encode the candidate entity representation into a candidate entity relationship table, so that the table can record the information of all events in the document, and through the corresponding event decoder, all events in the document can be decoded. The candidate entity relationship encoder uses an improved Attention mechanism to encode the candidate entity representation tensor, calculates whether any two entities have a relationship, and thus generates the table.
[0064] S40, event decoding: The event decoder uses a tree as a data structure to describe events, and generates a tree with candidate entities as nodes based on the candidate entity relationship table. Each node on the path from the root node to each leaf node in the tree constitutes an event. Each path from the root node to each leaf node is decoded to obtain a set of events predicted by the model. The event set is deduplicated and a predicted event is generated.
[0065] Specifically, the steps of candidate entity recognition in S10 are as follows:
[0066] For N s For a document with 10 sentences, the first Transformer is first used at the sentence level to identify candidate entities in the document sentences.
[0067] The first Transformer uses a multi-head self-attention mechanism to encode the word embedding of each sentence in the document and obtain the contextual representation of each word embedding. Among them, Transformer has shown excellent performance in many natural language processing tasks. Since Transformer is used multiple times in this embodiment, for the convenience of distinction, the Transformer module used for candidate entity recognition is named Transformer-1 in this step. The tensor of each sentence in the document is: Then the context encoding of each word embedding is recorded as h i , and h i = Transformer-1(S i ), it is concluded that Among them, d w is the size of word embedding, l is the length of the sentence, is a real number;
[0068] During training, we use BIO to annotate candidate entities and use softmax to encode the context of each word embedding h i Classify and obtain each sentence tensor S i Candidate entities Among them, N a is the number of candidate entities identified, a i is the i-th candidate entity, and i is the candidate entity serial number subscript.
[0069] Specifically, the steps of using the Attention mechanism to form multiple candidate entities in S21 are as follows: i After the candidate entity is obtained, in order to obtain a candidate entity with document awareness, this embodiment designs a document encoder to encode the candidate entity and the document sentence. A candidate entity is composed of multiple word embedding tensors. In this embodiment, the Attention mechanism is used to merge multiple word embedding tensors into a tensor representing the candidate entity. In form, for a candidate entity a composed of n word embedding tensors i , the candidate entity a i In the sentence tensor S i The subscript range in is recorded as from j to k, and the Attention mechanism is used to convert [h i,j ,…,h i,k ] is merged into candidate entity a i The embedding tensor of Among them, d w is the size of the word embedding. At the same time, the word embedding tensor h marked by the [cls] sentence header is used 0 As a sentence S i The embedding tensor of Then the embedding tensor set of all candidate entities will be obtained and a collection of sentence embedding tensors in, Embedding tensor for the i-th candidate entity, Embedding tensor for the i-th sentence, N a is the number of candidate entity embedding tensors, N s The number of sentence embedding tensors.
[0070] The step of adding event role information to each obtained candidate entity in S22 is:
[0071] Embedding tensor E of all candidate entities using event role encoder a Add event role information, that is, for a given event type T, it has N t Event roles, use the Embedding method to get the embedding tensor of each event role Then the event role embedding tensor set of event type T is The embedding tensor set E of each candidate entity a The set of tensors E that embed each event role of event type T t Add together to get a set of candidate entity embedding tensors with event type T event role information in, is the candidate entity embedding tensor with event role information added, N a is the number of candidate entity embedding tensors, and i is the serial subscript of the candidate entity embedding tensor with event role information added.
[0072] The steps of forming document-level perceived candidate entity representations and sentence representations in S23 are:
[0073] The position encoding information is added to the sentence embedding tensor;
[0074] The second Transformer (in order to distinguish it from the Transformer in step S10, the Transformer used in this step is named Transformer-2) is used to encode the embedding tensors of all candidate entities and the embedding tensors of sentences, and the document-aware candidate entities and sentences [H at ,H s ]=Transformer-2([E at ,E s ]).
[0075] Specifically, the candidate entity relationship encoding steps in S30 are:
[0076] In order to perform multi-event prediction in parallel, this embodiment uses a candidate entity relationship table to describe events. The table describes whether two candidate entities appear in the same event. If they appear in the same event, it means that the two candidate entities are related, otherwise they are unrelated. Since any two different events have at least one different element, the related events can be decoded from the candidate entity relationship table.
[0077] Specifically, the candidate entity relation encoder (using the improved Attention mechanism) needs to embed the document-aware candidate entity into the tensor H at Encoded as a candidate entity relationship table. The improved Attention mechanism uses the candidate entity embedding tensor H at They are used as the Query tensor Q and the Key tensor K respectively, and the Value tensor is discarded. The softmax function of the Attention mechanism is changed to the sigmoid function. The candidate entity relationship encoding expression is obtained as follows: Among them, the Q tensor is the candidate entity query tensor, the K tensor is the candidate entity keyword tensor, the Value tensor is the candidate entity content tensor, and the d k is the size of the Q and K tensors, and T represents the transpose operation of the K tensor. The improved Attention mechanism uses the candidate entity embedding tensor H at As the query tensor and key tensor respectively, the product of the query tensor and the transposed key tensor can obtain a tensor describing the candidate entity relationship. Since it is not multiplied with the value tensor again, a candidate entity relationship tensor can be formed. The softmax function cannot perform multi-classification calculations, and a candidate entity may have a relationship with multiple candidate entities. After using the sigmoid function, the improved attention mechanism can extract multiple relationship representations of candidate entities.
[0078] Specifically, since an event is usually composed of multiple event elements, and there is at least one event element different between multiple events, the event decoder filters out entities that are completely unrelated to other candidate entities based on the candidate entity relationship table, and generates a tree for each of the remaining candidate entities using the following method, wherein the event decoding step in S40 is:
[0079] For candidate entity a i ;
[0080] Step 1: Use a i As the root node h of the tree;
[0081] Step 2: Traverse a i In the row and column of the candidate entity relationship table T, let the row number be the i-th row and the column number be the j-th column, then T i，j Represents a i with aj whether it is relevant;
[0082] Step 3: If the h node is j If the node h is related to a, then traverse the child nodes with the node h as the parent node; if the node h is related to a j If it is irrelevant, then do not j Insert into tree;
[0083] Step 4: Take the child node as the new h node. If the h node is the same as a j If the node h is related to a, then traverse the child nodes with the node h as the parent node; if the node h is related to a j If a j Insert into the tree as a child of h node;
[0084] Step5: Repeat Step4 until a j Insert into tree;
[0085] Step 6: Take a i As the root node h of the tree, repeat Step 2 to Step 5 until the i-th row is traversed;
[0086] Among them, the trees generated by all candidate entities constitute a forest. All nodes on the path from the root node to a leaf node of each tree constitute an event. All trees in the forest are parsed into event lists, and events with the same elements are deduplicated, and finally all events of the document are obtained.
[0087] The end-to-end chapter event extraction technology proposed in this embodiment effectively solves the problem of scattered event elements in chapter documents. It encodes all event element relationships in the document through the candidate entity relation device, and encodes all event information into the candidate entity relationship table, so that multiple events can be predicted flexibly and in parallel. It adopts a multi-head self-attention mechanism for candidate word embedding, and uses the word embedding of the [CLS] sentence header marker as the sentence embedding, which better retains the semantic representation of word embedding and sentence embedding, so that the final event extraction performance can be improved. The following table compares the method given by Shun Zheng et al. in the document Doc2EDAG: An End-to-End Document-level Framework for Chinese Financial Event Extraction in 2019, and Hang Yang et al. in the document Document-level Event Extraction via Parallel Prediction Networks. Proceedings of the59 thAnnual Meeting of the Association for Computational Linguistics and the11 th The method given in the International Joint Conference on Natural Language Processing compares the accuracy, recall and F1 value of event extraction. Through the comparison, it is found that the final F1 value (the harmonic mean of precision and recall, an indicator used to measure the accuracy of the binary classification model) of the present invention is significantly higher than the former.
[0088]
[0089] Another aspect of the embodiment of the present application provides an end-to-end chapter event system based on deep learning, including:
[0090] A candidate entity recognition module is used to obtain document data, encode each sentence in the document, and identify word embedding tensor representations of multiple candidate entities;
[0091] The document-level encoding module uses the Attention mechanism to merge multiple word embedding tensors of the same category into an embedding tensor of a candidate entity and form a candidate entity. There are multiple categories of word embedding tensors and multiple candidate entities corresponding to them.
[0092] and adding event role information to each obtained candidate entity, and forming multiple candidate entity representations with event role information;
[0093] All candidate entity representations with event role information and sentence representations with added position encoding information are sent to the second Transformer model, and document-level feature extraction is performed to form document-level perceived candidate entity representations and sentence representations;
[0094] The candidate entity relationship encoder uses the improved Attention mechanism to encode the candidate entity representation tensor, calculates whether any two candidate entities have a relationship, and generates a candidate entity relationship table. The candidate entity relationship table is used to describe whether any two candidate entities appear in the same event. If they appear in the same event, it means that the two candidate entities are related, otherwise they are unrelated.
[0095] An event decoder generates a tree with candidate entities as nodes according to the candidate entity relationship table, wherein each node on the path from the root node to each leaf node of the tree constitutes an event;
[0096] Decode each path from the root node to each leaf node to obtain the set of events predicted by the model;
[0097] The event set is deduplicated and predicted events are generated.
[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. An end-to-end chapter event extraction method based on deep learning, characterized in that: include: S10, candidate entity recognition: obtaining document data, encoding each sentence in the document, and identifying word embedding tensor representations of multiple candidate entities; S20, document-level encoding: S21, uses the Attention mechanism to merge multiple word embedding tensors of the same category into an embedding tensor of a candidate entity, and forms a candidate entity. There are multiple categories of word embedding tensors and multiple candidate entities corresponding to them. The steps of using the Attention mechanism to form multiple candidate entities in S21 are: For a candidate entity a consisting of n word embedding tensors i , the candidate entity a i In the sentence tensor S i The subscript range in is from j to k, where a i is the i-th candidate entity, S i is the i-th sentence tensor; Use the Attention mechanism to convert [h i,j ,…,h i,k ] is merged into candidate entity a i The embedding tensor of Among them, d w is the size of word embedding, h i Encode the context of word embeddings, is a real number; At the same time, the word embedding tensor h0 marked by the [cls] sentence header is used as the sentence S i The embedding tensor of And get the embedding tensor set of all candidate entities and a collection of sentence embedding tensors Among them, N a is the number of candidate entities identified, Embedding tensor for the i-th candidate entity, is the embedding tensor of the i-th sentence, i is the embedding vector index, N s The number of sentence embedding tensors; S22, adding event role information to each obtained candidate entity embedding tensor, and forming multiple candidate entity tensor representations with event role information; The step of adding event role information to each obtained candidate entity in S22 is: For a given event type T, it has N t Event roles, use the Embedding method to get the embedding tensor of each event role Then the event role embedding tensor set of event type T is in, is the embedding tensor of the i-th event role, i is the event role index, N t The number of event role embedding tensors; The embedding tensor set E of each candidate entity a The set of tensors E that embed each event role of event type T t Add together to get a set of candidate entity embedding tensors with event type T event role information in, is the candidate entity embedding tensor with event role information added, N a is the number of candidate entity embedding tensors, i is the serial number subscript of the candidate entity embedding tensor with event role information added; S23, sending all candidate entity representations with event role information and sentence representations with added position encoding information to the second Transformer model, and performing document-level feature extraction to form document-level perceived candidate entity representations and sentence representations; S30, candidate entity relationship encoding: The improved Attention mechanism is used to encode the candidate entity representation tensor, and it is calculated whether any two candidate entities have a relationship, and a candidate entity relationship table is generated. The candidate entity relationship table is used to describe whether any two candidate entities appear in the same event. If they appear in the same event, it means that the two candidate entities are related, otherwise they are unrelated. S40, event decoding: generating a tree with candidate entities as nodes according to the candidate entity relationship table, wherein each node on the path from the root node to each leaf node of the tree constitutes an event; Decode each path from the root node to each leaf node to obtain the set of events predicted by the model; The event set is deduplicated and predicted events are generated.
2. The end-to-end chapter event extraction method based on deep learning according to claim 1 is characterized in that: The steps of candidate entity identification in S10 are as follows: At the sentence level, a first Transformer is used to identify candidate entities in the document sentence; The first Transformer encodes the word embedding of each sentence in the document and obtains the contextual representation of each word embedding; And each sentence tensor in the document is recorded as: Among them, d w is the size of word embedding, l is the length of the sentence, is a real number; The context encoding of each word embedding is denoted as h i , and h i = Transformer-1(S i ), that is, During training, we use BIO to annotate candidate entities and use softmax to encode the context of each word embedding h i Classify and obtain each sentence tensor S i Candidate entities Among them, N a is the number of candidate entities identified, a i is the i-th candidate entity, and i is the candidate entity serial number subscript.
3. The end-to-end chapter event extraction method based on deep learning according to claim 1 is characterized in that: The steps of forming document-level perceived candidate entity representations and sentence representations in S23 are: The position encoding information is added to the sentence embedding tensor; The second Transformer is used to encode the embedding tensors of all candidate entities and sentences, and obtain document-aware candidate entities and sentences [H at ,H s ]=Transformer-2([E at ,E s ]).
4. The end-to-end chapter event extraction method based on deep learning according to claim 1 is characterized in that: The candidate entity relationship encoding steps in S30 are: First, the document-aware candidate entities are embedded into the tensor H through the improved Attention mechanism. at Encoded as a candidate entity relationship table; The improved Attention mechanism uses the candidate entity embedding tensor H at Use them as Query tensor Q and Key tensor K respectively, discard the Value tensor, and change the softmax function of the Attention mechanism to the sigmoid function; The candidate entity relationship encoding expression is obtained as follows: Among them, Q tensor is the candidate entity query tensor, K tensor is the candidate entity keyword tensor, d k is the size of the Q and K tensors, and T represents the transpose operation of the K tensor.
5. The end-to-end chapter event extraction method based on deep learning according to claim 1 is characterized in that: The event decoding steps in S40 are: For candidate entity a i ; Step 1: Use a i As the root node h of the tree; Step 2: Traverse a i In the row and column of the candidate entity relationship table T, let the row number be the i-th row and the column number be the j-th column, then T i,j Represents a i with a j whether it is relevant; Step 3: If the h node is j If the node h is related to a, then traverse the child nodes with the node h as the parent node; if the node h is related to a j If it is irrelevant, then do not j Insert into tree; Step 4: Take the child node as the new h node. If the h node is the same as a j If the node h is related to a, then traverse the child nodes with the node h as the parent node; if the node h is related to a j If a j Insert into the tree as a child of h node; Step5: Repeat Step4 until a j Insert into tree; Step 6: Take a i As the root node h of the tree, repeat Step 2 to Step 5 until the i-th row is traversed; Among them, the trees generated by all candidate entities constitute a forest. All nodes on the path from the root node to a leaf node of each tree constitute an event. All trees in the forest are parsed into event lists, and events with the same elements are deduplicated, and finally all events of the document are obtained.
6. An end-to-end chapter event system based on deep learning, characterized in that: Used to implement the end-to-end chapter event extraction method based on deep learning as described in any one of claims 1 to 5; The system includes: A candidate entity recognition module is used to obtain document data, encode each sentence in the document, and identify word embedding tensor representations of multiple candidate entities; The document-level encoding module uses the Attention mechanism to merge multiple word embedding tensors of the same category into an embedding tensor of a candidate entity and form a candidate entity. There are multiple categories of word embedding tensors and multiple candidate entities corresponding to them. and adding event role information to each obtained candidate entity, and forming multiple candidate entity representations with event role information; All candidate entity representations with event role information and sentence representations with added position encoding information are sent to the second Transformer model, and document-level feature extraction is performed to form document-level perceived candidate entity representations and sentence representations; The candidate entity relationship encoder uses the improved Attention mechanism to encode the candidate entity representation tensor, calculates whether any two candidate entities have a relationship, and generates a candidate entity relationship table. The candidate entity relationship table is used to describe whether any two candidate entities appear in the same event. If they appear in the same event, it means that the two candidate entities are related, otherwise they are unrelated. An event decoder generates a tree with candidate entities as nodes according to the candidate entity relationship table, wherein each node on the path from the root node to each leaf node of the tree constitutes an event; Decode each path from the root node to each leaf node to obtain the set of events predicted by the model; The event set is deduplicated and predicted events are generated.