Document-level natural disaster event extraction method fusing attention adjacency matrix

Abstract

The invention discloses a document-level natural disaster event extraction method fusing an attention adjacency matrix, and belongs to the technical field of text data mining, and the method specifically comprises the following steps: 1, collecting natural disaster events to construct a text data set, and carrying out document-level label labeling on the natural disaster events; step 2, inputting the labeled document into a trigger word recognition layer for trigger word recognition; 3, inputting the context of the sentence where the trigger word is located into an entity recognition layer for event information detection; 4, obtaining a context label corresponding to the natural disaster event through the entity recognition layer, and generating an attention adjacency matrix; and step 5, obtaining arguments in the natural disaster event through the graph network layer and the event output layer, and completing filling of the natural disaster event. According to the method, event information extraction can be automatically completed from the unstructured corpus, accurate information of the natural disaster event is obtained, and structured display is carried out.

Description

Technical Field

[0001] The present invention belongs to the technical field of text data mining, and in particular relates to a document-level natural disaster event extraction method integrating an attention adjacency matrix. Background Art

[0002] In the face of the ever-increasing amount of news text in the field of natural disasters, more fine-grained event extraction is needed. Event detection and event monitoring have long been the focus of government public affairs management. Previous studies have lacked attention to information extraction in the field of natural disasters and subsequent disaster event information analysis. Extracting news text information in the field of natural disasters aims to extract the development information of related disaster events, where events express more complex semantics than entity relationships.

[0003] According to the definition in the International Conference on Automatic Content Extraction (ACE), event extraction is a process of structuring an unstructured sentence or a text. The purpose of event extraction is to detect event information and extract the arguments of the event from the text. According to statistics of manually annotated data sampled from Wikipedia, at least 40% of entity relationship facts must be obtained from multiple sentences. Due to the complexity of event composition and the diversity of language expression, in most cases, multiple sentences are needed to fully describe an event. However, existing work mainly focuses on extracting events at the single sentence level. It is easy to miss event information when only using single sentences and individual sentences for event information extraction, while document-level event extraction can extract more complete structured event information.

[0004] Document-level event extraction uses global information and generally considers the confidence of the current sentence, the context-related text of the text to be extracted, global information (such as the relationship between event information and event type), and co-occurrence information between events. At present, event extraction in the field of natural disasters faces many challenges, which are mainly reflected in the following three aspects: (1) There are many elements involved when a natural disaster occurs. For a natural disaster, it is necessary to determine the specific time and location of the disaster event, as well as the disaster-prone objects, disaster-prone object attributes and data involved in the natural disaster. (2) It is difficult to extract event information of natural disasters. Some natural disaster event information is scattered throughout the document, while others are mentioned multiple times in different sentences, and the relationship between their event information is complex and difficult to capture. (3) The source of the natural disaster corpus is relatively scarce. The corresponding natural disaster event extraction generally has high requirements for the data set in this field. The corpus text needs to have a detailed description of the natural disaster, and it needs to be authentic and objective. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a document-level natural disaster event extraction method that integrates the attention adjacency matrix, and proposes an upstream and downstream combined natural disaster event extraction method for document-level event extraction. The complete event information is identified and extracted from the text describing the event information, and then presented in a structured form. The general event extraction model is insufficient in the ability to extract the dependency relationship between entities, trigger words and arguments at the document level. A document-level natural disaster event extraction method that integrates the attention adjacency matrix is ​​proposed to calculate the document-level information using a knowledge-assisted attention mechanism. This method identifies the entity information in the document and cyclically classifies the roles of arguments for all events, which enhances the mutual connection between information in the event and improves the event information recognition performance.

[0006] The technical solution of the present invention is as follows:

[0007] A document-level natural disaster event extraction method integrating attention adjacency matrix includes the following steps:

[0008] Step 1: Collect natural disaster events to build a text dataset and annotate the natural disaster events at the document level;

[0009] Step 2: Input the annotated document into the trigger word recognition layer for trigger word recognition;

[0010] Step 3: Input the context of the sentence where the trigger word is located into the entity recognition layer for event information detection;

[0011] Step 4: Obtain the context labels corresponding to natural disaster events through the entity recognition layer and generate an attention adjacency matrix;

[0012] Step 5: Obtain arguments in natural disaster events through the graph network layer and event output layer to complete the filling of natural disaster events.

[0013] Furthermore, in step 2, the annotated document is d, and the i-th sentence s in document d is i As the input of the trigger word recognition layer, the process of trigger word recognition by the trigger word recognition layer is:

[0014] Step 2.1: First, use the pre-trained model Transformer-XL to average pool the input sentence to obtain the hidden state sequence h u , the calculation formula is expressed as:

[0015] h u =mean(Transformer_XL(s i )) (1);

[0016] Where, mean(·) is the average pooling calculation; Transformer_XL(·) is the pre-trained model Transformer-XL;

[0017] Step 2.2: Classify and judge the sentences to determine whether the input sentences contain one or more natural disaster trigger words, and classify each output sentence to obtain the predicted logical value p of different natural disaster classifications. u , the calculation formula is expressed as:

[0018] p u =o(W u h u +b u ) (2);

[0019] Where σ(·) is the sigmoid function; W u and b u They are the weight matrix and bias values ​​of the pre-trained model Transformer-XL respectively;

[0020] Step 2.3: Use the first cross entropy loss function L u For training, the calculation formula is expressed as:

[0021]

[0022]

[0023] Where N is the total number of samples; C is the total number of event types; y u is the event type label; is the predicted logical value for predicting the c-th event type; When the sample belongs to the cth event type, y u =1, otherwise y u =0; For sentence s i The predicted label of the c-th event type; For sentence s i The predicted logical value of the c-th event type; β is the threshold.

[0024] Furthermore, the specific process of step 3 is as follows:

[0025] Step 3.1: Document context c Input the pre-trained model ALBERT to get the entity hidden state sequence h in the sentence e , where the document context s c Contains the sentence where the trigger word is located t , the next k sentences and the title sentence of the sentence where the trigger word is located h , which is s c={s h ;s t-k ,s t-k+1 ,…,s t ,s t+1 ,…,s t+k},s t-k The sentence where the trigger word is located t The last k sentence, s t+k The sentence where the trigger word is located t The calculation formula for the next k sentences is:

[0026] h e =AlBert(s i ) (5);

[0027] Among them, AlBert(·) is calculated by the pre-trained model ALBERT;

[0028] Step 3.2: Classify and get the logical value of the word The calculation formula is expressed as:

[0029]

[0030] Among them, softmax(·) is the softmax activation function; For word w i Hidden state; W e and b e They are the weight matrix and bias value of the pre-trained model ALBERT respectively;

[0031] Step 3.3: Use the second cross entropy loss function L e For training, the calculation formula is expressed as:

[0032]

[0033]

[0034] in, is the total number of words; is the word w in event type c i Labels; To predict word w i The logical value of For word w i The predicted label of ; argmax(·) is to find the maximum value.

[0035] Furthermore, the specific process of step 4 is as follows:

[0036] Step 4.1: Consider the relationships between words, arguments, trigger words, and arguments in all natural disaster event types, and construct the attention adjacency matrix M. k , the attention adjacency matrix M k The calculation formula for the value of each position in is expressed as:

[0037]

[0038]

[0039] in, Indicates M k exist The value of the position; set(h) is the set of title sentences; set(t) is the node set of trigger words; set(a) is the node set of arguments; η and θ are different hyperparameters; g(·) is the nonlinear function log2(x+1), x is the function input parameter; g θ (·) is the nonlinear function g(θx+1); mdis is the average distance between two points;

[0040] Step 4.2: Use the entity h in the event obtained by the pre-trained model ALBERT e Perform weighted sum to obtain text information The calculation formula is expressed as:

[0041]

[0042] Among them, δ is a hyperparameter;

[0043] Step 4.3: Based on the generated attention adjacency matrix M k , the text information Input into the graph attention network KGAT for fusion calculation, the calculation formula is expressed as:

[0044]

[0045]

[0046]

[0047] in, For Node With neighbor nodes The similarity coefficient between For Node With neighbor nodes The attention weight coefficient between ; is the corrected attention weight coefficient; and Is a node and neighbor nodes The eigenvector of z is the weight matrix for the node and neighbor nodes The feature vector of is dimensionally increased; || is vector concatenation; f(·) is a single-layer feedforward neural network; LeakyReLU(·) is the LeakyRelu activation function; For Node The set of all adjacent nodes of ; For other neighbor nodes; For Node With other neighboring nodes The similarity coefficient between

[0048] Step 4.4: Obtain the new feature vector of each node by weighted summing of neighboring node features The calculation formula is expressed as:

[0049]

[0050] Among them, W t is the connection type mapping matrix; m is the serial number of the multi-head attention, M is the number of multi-head attention, || is the vector concatenation; Tanh(·) is the hyperbolic tangent function.

[0051] Furthermore, the specific process of step 5 is as follows:

[0052] Step 5.1: The graph network layer uses the knowledge graph attention network KGAT to perform cyclic calculations at the context location of the trigger word entity to determine whether the argument entity belongs to the current natural disaster event type;

[0053] Step 5.2: For each argument, select the hidden state sequence corresponding to the entity position in the output of the knowledge graph attention network The hidden state feature vector of the entity corresponding to each argument is obtained by average pooling. The calculation formula is expressed as:

[0054]

[0055] in, For the context The hidden state feature vector of the entity corresponding to each argument;

[0056] Step 5.3: Get the logical value of the corresponding position through classification. The calculation formula is expressed as:

[0057]

[0058] in, For the context The logical value of the corresponding position of the argument; W g and b g They are the weight matrix and bias value of the knowledge graph attention network respectively;

[0059] Step 5.4: Use the third cross entropy loss function L g For training, the calculation formula is expressed as:

[0060]

[0061] in, is the corresponding argument type; is the number of types of arguments; For the context Total number of class arguments; is the label of the argument; when the event type c selects the position belonging to Class Theory Meta-Time otherwise

[0062] When reasoning, take the maximum logical value corresponding to the argument entity Compared with the threshold ε, when When , take the entity of the corresponding position segment span as the argument in the current natural disaster event; when When the corresponding natural disaster event argument takes a null value, the calculation formula is expressed as:

[0063]

[0064] Finally, the natural disaster events are filled in according to the argument results.

[0065] Beneficial technical effects brought by the present invention:

[0066] The present invention proposes a document-level natural disaster event extraction method that integrates the attention adjacency matrix. Aiming at the problem that event information is not easy to capture due to the dispersion of event information in the text of natural disasters and the presence of multiple events in a sentence, a document-level extraction model combining upstream and downstream is proposed. The method of the present invention is based on the Transformer-XL and ALBERT pre-training models, and designs a knowledge-assisted attention mechanism KGAT loop to extract event information. The method enhances the mutual connection between information in document-level events, automatically completes event information extraction from unstructured corpus, obtains accurate information on natural disaster events and displays it in a structured manner, effectively improving the performance of natural disaster event information recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a flowchart of document-level natural disaster event extraction integrating attention adjacency matrix of the present invention.

[0068] Figure 2 This is the overall architecture diagram of the document-level natural disaster event extraction model that integrates the attention adjacency matrix of the present invention.

[0069] Figure 3 This is a specific value diagram of the attention adjacency matrix in the document-level natural disaster event extraction model that integrates the attention adjacency matrix of the present invention. DETAILED DESCRIPTION

[0070] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0071] like Figure 1 As shown, the present invention proposes a document-level natural disaster event extraction method integrating attention adjacency matrix, constructs a document-level natural disaster event extraction model integrating attention adjacency matrix, and uses the model to extract natural disaster events. The document-level natural disaster event extraction model mainly includes a trigger word recognition layer, an entity recognition layer, a graph network layer, and an event output layer; the method mainly includes the following steps:

[0072] Step 1: Collect natural disaster events to build a text dataset, annotate the natural disaster events at the document level, and divide them into training and test sets.

[0073] The logic of labeling is as follows: (1) The predefined key natural disaster argument must exist, where the disaster-affected entity (trigger word) and the disaster-affected entity attribute must exist simultaneously in a natural disaster; (2) The non-key natural disaster argument can be empty (NA) in a natural disaster event type, where the non-key natural disaster argument is the data argument, the time argument, and the location argument. The labeled document is denoted as document d.

[0074] Adopting upstream and downstream event extraction: In order to solve the challenges of document-level natural disaster extraction, the four tasks of traditional event extraction (trigger word recognition, event type classification, argument recognition and role classification) are reintegrated into two upstream and downstream tasks, namely, trigger word recognition and event classification tasks, and argument recognition and role classification tasks.

[0075] Step 2: Input the annotated document into the trigger word recognition layer for trigger word recognition. The trigger word recognition layer uses the pre-trained deep language model Transformer-XL to determine whether the sentence in the document contains one or more trigger words of natural disaster events, and classifies the corresponding sentences into six event types or no type.

[0076] The input of the document-level natural disaster event extraction model follows the fixed format of the pre-trained model, and the structure of the upstream task model input is s i ={[SEP] / [CLS],w1,w2,…,w n ,[SEP]},si represents the i-th sentence in document d, n is the length of the sentence, the i-th sentence contains n words, w n The nth word; the beginning of a paragraph is marked with [CLS], and sentences are separated by [SEP] tags, and the default relative position encoding of Transformer-XL is used. Figure 2 As shown, the document contains 6 sentences from s1 to s6, and the structure of sentence s3 is: s3 = {[SEP], w 1,1 ,w 1,2 ,w 1,3 ,[SEP],w 2,1 ,w 2,2 ,[SEP]}, where w 2,2 is the trigger word.

[0077] Step 2.1: Identify the trigger words. First, use the pre-trained model Transformer-XL to average pool the input sentence vector to obtain the hidden state sequence h. u , the calculation formula is expressed as:

[0078] h u =mean(Transformer_XL(s i )) (1);

[0079] Among them, mean(·) is the average pooling calculation; Transformer_XL(·) is the pre-trained model Transformer-XL.

[0080] Step 2.2: Classify and judge the sentences to determine whether the input sentences contain one or more natural disaster trigger words, and classify each output sentence to obtain the predicted logical value p of different natural disaster classifications. u , the calculation formula is expressed as:

[0081] p u =σ(W u h u +b u ) (2);

[0082] Where σ(·) is the sigmoid function; W u and b u They are the weight matrix and bias values ​​of the pre-trained model Transformer-XL respectively.

[0083] Step 2.3: Use the first cross entropy loss function L u Perform training and judge s during reasoning i The event type in When the cth label of sentence i is 1, the calculation formula is expressed as:

[0084]

[0085]

[0086] Where N is the total number of samples; C is the total number of event types; y u is the event type label; is the predicted logical value for predicting the c-th event type; When the sample belongs to the cth event type, y u =1, otherwise y u =0; For sentence s i The predicted label of the c-th event type; For sentence s i The predicted logical value of the c-th event type; β is the threshold.

[0087] Step 3: Input the context of the sentence where the trigger word is located into the entity recognition layer for event information detection. Use the pre-trained model ALBERT to identify the entities in the document and obtain the entity information in the event, namely the trigger word and the argument. The downstream task performs a cyclic calculation around the context of the sentence where the trigger word is located. A separate model instance is used for training for each type of natural disaster event to obtain the classification of each entity in the context.

[0088] Step 3.1: Document context c Input the pre-trained model ALBERT to obtain the entity hidden state sequence h in the event e , where the document context s c Contains the sentence where the trigger word is located t , the next k sentences and the title sentence of the sentence where the trigger word is located h , which is s c ={s h ;s t-k ,s t-k+1 ,…,s t ,s t+1 ,…,s t+k},s t-k The sentence where the trigger word is located t The last k sentence, s t+k The sentence where the trigger word is located t The calculation formula for the next k sentences is:

[0089] h e =AlBert(s i ) (5);

[0090] Among them, AlBert(·) is calculated by the pre-trained model ALBERT.

[0091] Step 3.2: Classify and get the logical value of the word The calculation formula is expressed as:

[0092]

[0093] Among them, softmax(·) is the softmax activation function; For word w i Hidden state; W e and b e They are the weight matrix and bias values ​​of the pre-trained model ALBERT respectively.

[0094] Step 3.3: Use the second cross entropy loss function L e For training, the calculation formula is expressed as:

[0095]

[0096]

[0097] in, is the total number of words; is the word w in the cth event type i Labels; To predict word w i The logical value of For word w i The predicted label of ; argmax(·) is to find the maximum value.

[0098] Step 4: Obtain the context labels corresponding to natural disaster events through the entity recognition layer and generate an attention adjacency matrix. Obtain the corresponding context labels through the entity recognition layer to generate an attention adjacency matrix. Construct a knowledge-assisted adjacency matrix to assist learning to help reduce attention overflow, and design node interaction position information and node connection type information in the attention adjacency matrix. In addition, the nodes between sentences are fused in the attention adjacency matrix to strengthen the global structured information, thereby enriching the connection information between nodes.

[0099] Step 4.1, attention adjacency matrix M k Considering the relationship between words in all natural disaster event types, the relationship between all arguments, between trigger words, and between arguments and trigger words are taken into account; at the same time, the node information of the sentence is adopted to consider the global information of the text; Indicates M k exist The value at the position is calculated as follows:

[0100]

[0101]

[0102] Where, set(h) is the set of title sentences; set(t) is the node set of trigger words; set(a) is the node set of arguments; η and θ are hyperparameters; g(·) is the nonlinear function log2(x+1), x is the function input parameter; g θ (·) is the nonlinear function g(θx+1); mdis is the average distance between two points, and its value is is the average distance between two points in the 2k+2 sentences, k is the threshold, and γ is the Euler-Mascheroni constant, which is approximately equal to 0.577.

[0103] Specifically: If Or the node at position j belongs to set(h), then if The position node belongs to the node set set(t) or set(a), and the j-position node belongs to [SEP] or [CLS], then if and j's position nodes belong to set(t) or set(a) at the same time, then if The position nodes of and j do not belong to set(t) or set(a), or and j where one of the position nodes is not in set(a) and does not belong to [SEP] or [CLS].

[0104] Step 4.2: Use the entity h in the event obtained by the pre-trained model ALBERT e Perform weighted sum to get text information The calculation formula is expressed as:

[0105]

[0106] Among them, δ is a hyperparameter.

[0107] Step 4.3: Based on the generated attention adjacency matrix M k , the text information Input into the graph attention network KGAT for fusion calculation. By calculating the attention weight coefficients between nodes, the document-level global structural information after the attention weighting in the neighborhood of the fusion node is obtained. First, calculate the attention weight coefficients between nodes That is, for the node Calculate its neighbor nodes one by one Similarity coefficient with oneself Then normalize the similarity coefficient between nodes to get the attention weight coefficient between nodes Through the attention adjacency matrix M output by the upper layer k The corrected attention weight coefficient is obtained by multiplying the values ​​at each position in The calculation formula is expressed as:

[0108]

[0109]

[0110]

[0111] in, and Is a node and nodes The eigenvector of z is the weight matrix for the node and neighbor nodes The feature vector is increased in dimension; || is vector concatenation; f(·) is a single-layer feedforward neural network that maps the concatenated high-dimensional features to a real output. LeakyReLU(·) is the LeakyRelu activation function; For Node The set of all adjacent nodes of ; For other neighbor nodes; For Node With other neighboring nodes The similarity coefficient between .

[0112] Step 4.4: Obtain the new feature vector of each node by weighted summing of neighboring node features The calculation formula is expressed as:

[0113]

[0114] Among them, W t is the connection type mapping matrix, and 4 connection type mapping matrices W are used t , t∈{1,2,3,4} represents learning of four different connection types, namely, between the same entity, between different entities, between entities and sentences, between nodes (including arguments, trigger words, sentence nodes), and between other positions; m is the sequence number of multi-head attention, M is the number of multi-head attention, || is vector concatenation; Tanh(·) is the hyperbolic tangent function.

[0115] Step 5: Obtain arguments in natural disaster events through the graph network layer and event output layer to complete the filling of natural disaster events. The graph network layer uses the knowledge graph attention network KGAT. The graph network layer loops the entire document and uses the context centered on the sentence where each trigger word is located as the input of the graph network layer to determine whether the argument entity belongs to the current natural disaster event type. According to the natural disaster event corresponding to the trigger word, the corresponding argument is found through the event output layer to fill the natural disaster event quintuple.

[0116] Step 5.1: The graph network layer uses the knowledge graph attention network KGAT to perform cyclic calculations at the context location of the trigger word entity to determine whether the argument entity belongs to the current natural disaster event type;

[0117] Step 5.2: For each argument, select the hidden state sequence corresponding to the entity position in the output of the knowledge graph attention network The hidden state feature vector h of each entity corresponding to each argument is obtained by average pooling g , the calculation formula is expressed as:

[0118]

[0119] in, For the context The hidden state feature vector of the entity corresponding to each argument;

[0120] Step 5.3: Get the logical value p of the corresponding position through classification g , the calculation formula is expressed as:

[0121]

[0122] in, For the context The logical value of the corresponding position of the argument; and are the weight matrix and bias value of the knowledge graph attention network, n r is the dimension of the entity hidden layer.

[0123] Step 5.4: Use the third cross entropy loss function L g For training, the calculation formula is expressed as:

[0124]

[0125] in, is the corresponding argument type; is the number of types of arguments; For the context Total number of class arguments; is the label of the argument; when the event type c selects the position belonging to Class Theory Meta-Time otherwise

[0126] When reasoning, take the maximum logical value corresponding to the argument entity When When , the entity of the corresponding position segment span is taken as the argument in the current natural disaster event, and ε is the threshold; when When the corresponding natural disaster event argument takes a null value, the calculation formula is expressed as:

[0127]

[0128] Finally, fill in the natural disaster events based on the argument results. Figure 3 As shown, the node at the bottom of the figure is the value of a certain position node in the attention adjacency matrix, and the top of the figure includes three entity fragments w 1,1 ,w 1,2 ,w 1,3 、w 2,1 ,w 2,2 ,w 2,3 ,w 2,4 、w 3,1 ,w 3,2 , when the maximum logical value corresponding to the argument entity is greater than the threshold ε, the entity of the corresponding position fragment span is taken as the argument in the current natural disaster event.

[0129] Table 1 is a pseudo code table for extracting five-tuples of natural disaster events from Algorithm 1. h 、e p 、e d 、e t and e l The disaster-prone entities, disaster-prone entity attributes, disaster-prone entity loss data, time and location entities are marked in the sentence after model training. After determining the event type corresponding to the trigger word, the determined natural disaster event quintuple ME = {e h ,e p ,e d ,e t ,e l}. First, sentence s is identified through the trigger word recognition layer. i Classify and extract sentence sets with different types of trigger words (T i )(rows 1-3); then get the context s consisting of the sentence where the trigger word is located, the next k sentences of the sentence where the trigger word is located, and the title sentence c .(t i ), and obtain s through the entity recognition layer c .(t i )’s entity tag L M and hidden layer H M(Lines 4-7); Finally, we get the trigger word context s with entity tags c .(e i ,L M ) and the attention adjacency matrix A, and obtain the trigger word Me in the natural disaster event through the graph network layer and the event output layer h And the theory of Me p 、Me d 、Me t and Me l Complete the filling of the quintuple M (lines 8-14). h and e p is the parameter of the key event argument, e d 、e t and e l It is a non-key event argument parameter and can be an empty parameter NA.

[0130] Table 1 Pseudo code table for extracting quintuples of natural disaster events

[0131]

[0132]

[0133] In order to prove the feasibility and superiority of the model trained by the method of the present invention, the following comparative experiments are given on two data sets, SSD-HDS and SSD-NDS. The existing technologies are: the DCFEE model extracts information from the central sentence of the recognition, and queries the missing event information from around the sentence; the Doc2EDAG model uses Transformer to obtain sentence and entity embeddings, uses another Transformer model to fuse cross-sentence contexts, and then extracts multiple events at the same time; the Greedy-Dec model is a variant of Doc2EDAG, which only generates one record information each time; the GIT model uses a heterogeneous graph-based tracking interaction model to model the global interaction between sentences and event information, and uses Tracker to track and extract information from events. The method of the present invention was compared with the above methods. The results of the comparative experiment are shown in Table 2:

[0134] Table 2 Performance comparison of each model

[0135]

[0136] The precision, recall and F1 score are used to evaluate the model. The larger the precision, recall and F1 score, the stronger the performance of the model in the document-level natural disaster event extraction task. It can be seen from the experimental results that the method of the present invention has achieved the best prediction results, and the precision, recall and F1 score are all maximum values.

[0137] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A document-level natural disaster event extraction method integrating attention adjacency matrix, characterized in that: The steps include: Step 1: Collect natural disaster events to build a text dataset and annotate the natural disaster events at the document level; Step 2: Input the annotated document into the trigger word recognition layer for trigger word recognition; Step 3: Input the context of the sentence where the trigger word is located into the entity recognition layer for event information detection; Step 4: Obtain the context labels corresponding to natural disaster events through the entity recognition layer and generate an attention adjacency matrix; Step 5: Obtain arguments in natural disaster events through the graph network layer and event output layer to complete the filling of natural disaster events.

2. According to the document-level natural disaster event extraction method integrating attention adjacency matrix according to claim 1, it is characterized in that: In step 2, the annotated document is d, and the i-th sentence s in document d is i As the input of the trigger word recognition layer, the process of trigger word recognition by the trigger word recognition layer is: Step 2.1: First, use the pre-trained model Transformer-XL to average pool the input sentence to obtain the hidden state sequence h u , the calculation formula is expressed as: h u =mean(Transformer_XL(s i )) (1); Where, mean(·) is the average pooling calculation; Transformer_XL(·) is the pre-trained model Transformer-XL; Step 2.2: Classify and judge the sentences to determine whether the input sentences contain one or more natural disaster trigger words, and classify each output sentence to obtain the predicted logical value p of different natural disaster classifications. u , the calculation formula is expressed as: p u =σ(W u h u +b u ) (2); Where σ(·) is the sigmoid function; W u and b u They are the weight matrix and bias values ​​of the pre-trained model Transformer-XL respectively; Step 2.3: Use the first cross entropy loss function L u For training, the calculation formula is expressed as: Where N is the total number of samples; C is the total number of event types; y u is the event type label; is the predicted logical value for predicting the c-th event type; When the sample belongs to the cth event type, y u =1, otherwise y u =0; For sentence s i The predicted label of the c-th event type; For sentence s i The predicted logical value of the vth event type; β is the threshold.

3. According to the document-level natural disaster event extraction method integrating attention adjacency matrix according to claim 2, it is characterized in that: The specific process of step 3 is as follows: Step 3.1: Document context c Input the pre-trained model ALBERT to get the entity hidden state sequence h in the sentence e , where the document context s c Contains the sentence where the trigger word is located t , the next k sentences and the title sentence of the sentence where the trigger word is located g , which is s c ={s h ;s t-k ,s t-k+1 ,…,s t ,s t+1 ,…,s t+k },s t-k The sentence where the trigger word is located t The last k sentence, s t+k The sentence where the trigger word is located t The calculation formula for the next k sentences is: h e =AlBert(s i ) (5); Among them, AlBert(·) is calculated by the pre-trained model ALBERT; Step 3.2: Classify and get the logical value of the word The calculation formula is expressed as: Among them, softmax(·) is the softmax activation function; For word w i Hidden state; W e and b e They are the weight matrix and bias value of the pre-trained model ALBERT respectively; Step 3.3: Use the second cross entropy loss function L e For training, the calculation formula is expressed as: in, is the total number of words; is the word w in event type c i Labels; To predict word w i The logical value of label wi For word w i The predicted label of ; argmax(·) is to find the maximum value.

4. According to the document-level natural disaster event extraction method integrating attention adjacency matrix according to claim 3, it is characterized in that: The specific process of step 4 is as follows: Step 4.1: Consider the relationships between words, arguments, trigger words, and arguments in all natural disaster event types, and construct the attention adjacency matrix M. k , the attention adjacency matrix M k The calculation formula for the value of each position in is expressed as: in, Indicates M k exist The value of the position; set(h) is the set of title sentences; set(t) is the node set of trigger words; set(a) is the node set of arguments; η and θ are different hyperparameters; g(·) is the nonlinear function log2(x+1), x is the function input parameter; g θ (·) is the nonlinear function g(θx+1); mdis is the average distance between two points; Step 4.2: Use the entity h in the event obtained by the pre-trained model ALBERT e Perform weighted sum to obtain text information The calculation formula is expressed as: h r0 =δh e +(1-δ)Albert(s i ) (11); Among them, δ is a hyperparameter; Step 4.3: Based on the generated attention adjacency matrix M k , the text information Input into the graph attention network KGAT for fusion calculation, the calculation formula is expressed as: in, For Node With neighbor nodes The similarity coefficient between For Node With neighbor nodes The attention weight coefficient between ; is the corrected attention weight coefficient; and Is a node and neighbor nodes The eigenvector of z is the weight matrix for the node and neighbor nodes The feature vector of is dimensionally increased; || is vector concatenation; f(·) is a single-layer feedforward neural network; LeakyReLU(·) is the LeakyRelu activation function; For Node The set of all adjacent nodes of ; For other neighbor nodes; For Node With other neighboring nodes The similarity coefficient between Step 4.4: Obtain the new feature vector of each node by weighted summing of neighboring node features The calculation formula is expressed as: Among them, W t is the connection type mapping matrix; m is the serial number of the multi-head attention, M is the number of multi-head attention, || is the vector concatenation; Tanh(·) is the hyperbolic tangent function.

5. According to the document-level natural disaster event extraction method integrating attention adjacency matrix according to claim 4, it is characterized in that: The specific process of step 5 is as follows: Step 5.1: The graph network layer uses the knowledge graph attention network KGAT to perform cyclic calculations at the context location of the trigger word entity to determine whether the argument entity belongs to the current natural disaster event type; Step 5.2: For each argument, select the hidden state sequence corresponding to the entity position in the output of the knowledge graph attention network The hidden state feature vector of the entity corresponding to each argument is obtained by average pooling. The calculation formula is expressed as: in, For the context The hidden state feature vector of the entity corresponding to each argument; Step 5.3: Get the logical value of the corresponding position through classification. The calculation formula is expressed as: in, For the context The logical value of the corresponding position of the argument; W g and b g They are the weight matrix and bias value of the knowledge graph attention network respectively; Step 5.4: Use the third cross entropy loss function L g For training, the calculation formula is expressed as: in, is the corresponding argument type; is the number of types of arguments; For the context Total number of class arguments; is the label of the argument; when the event type c selects the position belonging to Class Theory Meta-Time otherwise When reasoning, take the maximum logical value corresponding to the argument entity Compared with the threshold ε, when When , take the entity of the corresponding position segment span as the argument in the current natural disaster event; when When the corresponding natural disaster event argument takes a null value, the calculation formula is expressed as: Finally, the natural disaster events are filled in according to the argument results.

Images