Event detection method and system based on document-level graph convolutional network

Abstract

The present disclosure relates to an event detection method and system based on a document-level graph convolutional network. The method comprises: receiving an input document, the input document comprising a plurality of sentences; detecting an event in the input document using a document-level graph convolutional network model, the document-level graph convolutional network model comprising a global node and candidate trigger word nodes, wherein the global node is connected to each candidate trigger word node and an initial vector of the global node is randomly initialized, and the candidate trigger word is associated with the event; and outputting a detected event and an event type corresponding to the event.

Description

Technical Field

[0001] The present disclosure relates to an event detection method and system, and in particular to an event detection method and system based on a document-level graph convolutional network. Background Art

[0002] Event detection involves analyzing text data to determine whether and which events are contained in the text. Existing text classification methods can determine whether a text contains events, but they cannot identify which events are contained in the text, the type of each event, or the relationships between events.

[0003] The present disclosure is improved in view of but not limited to the above-mentioned factors. Summary of the Invention

[0004] To solve the above problems, this paper provides an event detection method for document-level long text modeling, which further improves the automation efficiency of event detection, more accurately identifies events in text, and can identify event types and dependencies between events.

[0005] The present disclosure uses a document-level graph convolution network (GCN) structure to perform end-to-end training on long text documents. On the one hand, it can learn sentence-level knowledge through graph convolution network modeling within the sentence, and at the same time use global nodes to connect subgraphs at different sentence levels so that information can be transmitted across sentences. In this way, the present disclosure solves the modeling problem of long texts at the document level on the one hand, and on the other hand, it can learn the relationship between events through information interaction between events. The method of the present disclosure thus greatly improves the recognition effect of event detection. For example, the evaluation index F1 value of the present disclosure (an evaluation index that combines the two indicators of precision and recall, used to comprehensively reflect the overall index) is better than the existing graph convolution network model, with an improvement of 2.9 percentage points.

[0006] According to a first aspect of the present disclosure, an event detection method based on a document-level graph convolutional network is provided, comprising: receiving an input document, the input document comprising a plurality of sentences; detecting an event in the input document through a document-level graph convolutional network model, the document-level graph convolutional network model comprising a global node and a candidate trigger word node, wherein the global node is connected to each candidate trigger word node, and the initial vector of the global node is randomly initialized, and the candidate trigger word is associated with the event; and outputting a detected event and an event type corresponding to the event.

[0007] According to one embodiment, detecting events in the input document through a document-level graph convolutional network model also includes, for each sentence included in the input document: converting the sentence into a vector at the word level through word embedding; establishing syntactic connections between words in the sentence through a syntactic analyzer; performing modeling on the vector to generate a context-aware vector; and inputting the generated context-aware vector and the established connections between words into the document-level graph convolutional network model for detecting events in the sentence.

[0008] According to another embodiment, the word embedding concatenates information of the word itself, entity type from named entity recognition (NER), position information from relative position relationship of words, and / or part-of-speech information from part-of-speech tags (POS tags).

[0009] According to yet another embodiment, the context-aware vector is modeled by BiLSTM.

[0010] According to yet another embodiment, the global node is updated as follows:

[0011]

[0012] Among them, Q l is the vector of the global node after update, M is the graph convolution network function, A d is the adjacency matrix of the vector matrix corresponding to the candidate trigger words and the vector of the global node, is the vector matrix corresponding to the candidate trigger word before updating, Q l-1 is the vector of the global node before updating, W d is a learnable parameter, and l is an integer greater than or equal to 1.

[0013] According to yet another embodiment, the adjacency matrix is ​​constructed by concatenating a vector matrix corresponding to candidate trigger words in the input text with a vector of the global node.

[0014] According to yet another embodiment, word nodes are updated as follows:

[0015]

[0016]

[0017] in, is the updated vector matrix corresponding to the i-th sentence in the input document, M is the graph convolutional network function, is the adjacency matrix of the vector matrix corresponding to the i-th sentence in the input document and the vector of the global node, is the vector matrix corresponding to the i-th sentence in the input document before updating, Q l is the vector of the global node after update, W s is a learnable parameter, H l is the updated vector matrix corresponding to the input document, n is the number of sentences in the input document, l is an integer greater than or equal to 1, and i is an integer greater than or equal to 1 and less than or equal to n.

[0018] According to yet another embodiment, the adjacency matrix is ​​constructed by concatenating a matrix of vectors in the input text corresponding to corresponding sentences in the input document with the vector of the global node.

[0019] According to yet another embodiment, the global nodes and the word nodes are updated alternately.

[0020] According to yet another embodiment, the document-level graph convolutional network model is trained using an end-to-end training method.

[0021] According to a second aspect of the present disclosure, an event detection system based on a document-level graph convolutional network is provided, the system comprising: a processor; and a memory arranged to store computer-executable instructions, which, when executed by the processor, cause the processor to perform the method described in the first aspect of the present disclosure.

[0022] Aspects generally include methods, apparatus, systems, computer program products, and processing systems substantially as described herein with reference to and as illustrated by the accompanying figures.

[0023] The foregoing has broadly outlined the features and technical advantages of examples according to the present disclosure so that the following detailed description may be better understood. Additional features and advantages will be described hereinafter. The concepts and specific examples disclosed may be readily used as a basis for modifying or designing other structures for implementing the same purposes as the present disclosure. Such equivalent training does not depart from the scope of the appended claims. The characteristics of the concepts disclosed herein, both in terms of their organization and method of operation, and the associated advantages will be better understood by considering the following description in conjunction with the accompanying drawings. Each of the figures is provided for illustration and description purposes and does not define limitations on the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to understand in detail the manner in which the above-recited features of the present disclosure may be employed, reference may be made to various aspects of a more particular description of the content briefly summarized above, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only certain typical aspects of the disclosure and are not to be considered limiting of its scope, as the description may admit to other equally effective aspects. The same reference numerals in different drawings may identify the same or similar elements.

[0025] Figure 1 An example structure diagram of an existing graph convolutional network is shown;

[0026] Figure 2 A flowchart of an example event detection method based on a document-level graph convolutional network according to an embodiment of the present disclosure is shown;

[0027] Figure 3 A diagram illustrating an event detection model based on a document-level graph convolutional network according to an embodiment of the present disclosure is shown;

[0028] Figure 4 The following is a specific example illustrating an example process of an event detection method based on a document-level graph convolutional network according to the present disclosure.

[0029] Figure 5 An example method of constructing the adjacency matrix of candidate trigger words and global nodes during the update process of the global node is shown;

[0030] Figure 6 shows an example construction method of the adjacency matrix of the corresponding sentence and the global node during the update process of the word node;

[0031] Figure 7 shows an example update process for global nodes and word nodes;

[0032] Figure 8 shows corresponding event detection results for two documents according to an embodiment of the present disclosure; and

[0033] Figure 9 A schematic diagram of an example event detection system 900 based on a document-level graph convolutional network is shown. DETAILED DESCRIPTION

[0034] The following is combined with Figure 1-9 The detailed description set forth is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. This detailed description includes specific details to provide a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details.

[0035] Explanation of terms:

[0036] Event detection: In this disclosure, it refers to the process of using machine learning to determine which events are contained in the input text data, specifically locating the position of event trigger words and determining the event type.

[0037] Graph Convolutional Network (GCN): A deep learning model used to learn the relationships between nodes in a graph structure.

[0038] Graph structure: An algorithmic structure in which each node establishes edge relationships with a subset of other nodes.

[0039] There are many event detection algorithms, among which graph convolutional network algorithm is one of them. Figure 1 As shown in Figure 1, it shows the model structure of an existing graph convolutional network, in which the input graph is modeled and processed through hidden layers, ReLU, and other methods to obtain the output. However, with the widespread application of graph convolutional network models, the shortcomings of this model have become increasingly apparent. First, graph convolutional network models can only model at the sentence level. This is mainly because the edge connections in graph convolutional networks come from the syntactic analyzer, which can only establish sentence-level node dependencies. Second, the events recognized by graph convolutional networks are isolated from each other, and the mutual influence and dependencies between events are missing.

[0040] Therefore, the present disclosure proposes an event detection method for modeling long texts at the document level, thereby further improving the automation efficiency of event detection, more accurately identifying events in the text, and being able to identify event types and dependencies between events. The present disclosure proposes a document-level graph convolutional network model to solve the problem of long text composition, and solves the problem of excessive computational overhead by alternating updates of sentence-level subgraphs and document-level graphs. At the same time, an end-to-end training method is provided to train the document-level graph convolutional network model. As a result, the present disclosure greatly improves the recognition effect of event detection.

[0041] The following combination Figure 2 The event detection method based on document-level graph convolutional network of the present disclosure is described as follows. As shown in the figure, it shows a flowchart of an example event detection method 200 based on document-level graph convolutional network according to an embodiment of the present disclosure.

[0042] refer to Figure 2 Method 200 may include, at block 210, receiving an input document. In one example, the input document may include a plurality of sentences. For example, the input document may be a news article included in an HTML-formatted web page. Those skilled in the art will appreciate that the input document may be in any suitable format including a plurality of sentences.

[0043] Next, the method 200 may include, in box 220, detecting events in the input document through a document-level graph convolutional network model. In one embodiment of the present disclosure, the document-level graph convolutional network model may include a global node and a candidate trigger word node. In this embodiment, the candidate trigger word is a trigger word associated with an event, that is, if the candidate trigger word exists in the text, it can be considered that there is a certain probability that a related event exists in the text. In the document-level graph, the global node is connected to each candidate trigger word node. In this way, during the training process, the trigger word can, on the one hand, absorb the internal information of the sentence for identifying the corresponding event type of the trigger word, and on the other hand, the global node can act as an information bridge between sentences so that document-level information can interact between sentences. In another preferred embodiment of the present disclosure, the global node does not establish connections with all the word nodes in the sentence, thereby reducing unnecessary noise information.

[0044] In one embodiment of the present disclosure, detecting events in an input document through a document-level graph convolutional network model may also include first preprocessing the input document, for example, processing the input document through an embedding layer, a BiLSTM layer, a syntactic analyzer, etc.

[0045] like Figure 3 As shown, the example document-level graph convolutional network model of the present disclosure may include an embedding layer, a BiLSTM layer, a document-level graph convolutional network layer, a CRF (Conditional Random Fields) layer, and the like. In this embodiment, detecting events in an input document by using a document-level graph convolutional network model may include, for each sentence included in the input document: converting the sentence into a vector at the word level by word embedding; performing modeling on the vector (such as by BiLSTM modeling) to generate a context-aware vector; establishing syntactic connections between the words in the sentence by a syntactic analyzer (thereby forming edge connections between nodes corresponding to the words in the subgraph corresponding to the sentence); and inputting the generated context-aware vector and the established connections between the words into the document-level graph convolutional network model for detecting events in the sentence. Further according to this embodiment, word embedding may concatenate information about the word itself, entity types from named entity recognition (NER), position information from relative position relationships of words, and / or part-of-speech information from part-of-speech tags (POStags), and the like.

[0046] In summary, word embedding is responsible for word-level encoding, BiLSTM is responsible for context-aware word encoding, the document-level graph convolutional network is responsible for document-level multi-event-aware word encoding, the CRF layer is used to comprehensively model the output probability and transition probability, and the syntactic analyzer is responsible for building edge connection relationships between nodes.

[0047] For example, Figure 4 This example illustrates the process of the document-level graph convolutional network-based event detection method according to the present disclosure, using a specific document example. In this example, the document contains the phrase "...Shanghai Bank was fined 4.5 million yuan...." Since the event is related to anti-money laundering, "fine" is identified as a candidate trigger word.

[0048] like Figure 4 As shown, the input text is converted into a numerical vector at the word level. For example, the sentence "Shanghai Bank was fined 4.5 million" is divided into the individual words "Shanghai", "Bank", "was", "fine (fined)", and "4.5 million" through word embedding; these word vectors are processed by BiLSTM to learn context-aware vectors, which are executed once forward and once backward, and the two layers of vectors are spliced. Subsequently, it is fed into the document-level graph convolutional network. It will be understood that each sentence in the document can be used as a subgraph in the document-level graph convolutional network. Figure 4 As can be seen in the figure, the global node is connected to each candidate trigger word node, thus acting as an information bridge between sentences, allowing document-level information to be exchanged between sentences. It can also be seen that the global node is not connected to all word nodes in the sentence, thus reducing unnecessary noise information and making the event detection results more accurate.

[0049] In one embodiment of the present disclosure, the document-level graph convolutional network is updateable. Considering the limitation of GPU performance, in one embodiment, instead of updating the entire document-level graph convolutional network, the document-level graph convolutional network is updated. Figure 1 Instead of iterating sequentially, global nodes (corresponding to the document-level graph) and word nodes (corresponding to the sentence-level subgraph) are updated alternately. For example, first use the global nodes and trigger word candidate nodes to form a graph (for example, Figure 4 The global nodes and candidate trigger word nodes in the graph and the solid lines between them are used to update the global node; and then the updated global node is used to update each subgraph in turn (for example, Figure 4 The word nodes and candidate trigger word nodes within each subgraph and the connecting lines between them).

[0050] In one embodiment, during the global node update process, the vector matrix corresponding to the candidate trigger word in the input text can be taken out and spliced ​​with the vector Q of the global node (for example, the initial vector of the global node is randomly initialized, and the parameters are updated as the network continues to learn) to construct the adjacency matrix Ad, thereby calculating the updated vector matrix. Figure 5 An example method for constructing the adjacency matrix of candidate trigger words and global nodes during the update process of the global nodes is shown.

[0051] After the update is completed, the vector of the global node is taken as an update of Q. For example, the global node is updated as follows:

[0052]

[0053] Among them, Q 1 is the vector of the global node after update, M is the graph convolution network function, A d is the adjacency matrix of the vector matrix corresponding to the candidate trigger words and the vector of the global node, is the vector matrix corresponding to the candidate trigger word before updating, Q l-1 is the vector of the global node before updating, W d is a learnable parameter, and l is an integer greater than or equal to 1.

[0054] In another embodiment, during the word node update process, the subgraph vector of each sentence and the vector of the global node can be taken and concatenated together to construct the corresponding adjacency matrix and perform calculation updates. For example, Figure 6 This shows an example of how to construct the adjacency matrix of the corresponding sentence and the global node during the word node update process. After the update is completed, the corresponding part of the word vector is taken as an update of the word vector.

[0055] For example, the word node is updated as follows:

[0056]

[0057]

[0058] in, is the updated vector matrix corresponding to the i-th sentence in the input document, M is the graph convolutional network function, is the adjacency matrix of the vector matrix corresponding to the i-th sentence in the input document and the vector of the global node, is the vector matrix corresponding to the i-th sentence in the input document before updating, Q 1 is the vector of the global node after update, W s is a learnable parameter, H l is the updated vector matrix corresponding to the input document, n is the number of sentences in the input document, l is an integer greater than or equal to 1, and i is an integer greater than or equal to 1 and less than or equal to n.

[0059] In the above way, global nodes and word nodes can be updated alternately. In the actual calculation process, due to the multi-layer document-level graph convolutional network, the global nodes are updated first and then the word nodes are updated in each layer. From the overall point of view, the two are updated alternately. The vector matrix and adjacency matrix operation diagram in this update process is shown in Figure 7 , which shows an example update process of global nodes and word nodes. Figure 7As can be seen in , the embedding of the global node is updated first, and then the updated embedding of the global node is used to update each word node of the corresponding subgraph of each sentence, thereby forming the updated embedding of each sentence.

[0060] Continue to refer Figure 2 Finally, the method 200 may include outputting the detected event and the event type corresponding to the event at block 230. For example, as described above in conjunction with Figure 3 As described above, the CRF layer can be used to comprehensively model the output probability and the transition probability, and output the detected event and the event type corresponding to the event. In another embodiment, the output result can also include the dependency relationship between the events.

[0061] The following takes an anti-money laundering related event as an example to provide the detection results output by the method 200 of the present disclosure.

[0062] like Figure 8 , which shows corresponding event detection results for two documents using the method described above (eg, method 200 ) according to an embodiment of the present disclosure.

[0063] The first document contains the following: ...The China Banking and Insurance Regulatory Commission recently published the Guangdong Banking and Insurance Regulatory Bureau's administrative penalty information disclosure form. ...[PER] was given a warning and fined 40,000 yuan. ...

[0064] The output results of method 200 show that it identifies "announcement", "warning", "fine", etc. as event trigger words, and also identifies two anti-money laundering events: the "anti-money laundering regulatory announcement" event and the "anti-money laundering regulatory punishment" event, and identifies the dependency relationship between the two events.

[0065] The second document contains the following: ...Multiple companies' community group-buying businesses were triggered by the Market Supervision Bureau. ...[COM] was fined 500,000 RMB. ...

[0066] The output result of method 200 shows that it identifies event trigger words such as "penalty" and "fine", but also identifies two events that are not anti-money laundering events, and also identifies the dependency relationship between the two events.

[0067] It can be seen that the method disclosed in the present invention can identify the types of events, the mutual influence and dependencies between events, and greatly improve the recognition effect of event detection.

[0068] Figure 9Schematic diagram of an example event detection system 900 based on a document-level graph convolutional network is shown. The system 900 may include a processor 905 and a memory 910 arranged to store computer-executable instructions 915, which, when executed by the processor 905, may cause the processor 905 to perform the process according to the present disclosure. Figure 2-8 Method 200 is described.

[0069] The above detailed description includes references to the accompanying drawings, which form part of the detailed description. The accompanying drawings illustrate specific embodiments that can be put into practice by way of illustration. These embodiments are also referred to herein as "examples." Such examples may include elements other than those shown or described. However, examples that include the shown or described elements are also contemplated. In addition, examples using any combination or arrangement of those elements shown or described are also contemplated, or with reference to the specific examples (or one or more aspects thereof) shown or described herein, or with reference to other examples (or one or more aspects thereof) shown or described herein.

[0070] In the appended claims, the terms "including" and "comprising" are open-ended, that is, systems, apparatuses, articles, or processes that include elements in addition to those listed after such terms in a claim are considered to fall within the scope of that claim. Furthermore, in the appended claims, the terms "first," "second," and "third," etc. are used merely as labels and are not intended to indicate a numerical order to their objects.

[0071] In addition, the order of each operation explained in this specification is exemplary. In alternative embodiments, each operation can be performed in a different order than that shown in the drawings, and each operation can be combined into a single operation or split into more operations.

[0072] The above description is intended to be illustrative, not restrictive. For example, the examples described above (or one or more aspects thereof) may be used in conjunction with other embodiments. Other embodiments may be used, such as by a person of ordinary skill in the art after reviewing the above description. The abstract allows the reader to quickly determine the nature of the present disclosure. This abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the above specific embodiments, various features may be grouped together to make the disclosure smooth. However, the claims may not state every feature disclosed herein, as the embodiments may characterize a subset of the features. In addition, an embodiment may include fewer features than those disclosed in a particular example. Therefore, the appended claims are thereby incorporated into the specific embodiments, and a claim exists independently as a separate embodiment. The scope of the embodiments disclosed herein should be determined with reference to the appended claims and the full scope of equivalents to which such claims are entitled.

Claims

1. An event detection method based on a document-level graph convolutional network, comprising: receiving an input document, wherein the input document includes a plurality of sentences; Detecting events in the input document using a document-level graph convolutional network model, the document-level graph convolutional network model including a global node and a candidate trigger word node, wherein the global node is connected to each candidate trigger word node, and an initial vector of the global node is randomly initialized, and the candidate trigger word is associated with the event, wherein detecting events in the input document using the document-level graph convolutional network model further includes, for each sentence included in the input document: Convert the sentence into a vector at the word level through word embedding; Use a syntactic analyzer to establish the syntactic connections between the words in the sentence; performing modeling on the vector to generate a context-aware vector; as well as Inputting the generated context-aware vector and the established connections between words into the document-level graph convolutional network model for detecting events in the sentence; as well as Outputs the detected event and the event type corresponding to the event.

2. The method according to claim 1, wherein the word embedding concatenates information about the word itself, entity types from named entity recognition (NER), position information from the relative position relationship of words, and / or part-of-speech information from part-of-speech tags (POS tags).

3. The method according to claim 1, wherein the context-aware vector is modeled by BiLSTM.

4. The method of claim 1 , wherein the global node is updated as follows: in, Q l is the vector of the global node after update, M is the graph convolution network function, A d is the adjacency matrix of the vector matrix corresponding to the candidate trigger words and the vector of the global node, is the vector matrix corresponding to the candidate trigger word before updating, Q l-1 is the vector of the global node before updating, W d is a learnable parameter, and l is an integer greater than or equal to 1.

5. The method according to claim 4, wherein the adjacency matrix is ​​constructed by concatenating the vector matrix corresponding to the candidate trigger words in the input text with the vector of the global node.

6. The method according to claim 1, wherein the word node is updated as follows: in, is the updated vector matrix corresponding to the i-th sentence in the input document, M is the graph convolutional network function, is the adjacency matrix of the vector matrix corresponding to the i-th sentence in the input document and the vector of the global node, is the vector matrix corresponding to the i-th sentence in the input document before updating, Q l is the vector of the global node after update, W s is a learnable parameter, H l is the updated vector matrix corresponding to the input document, n is the number of sentences in the input document, l is an integer greater than or equal to 1, and i is an integer greater than or equal to 1 and less than or equal to n.

7. The method according to claim 6, wherein the adjacency matrix is ​​constructed by concatenating a matrix of vectors in the input text corresponding to corresponding sentences in the input document with the vector of the global node. The method according to claim 1 , wherein the global node and the word node are updated alternately.

9. The method according to claim 1, wherein the document-level graph convolutional network model is trained using an end-to-end training method.

10. An event detection system based on a document-level graph convolutional network, the system comprising: processor; as well as A memory arranged to store computer executable instructions which, when executed by the processor, cause the processor to perform the method of any one of claims 1 to 9.

Images