A neural network-based event coreference resolution method

Abstract

The application discloses a neural network-based event co-reference resolution method, which comprises the following steps: S1, acquiring coarse-grained annotation information and fine-grained annotation information, and hierarchically processing the information to be used for training high-layer and low-layer double-layer graph neural networks respectively; S2, establishing a high-layer network graph for modeling coarse-grained annotation information and guiding edge connection information of a low-layer network graph; S3, establishing a low-layer network graph for modeling fine-grained annotation information and transmitting vector features to the high-layer network graph; S4, performing multiple rounds of circulation, in each round of circulation, the high-layer network graph guides edge connection of the low-layer network graph, and the low-layer network graph also updates node features of the high-layer network graph; the circulation improves an event co-reference prediction process until an optimal result is achieved; in the last circulation, the low-layer network graph gives all event co-reference predictions participating in calculation at one time. The method significantly reduces the dependence of the model on training data, greatly improves prediction efficiency, and exhibits excellent performance when processing large-scale data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of graph neural networks and natural language processing, and particularly relates to an event coreference resolution method based on a neural network. BACKGROUND

[0002] Event coreference resolution refers to finding out that a plurality of event texts refer to the same real event and merging such texts to eliminate ambiguity. This task belongs to the information extraction direction of the natural language processing field and is an essential part of establishing a knowledge graph in modern computer technology. Generally speaking, the daily texts used by people will contain various events, both timely news events and past historical events. How to sort out and summarize these event information is a very concerned problem in the field of natural language processing at present. In order to accurately capture the relationship between events from the text, researchers introduce the concept of "mention", which refers to the trigger word or phrase representing a specific event. Then, based on these mentions, the relationship between events can be established. Especially when multiple mentions point to the same real-world event, the task of event coreference resolution is involved.

[0003] In order to solve this task, the current event coreference resolution method usually relies on deep learning technology and includes two main stages of training and reasoning. In the training stage, the current method uses pre-labeled mention pair data to train a pair classifier, aiming to teach the model to identify which mentions are related to each other. Subsequently, in the reasoning stage, this classifier is used to evaluate the identified mention pairs in one or more documents to determine whether there is a coreference relationship between them. At present, the event coreference resolution task can be divided into intra-document event coreference resolution and cross-document event coreference resolution according to whether the events are distributed in the same document.

[0004] For intra-document event coreference, early techniques include support vector machines used by Chen and Ng in 2014 and decision trees used by Cybulska and Vossen in 2015. Since 2016, deep neural networks have been introduced into event coreference resolution, leading to a large number of related work, among which Lu Jie proposed an end-to-end neural event coreference resolution (E2ENeRC) in 2022. 3C) is the latest result of this technology. In addition, there are some methods that try to use techniques in other tasks to enhance the effect of event coreference resolution. Including the entity coreference resolution method joined by Lu et al. in 2016 and the trigger word detection method joined by Lu and Ng in 2017. In recent years, a new trend is to regard the event coreference resolution task as a graph learning problem, which is studied by Tran and Phung et al. in 2021. The idea of this method is that event mentions and their relationships can be regarded as nodes and edges in a graph, and modeling based on this assumption can better complete the task. However, when the graph is constructed based on fixed rules or pre-trained classifiers, this type of method cannot be well implemented.

[0005] For the cross-document event coreference resolution task, Lee et al. and Choubey and Huang try to simplify the problem by reducing the number of documents processed, eventually transforming it into an intra-document event coreference resolution task. In addition, in order to make more effective use of the information in the document, Kenyon-Dean et al. proposed CORE in 2018, which fuses the representation of the document with its corresponding mention pair to train a more accurate classifier. Finally, Cattan et al. proposed an end-to-end cross-document method in 2021, which can process all parts of the document, from obtaining candidate mention pairs to constructing reference clusters.

[0006] Although the above methods perform well on small-scale data, their performance is not satisfactory when faced with large-scale, diverse and complex document sets. The main problems are in two aspects: first, these methods are often constrained by the lack of training samples. The annotation of large-scale document sets requires huge manpower and time, which makes it very difficult and expensive to obtain high-quality and large amounts of training data. Second, from the perspective of computation, most existing methods adopt a pair-wise discriminative strategy. This means that in order to determine whether two mentions refer to the same event, all possible mention pairs need to be discriminated. Therefore, the computational complexity is in quadratic relationship with the number of mention pairs, i.e. O(n^2). With the growth of data size, this quadratic computational complexity makes these methods extremely inefficient in processing large data, and even may not be able to complete the task within a practical time.

[0007] It should be noted that the information disclosed in the above background section is only for understanding the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0008] The main purpose of the present application is to overcome the defects of the above background technology, and provide an event coreference resolution method based on neural network.

[0009] To achieve the above object, the application adopts the following technical solutions:

[0010] An event co-reference resolution method based on a neural network comprises the following steps:

[0011] S1, coarse-grained annotation information and fine-grained annotation information are acquired, and they are processed hierarchically to be used for training high-layer and low-layer double-layer graph neural networks respectively;

[0012] S2, a high-layer network graph is established, which is used for modeling coarse-grained annotation information and guiding edge connection information of a low-layer network graph; in the high-layer network graph, node features are not completely covered by new features, but a part of original features is kept unchanged as anchor points, and another part of features is dynamically updated;

[0013] S3, a low-layer network graph is established, which is used for modeling fine-grained annotation information and delivering vector features to the high-layer network graph; the low-layer network graph periodically delivers vector features to the high-layer network graph, and meanwhile the high-layer network graph samples a part of edges and delivers them to the low-layer network graph as a supplement to the edges of the low-layer network graph;

[0014] S4, a plurality of rounds of circulation are performed, in each round of circulation, the high-layer network graph guides the edge connection of the low-layer network graph, and the low-layer network graph also updates the node features of the high-layer network graph; the circulation improves the event co-reference prediction process until an optimal result is reached; in the last circulation, the low-layer network Figure One gives all event co-reference predictions participating in the calculation at one time.

[0015] Further,

[0016] The high-layer network graph is constructed using document-level information, and the low-layer network graph is constructed using event-level information.

[0017] In the preprocessing stage, given an input document and mentions therein, a pre-trained model is used to obtain character representations, and a high-layer network graph at the document level and a low-layer network graph at the event level are constructed; the high-layer network graph takes a document as a node and a relationship between documents as an edge, and an adjacency matrix representing the relationship between documents is obtained; the low-layer graph takes an event as a node and a co-reference event within a document as an edge, and an adjacency matrix representing the relationship between events is obtained; preferably, for the low-layer graph, its initial node features are obtained through maximum pooling; for the high-layer graph, the average representation of mentions within a document is used to initialize its nodes.

[0018] In the high-layer network graph, a variational graph autoencoder VGAE is used to construct a graph inference model; for the graph of the initial adjacency matrix, its connection relationship will be continuously corrected in subsequent calculations, and the correction result is derived from the inner product of the transformed representations of the nodes, and the representation of the node is continuously updated in the previous circulation under the influence of the low-layer graph.

[0019] In the low-layer network graph, the graph features are encoded using the same variational graph encoder, but two independent decoders are used for intra-document and cross-document tasks; in the loop process, the update of the node input feature matrix of the low-layer network graph is generated spontaneously by the graph neural network of the low-layer network graph, and the change of the adjacency matrix is completed by the low-layer network graph and the high-layer network graph.

[0020] In the low-layer network graph, the cross-document connection relationship is obtained by the encoder of the low-layer graph, and the intra-document connection is obtained by the enhanced representation inner product.

[0021] The node feature update of the low-layer network graph to the high-layer network graph comprises the following steps: before each training cycle of the high-layer network graph, an updated feature is spliced to the initial feature of the high-layer network graph, and the update ensures that all mentions come from the same document, and the weight of the cross-document mention is forcibly set to 0.

[0022] The edge connection information of the low-layer network graph guided by the high-layer network graph comprises the following steps: after the adjacency matrix of the current high-layer network graph is given, the first k1 most relevant document pairs are selected according to the numerical ordering of the adjacency matrix, and then the importance score of the mention in each document is obtained by normalizing the row of the weight matrix, thereby the maximum numerical index u in the column direction is selected, and T = u is used. T The importance table T is calculated, wherein represents Hadamard product, and according to the importance table T, the k2 most important mention pairs are extracted from the selected several documents, and the mention pairs are added to the adjacency matrix of the low-layer network graph as additional edges.

[0023] In the model optimization process, the high-layer network graph adopts binary cross entropy BCE as the loss function; the low-layer network graph also uses binary cross entropy as the loss function, but calculates the intra-document loss and the cross-document loss respectively, and adds them with different weights.

[0024] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the event coreference resolution method.

[0025] The application has the following beneficial effects:

[0026] The application provides an event coreference resolution technology based on a double-layer graph neural network. The method adopts a double-layer graph structure, so that two types of annotation information can be used in parallel. Key content is extracted from coarse-grained information, and fine-grained information is further optimized. Conversely, through fine-grained information, the representation ability of the coarse-grained network graph is enhanced. This mechanism forms a mutual iteration process, effectively enhances the prediction accuracy of event coreference, and significantly reduces the dependence of the model on a large amount of training data. In the prediction stage, the traditional pair prediction strategy is abandoned, and a graph neural network is used to directly generate all event coreference pairs. The method of the application significantly reduces the dependence of the model on training data, greatly improves the prediction efficiency, and exhibits excellent performance when processing large-scale data.

[0027] Other beneficial effects of the embodiments of the application will be further described below. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A flowchart of the event coreference resolution method based on the neural network of the embodiments of the application.

[0029] Figure 2 A schematic diagram of the overall framework of the event coreference resolution method based on the neural network of the embodiments of the application.

[0030] Figure 3 A pseudo code diagram of the training algorithm of the embodiments of the application. DETAILED DESCRIPTION

[0031] The embodiments of the application will be described in detail below. It should be emphasized that the following description is merely exemplary and is not intended to limit the scope of the application and its applications.

[0032] Referring to Figure 1 and Figure 2 The embodiments of the application provide an event coreference resolution method based on a neural network, which comprises the following steps:

[0033] S1, coarse-grained annotation information and fine-grained annotation information are obtained, and the information is processed in layers to be used for training high-layer and low-layer double-layer graph neural networks, respectively;

[0034] S2, a high-layer network graph is established, which is used for modeling coarse-grained annotation information and guiding the edge connection information of the low-layer network graph; wherein the node features in the high-layer network graph are not completely covered by new features, but a part of the original features is kept unchanged as an anchor point, and another part of the features is dynamically updated;

[0035] S3, a low-layer network graph is established to model fine-grained annotation information and deliver vector features to a high-layer network graph; the low-layer network graph periodically delivers vector features to the high-layer network graph, while the high-layer network graph samples a part of edges with high confidence to deliver to the low-layer network graph as a supplement to the edges of the low-layer network graph;

[0036] S4, a multi-round cycle is performed, in each round, the high-layer network graph guides the edge connection of the low-layer network graph, while the low-layer network graph updates the node features of the high-layer network graph; the cycle improves the event coreference prediction process until an optimal result is reached; in the last cycle, the low-layer network graph gives all the event coreferences involved in the calculation at one time. Figure One

[0037] In the present application, in view of the situation that coarse-grained annotation information is easy to obtain but fine-grained annotation information is scarce in the coreference resolution problem, the above two types of annotation information are simultaneously utilized by the double-layer graphing method, valuable content is extracted from the coarse-grained information to supplement the fine-grained information, and representation is obtained from the fine-grained information to perfect the coarse-grained network graph, the prediction process of event coreference is iteratively improved, all event coreferences are predicted at one time, and thus the dependence of the model on training data is reduced.

[0038] Through the above steps of the embodiments of the present application, the event coreference resolution method of the present application can utilize coarse-grained annotation information and fine-grained annotation information, iteratively improve the prediction process of event coreference, predict all event coreferences at one time, significantly reduce the dependence of the model on training data, greatly improve the prediction efficiency, and exhibit excellent performance when processing large-scale data.

[0039] The specific embodiments of the present application are further described below.

[0040] In some specific embodiments, a method for event coreference resolution is used to relieve the problem of lack of training data by a double-layer graph neural network modeling, and to improve the prediction speed of coreference pairs. The model framework of the present method is as follows: Figure 2 ​As shown: the whole is divided into two parts, the upper layer is the high-level network graph constructed according to the coarse-grained annotation data, and the lower layer is the low-level network graph constructed according to the fine-grained annotation data. For the cross-document event coreference resolution task, the upper layer network graph takes the document as the node and the document type relationship as the edge; the lower layer network graph takes the event mention as the node and the event coreference relationship as the edge. After the construction of the upper and lower double-layer graphs, the document representation and the event representation are obtained by means of the pre-trained language model, and the nodes in the graph are initialized. Then, the standard variational graph autoencoder (VGAE) is used to connect and predict the two graphs, as shown by the trapezoids in the figure, and the trapezoids in different directions represent the encoder and the decoder, respectively. The responsibility of the encoder is to transform the node representation into a low-dimensional hidden vector, and the decoder uses these hidden vectors to perform self-supervised graph reconstruction. Both the high-level graph and the low-level graph are equipped with an encoder and a decoder, and they do not share parameters with each other. During the learning process, the two graph structures constantly interact and share information, and the upper layer graph provides connection guidance for the lower layer graph, and the lower layer graph updates the node representation of the upper layer graph.

[0041] In the preprocessing stage, given the input document and the mentions therein, the pre-trained model BERT-base is used to obtain the character representation. In this stage, two graphs will be constructed, one high-level graph at the document level and one low-level graph at the event level. The high-level graph takes the document as the node and the relationship between documents as the edge, and its adjacency matrix is denoted as A D . The low-level graph takes the event as the node and the coreferential event within the document as the edge, and the adjacency matrix is denoted as A E . For the low-level graph, the initial node feature is obtained by maximum pooling, denoted as X E . At the same time, in the high-level graph, the average representation of the mentions within the document is used to initialize the node, denoted as X D . In this way, the preprocessing of the document and the initialization and construction of the double-layer graph are completed.

[0042] In the high-level graph, the graph inference model is constructed according to the standard variational graph autoencoder (VGAE). For the graph with the initial adjacency matrix A D , its connection relationship will be continuously modified in the subsequent calculation, and the modification result comes from the inner product of the transformed node representation. The node representation X h will be affected by the low-level graph and will be updated in each cycle, and its calculation method is: X h = [X D ; X update ]. Where [A; B] represents the concatenation of A vector and B vector, X D represents the initialized node representation, X update represents the updated content fed back from the lower layer graph, and the new high-level node representation will be the concatenation of the initial representation and the updated representation. Then, using VGAE, an encoded intermediate variable Z h, whose formula is: Z h = GNN h (X h , A h ), where GNN h represents the encoder of the upper layer graph, and A h represents the adjacency matrix of the current upper layer graph. Finally, the new adjacency matrix will be obtained by the inner product of Z h , and the calculation method is: A’ h = Z h · Z h T During the model optimization process, binary cross-entropy (BCE) is used as the loss function to ensure that A’ h is as close as possible to A h in structure.

[0043] In the lower layer graph, the model will solve the intra-document and cross-document event coreference problems at the same time. The same variational graph encoder is used to encode the graph features, but two independent decoders are used for intra-document and cross-document tasks. Similar to the upper layer graph, the encoder of the lower layer graph can be represented as: Z l = GNN l (X l , A l ), where the subscript l represents the lower layer graph, and the initialization of X l and A l can be referred to the preprocessing part. During the loop process, the update of X l is generated spontaneously by the graph neural network, but the change of A l will be completed by the lower layer graph and the upper layer graph together, denoted as A l = A E + A guide . Here, A E represents the initialized adjacency matrix, and A guide is the guidance matrix of the upper layer graph to the lower layer graph. Similar to the upper layer graph, the cross-document connection relationship of the lower layer graph will also be obtained by Z l : A’ w = Z l · Z l T But the intra-document connection of the lower layer graph will be obtained by an enhanced representation inner product, and the calculation process is: Z aug = [Z l ; (ε⊙A l )·Z l ], A’ c = Z aug · Z aug Twhere represents Hadamard product. In the optimization process, the lower-layer graph also uses binary cross-entropy as the loss function, but the intra-document loss and cross-document loss are calculated respectively and added with different weights.

[0044] Information interaction is the core technology of the application, and how to guide the high and low layer graphs to interact information in multiple rounds of circulation is described in detail. First, the low layer graph updates the features of the high layer graph, and before each training cycle of the high layer graph, the initial feature X D of the high layer graph is updated to X update , and the calculation method is: X update =W·Z l where W is a learnable weight matrix, and its physical meaning is the weight of each mention in the sum, representing the importance of the mention in the document. Note that the update here must ensure that all mentions come from the same document, so the weight of the cross-document mention will be forcibly set to 0. Secondly, the connection guidance from the high layer graph to the low layer graph, after giving the adjacency matrix A h of the current high layer graph, first, according to the numerical value of the adjacency matrix, the first k1 most relevant document pairs are selected. At the same time, because the weight matrix W has been obtained in the previous step, the importance score of the mention in each document can be obtained after normalizing the rows of W, and the maximum value index u in the column direction is selected, and the table T is calculated by T=u T . According to the importance table T, the k2 most important mention pairs are extracted from the selected several documents, and these mention pairs will be added as additional edges to the adjacency matrix of the low layer graph.

[0045] Finally, in the specific training process, a cyclic interaction algorithm is designed, which can make the high and low layer graphs fully exchange information in training. Figure 3 The pseudo code of this algorithm is given in the following table, where represents the low layer graph model parameters, and represents the high layer graph model parameters. In addition, it is noted that the edge connection guidance from high to low and the representation enhancement of the low layer graph itself only occur at the beginning of each training cycle, while the update of the low layer graph to the high layer graph can occur multiple times.

[0046] The detailed situation of the experimental test part of the application is introduced below.

[0047] MAVEN-ERE is a commonly used data set for event coreference resolution, and has more diverse coreference categories and test numbers than other data sets. However, the MAVEN-ERE data set lacks document-level annotation information, which is needed by the invention to construct high-level graphs, so the invention additionally provides a re-annotated data set for MAVEN-ERE, and subsequent experiments are based on this data set. In actual operation, 739 non-isolated documents are selected according to the annotations of EventWiki, which involve 20,970 event mentions and more than 200 million pairs of to-be-identified mentions. A part of the intra-document connections is cut out as the training set for the model to learn, and the remaining intra-document and cross-document connections are equally divided into a validation set and a test set. Table 1 gives some statistical characteristics of the new data set.

[0048] Table 1

[0049]

[0050]

[0051] In the experimental setup, to ensure uniformity and reliability, all related experiments are performed on an A100 graphics card with a 40G memory capacity. For the training process of the model, a total of 10 rounds of training iterations are preset to ensure that the model can fully learn and adapt to the data characteristics. For the designed high and low level graph network, the number of internal iterations (training rounds of high and low level graphs) is set to 200 rounds. In order to ensure the stability and rapid convergence of the model, 0.001 is used as the initial learning rate. When calculating the loss function, both the intra-document and cross-document low-level graphs are assigned a BCE loss weight value of 1. Considering the guidance and guidance of the model, when performing edge connection guidance from high to low, the top 100 document pairs are selected as the benchmark, and further 200 mention edges are selected as the optimization guidance of the low-level graph.

[0052] The comparison method of the experiment selects the one-stage scheme and the two-stage scheme constructed by the variational graph autoencoder. The former directly applies the variational graph autoencoder to the high and low level graphs, and does not interact any information between them; the latter first predicts the relevance of the documents in the high level graph, then uses the Louvain algorithm to segment each document, and finally uses the segmentation result of the document to divide the mentions, and then predicts whether the mention pairs are coreferential.

[0053] In the experimental part, two strategies based on variational graph autoencoder were chosen as the control group, one is one-stage scheme, the other is two-stage scheme. For one-stage strategy, it directly applies variational graph autoencoder to two hierarchical graph structures. In this strategy, high and low layer graphs are independent of each other throughout the whole process, and there is no information interaction or sharing between them. While two-stage strategy adopts a phased processing method. In the first stage, it first focuses on the high-level graph to predict the relevance between documents. When this stage is completed, further use Louvain algorithm to segment and cluster each document. After that, in the second stage, based on the segmentation result of the document, the event mention is classified, and then the variational graph autoencoder is used to predict the coreference relationship between event mentions.

[0054] Three core evaluation indicators are used in the experiment to measure the performance of the algorithm, which are precision (P), recall (R) and F1 value. These three indicators are widely used in natural language processing and information retrieval fields. Precision (P) reflects the proportion of positive samples in all samples predicted by the model. Its calculation formula is defined as: P = TP / (TP+FP), where TP represents true positive, and FP represents false positive. Recall (R) measures the proportion of positive samples correctly identified by the model among all true positive samples. Its calculation formula is defined as: R = TP / (TP+FN), where FN represents false negative. F1 value, as the harmonic mean of precision and recall, aims to consider the trade-off between the two. Its calculation formula is: F1 = (2*P*R) / (P+R). The experimental results on the data set are shown in Table 2.

[0055] Table 2

[0056]

[0057] To sum up, the method of the present application can utilize the coarse-grained annotation information ignored by the prior art, convert the coarse-grained annotation information into a beneficial supplement of the training data through hierarchical processing and hierarchical blending, and design a double-layer graph neural network to process coarse-grained information and fine-grained information respectively, so as to achieve the same or even better prediction level as the prior art with less training data. According to the method of the present application, under the guidance of the double-layer graph neural network, the guidance of event coreference can be obtained from coarse-grained information, and the representation of the coarse-grained graph neural network can be optimized from fine-grained information, thereby achieving the purpose of reducing the demand for labeled data. Moreover, the method of the present application abandons the pair-wise decision prediction mode in the prior art and uses edge prediction to determine event coreference, thereby greatly improving the prediction speed of the model and enabling it to be applied in large-scale data scenarios. The accuracy, recall rate, F1 value and other indicators of the present application on the MAVEN-ERE data set are significantly improved compared with the traditional method.

[0058] The embodiment of the present application also provides a storage medium for storing a computer program, which is executed to perform at least the method described above.

[0059] The embodiment of the present application also provides a control device, which includes a processor and a storage medium for storing a computer program; wherein the processor is used to execute the computer program to perform at least the method described above.

[0060] The embodiment of the present application also provides a processor, which executes a computer program to perform at least the method described above.

[0061] The storage medium can be implemented by any type of volatile or nonvolatile storage devices, or a combination thereof. Among them, the nonvolatile memory can be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Ferromagnetic Random Access Memory (FRAM), a Flash Memory, a magnetic surface memory, an optical disc, or a Compact Disc Read-Only Memory (CD-ROM). The magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM). The storage medium described in the embodiments of the present application is intended to include, but not limited to, these and any other suitable types of memory.

[0062] In several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other manners. The described device embodiments are merely illustrative, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling, or direct coupling or communication connection between the components can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0063] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0064] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0065] Those skilled in the art can understand that all or part of the steps of the above method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the above method embodiments when executed; and the foregoing storage medium includes mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and various media that can store program codes.

[0066] Alternatively, the above-mentioned integrated unit of the present application, if realized in the form of a software function module and sold or used as an independent product, can also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present application. The aforementioned storage medium includes: mobile storage devices, ROM, RAM, magnetic disks or optical disks, and various other media that can store program codes.

[0067] The methods disclosed in the several method embodiments of the present application can be combined arbitrarily without conflict to obtain new method embodiments.

[0068] The features disclosed in the several product embodiments of the present application can be combined arbitrarily without conflict to obtain new product embodiments.

[0069] The features disclosed in the several method or device embodiments of the present application can be combined arbitrarily without conflict to obtain new method or device embodiments.

[0070] The above is a further detailed description of the present application in combination with specific preferred embodiments, and the specific implementation of the present application cannot be limited to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, a number of equivalent substitutions or obvious modifications can be made, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present application.

Claims

1. A neural network-based event coreference resolution method, characterized in that, The method comprises the following steps: S1, obtaining coarse-grained annotation information and fine-grained annotation information, and processing the information in layers to be used for training high-layer and low-layer double-layer graph neural networks respectively; S2, establishing a high-layer network graph for modeling coarse-grained annotation information and guiding edge connection information of a low-layer network graph; In the high-layer network graph, node features are not completely covered by new features, but a part of original features is kept unchanged as anchor points, and another part of features is dynamically updated; the high-layer network graph takes a document as a node, and a relationship between documents as an edge, and an adjacency matrix representing the relationship between the documents is obtained; S3, establishing a low-layer network graph for modeling fine-grained annotation information and transmitting vector features to the high-layer network graph; the low-layer network graph periodically transmits vector features to the high-layer network graph, and the high-layer network graph samples a part of edges and transmits the edges to the low-layer network graph as a supplement to the edges of the low-layer network graph; the low-layer graph takes an event as a node, and a co-reference event in a document as an edge, and an adjacency matrix representing the relationship between the events is obtained; S4, performing multiple rounds of circulation, in each round of circulation, the high-layer network graph guides the edge connection of the low-layer network graph, and the low-layer network graph updates the node features of the high-layer network graph; the circulation improves the co-reference prediction process until an optimal result is reached; in the last circulation, the low-layer network graph gives all the co-reference predictions of the events participating in the calculation at one time.

2. The neural network-based event coreference resolution method of claim 1, wherein, The high-layer network graph is constructed using document-level information, and the low-layer network graph is constructed using event-level information.

3. The neural network-based event coreference resolution method of claim 2, wherein, In the preprocessing stage, given an input document and mentions therein, a pre-trained model is used to obtain character representations, and a high-layer network graph at the document level and a low-layer network graph at the event level are constructed; for the low-layer graph, initial node features thereof are obtained through maximum pooling; for the high-layer graph, average representations of the mentions in the document are used to initialize nodes thereof.

4. The neural network-based event coreference resolution method of claim 3, wherein, In the high-layer network graph, a variational graph autoencoder VGAE is used to construct a graph inference model; for a graph of an initial adjacency matrix, the connection relationship thereof will be continuously corrected in subsequent calculations, and the correction result is derived from the inner product of the transformed representations of the nodes, and the representations of the nodes are continuously updated in the circulation.

5. The neural network-based event coreference resolution method of claim 4, wherein, In the low-layer network graph, the same variational graph encoder is used to encode graph features, but two independent decoders are used for intra-document and cross-document tasks; in the circulation process, the update of the node input feature matrix of the low-layer network graph is generated spontaneously by the graph neural network of the low-layer network graph, and the change of the adjacency matrix is completed by the low-layer network graph and the high-layer network graph.

6. The neural network-based event coreference resolution method of claim 5, wherein, In the low-layer network graph, cross-document connection relationships are obtained by the encoder of the low-layer graph, and intra-document connections are obtained by enhanced representation inner products.

7. The neural network-based event coreference resolution method of any one of claims 3 to 6, wherein, The update of the node features of the low-layer network graph to the high-layer network graph comprises: before each training period of the high-layer network graph, an updated feature is spliced to the initial features of the high-layer network graph, and the update ensures that all mentions come from the same document, and the weight of cross-document mentions is forcibly set to 0.

8. The neural network-based event coreference resolution method of any one of claims 1 to 6, wherein, The edge connection information of the low-level network graph guided by the high-level network graph includes: after the adjacency matrix of the current high-level network graph is given, the first k1 most relevant document pairs are selected according to the numerical ordering of the adjacency matrix, and the important degree score of each document mentioned is obtained after the row of the weight matrix is normalized, thereby the maximum numerical index u in the column direction is selected, and T = u T The importance table T is calculated, wherein represents Hadamard product, and according to the importance table T, k2 most important mention pairs are extracted from the selected several documents, and the mention pairs are added as additional edges to the adjacency matrix of the low-level network graph.

9. The neural network-based event coreference resolution method of any one of claims 1 to 6, wherein, In the model optimization process, the high-layer network graph adopts binary cross-entropy BCE as a loss function; the low-layer network graph also uses binary cross-entropy as a loss function, but calculates intra-document loss and cross-document loss respectively, and adds them with different weights.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the event coreference resolution method according to any one of claims 1 to 9. The computer program, when executed by a processor, implements the event coreference resolution method according to any one of claims 1 to 9.

Images