Event representation learning technology based on graph isomorphic network

Abstract

The invention discloses an event representation learning technology based on a graph isomorphic network, and relates to the technical field of event knowledge representation, and the technical key points are as follows: an event representation learning framework which accepts variable number of arguments as input, can effectively utilize all argument information, and carries out distributed coding on events is constructed; the graph isomorphic network receives any number of event arguments as input, all argument information is effectively utilized, representation learning is carried out according to the relation between the arguments, and representation vectors containing complete event information are provided for downstream tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of event knowledge representation, and in particular to an event representation learning technology based on graph isomorphic networks. Background Art

[0002] With the popularization of computers, a huge amount of text data has been generated on the Internet, which contains rich event information. Event extraction technology uses a discrete structured record representation method to represent events with trigger words that can most clearly express the occurrence of events and arguments that play a certain role in the events, and extracts structured event information from natural language texts based on event templates. On this basis, event representation learning technology uses distributed embedding coding to convert event information in text form into event representation vectors, thereby reducing the difficulty of downstream tasks using event information.

[0003] Existing event representation learning technologies usually use models such as tensor neural networks, and use trigger word semantics to model the relationship between subject arguments and object arguments; however, real events usually contain multiple arguments, and the relationship between arguments is more complex. Existing research ignores other arguments besides subject and object, and cannot fully utilize event information; the tensor neural network model used performs linear transformations between input elements in pairs, and obtains the event representation vector through multiple transformations. The transformation matrix has a large number of parameters, which is difficult to converge during training, and can only accept a fixed number of arguments as input, which cannot cope with the situation where the number of arguments changes in actual events.

[0004] To this end, the present invention aims to provide an event representation learning technology based on graph isomorphic networks to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to solve the above problems and provide an event representation learning technology based on graph isomorphism network. The graph isomorphism network in the method of the present invention accepts any number of event arguments as input, effectively utilizes all argument information and performs representation learning based on the relationship between arguments, providing downstream tasks with representation vectors containing complete event information.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] The present invention provides an event representation learning technology based on a graph isomorphic network. The event arguments are converted into word vectors by using the Word2Vector algorithm, and training is performed in the graph isomorphic network. Finally, the event representation vector is generated by pooling. Specifically, the following steps are included:

[0008] S1. Word vector model training: Use the Word2Vector algorithm to convert event arguments into corresponding word vectors;

[0009] S2, graph isomorphism network construction: According to S1, the word vectors of event arguments are used as feature vectors of nodes in the graph, and corresponding edges are added to the graph according to the relationship between arguments to construct a graph isomorphism network;

[0010] S3, graph isomorphic network training and output: According to S2, the feature vector of the node is iteratively updated by message passing, and the representation vector of the event is generated through pooling operation.

[0011] Furthermore, the specific method of S1 is: to perform word segmentation on the text describing the event; to train the word vector model for the event-related vocabulary after word segmentation through the Word2Vector algorithm; and to convert the event arguments into word vectors using the Skip-grams network training parameters.

[0012] Furthermore, the specific method of S2 is: based on the event template, for each event instance in the training set and the test set, the arguments contained in the event are used as nodes, and connections are established between the argument nodes with associated relationships to form a graph-structured event representation; according to S1, the word vectors of the argument nodes are used as the feature vectors of the nodes in the graph, and corresponding edges are added to the graph according to the connection relationship of the nodes to construct a graph isomorphism network.

[0013] Furthermore, the specific method of S3 is: according to S2, the constructed graph isomorphic network is trained by message passing, and the feature vectors of the nodes are updated iteratively; the feature vectors of all nodes are subjected to second-order pooling and bilinear transformation, and the event representation vector is output.

[0014] Compared with the prior art, this solution has the following beneficial effects:

[0015] 1. The technical solution of the present invention introduces a graph isomorphic network that supports any number of arguments and can express the relationship between arguments to perform representation learning, which effectively utilizes the information of all arguments in the event and avoids information loss;

[0016] 2. The graph isomorphic network model based on the message passing method in the technical solution of the present invention has a model parameter quantity that increases quadratically with the increase of input elements, requires fewer resources for training, has a faster convergence speed, and can achieve better training effects in application scenarios with a large number of arguments;

[0017] 3. Through the technical solution of the present invention, an algorithm model for converting event text into event representation vectors can be built during the application process, which is convenient for direct calling of downstream tasks and can greatly reduce the difficulty of processing and utilizing event information. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of an event representation learning method in an embodiment of the present invention;

[0019] Figure 2 It is a Word2Vector model diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below in conjunction with the embodiments.

[0022] Example:

[0023] The event arguments are converted into word vectors using the Word2Vector algorithm, trained in a graph isomorphism network, and finally pooled to generate event representation vectors. The specific steps include:

[0024] S1. Word vector model training: Use the Word2Vector algorithm to convert event arguments into corresponding word vectors;

[0025] S2, graph isomorphism network construction: According to S1, the word vectors of event arguments are used as feature vectors of nodes in the graph, and corresponding edges are added to the graph according to the relationship between arguments to construct a graph isomorphism network;

[0026] S3, graph isomorphic network training and output: According to S2, the feature vector of the node is iteratively updated by message passing, and the representation vector of the event is generated through pooling operation.

[0027] Furthermore, the specific method of S1 is: to perform word segmentation on the text describing the event; to train the word vector model for the event-related vocabulary after word segmentation through the Word2Vector algorithm; and to convert the event arguments into word vectors using the Skip-grams network training parameters.

[0028] Furthermore, the specific method of S2 is: based on the event template, for each event instance in the training set and the test set, the arguments contained in the event are used as nodes, and connections are established between the argument nodes with associated relationships to form a graph-structured event representation; according to S1, the word vectors of the argument nodes are used as the feature vectors of the nodes in the graph, and corresponding edges are added to the graph according to the connection relationship of the nodes to construct a graph isomorphism network.

[0029] Furthermore, the specific method of S3 is: according to S2, the constructed graph isomorphic network is trained by message passing, and the feature vectors of the nodes are updated iteratively; the feature vectors of all nodes are subjected to second-order pooling and bilinear transformation, and the event representation vector is output.

[0030] The algorithm principle of the present invention is:

[0031] By using the Word2Vec model and graph isomorphism network, the event arguments in text form are converted into word vectors. According to the correlation between the arguments, the graph isomorphism network is used to update the argument vectors, and finally the representation vector of the entire event is output through pooling. The model supports the input of any number of arguments and can effectively utilize the information of all arguments in the event, thereby generating a more accurate event representation vector.

[0032] The Word2Vec model in the present invention:

[0033] The Word2Vec word vector model is an unsupervised text corpus training model. Its basic structure is shown in the figure below. Figure 2 As shown in Figure 1, it is a distributed representation of words, using a fixed-length vector to represent words. This vector also contains the semantic information of the words. In the input layer of the Word2Vec model, the input received is the one-hot vector x of the word. After the shared weight matrix W V×N Get the hidden layer vector h, that is:

[0034] h=x·Wv×n

[0035] Then through the weight matrix W' V×N Get the prediction of the output word, that is:

[0036] y=h·W′v×n

[0037] In the embodiment of the present invention, the focus is not on the prediction of the word context, but on the byproduct generated during the training process of this task, namely, the model parameters. In the model parameters, each vector corresponds to a word, and these parameters are used as the word vector representation of the word.

[0038] The graph isomorphism network takes graph structure data as input and learns the representation vector of the node and the representation vector h of the entire graph based on the topological structure and node features. G The graph isomorphic network adopts a neighborhood aggregation strategy, that is, iteratively updates the representation of the central node by aggregating the representations of neighboring nodes. After k rounds of aggregation iterations, the representation vector of the node contains the characteristics of its neighboring nodes within the k-hop range and the topological structure information of the graph.

[0039] The graph isomorphism network recursively updates the feature vector of each node to learn the network structure and features. The message passing function is the core of this process, acting on the multiset composed of the central node and the neighboring nodes. Its output is independent of the order of the elements in the set, but only related to the type and number of elements. The message passing method in the graph isomorphism network is similar to the first-order Weisfeiler-Lehman test. Only when two nodes have the same features and the same adjacent node structure, the features of the two nodes are updated to the same vector, which means that the multiset function used is injective.

[0040] In this embodiment of the present invention, word vectors are used as node labels, and the injective message passing function used is:

[0041]

[0042] Among them, σ(·) is an activation function such as sigmoid and tanh, and A, η, and b are trainable parameters of the model.

[0043] The second-order pooling method in the present invention:

[0044] The scheme of the present invention has two main requirements for the pooling output method. First, the number of nodes in different graphs is different, and downstream machine learning models such as classifiers usually require a fixed-size graph representation vector as input, which requires graph pooling to have the ability to convert a variable number of node representations into a fixed-size graph representation. Secondly, there is no inherent ordering relationship between the nodes in the graph structure, and the pooling operation requires the input of any node order to produce the same graph representation vector. The second-order pooling method can meet the above conditions and produce a graph representation vector that effectively utilizes all node information.

[0045] Assume that the node feature output after GIN network learning is represented as:

[0046] H=GIN(A,X)=[h1,...,h n ] T ∈R n×m

[0047] The second-order pooling method is written as:

[0048] SOPOOL(H)=H T H∈R n×m

[0049] However, compared with the node representation vector dimension m, the number of nodes n is usually small, and directly using the second-order pooling method will result in the number of parameters of the downstream classifier reaching m. 2The magnitude is much larger than the m×n magnitude of the GIN output, which will make the classifier training process more difficult to converge and easy to overfit to the training data set. To solve this problem, the bilinear mapping method is used to reduce the matrix dimension before the pooling operation:

[0050] h G =SOPOOL(HW)=W T H2 T H2W∈R m′×m

[0051] Where m′<m, W∈R m′×m is a trainable parameter matrix representing the bilinear mapping operation on the matrix. The additional training parameters introduced are m′×m, but the number of parameters of the downstream classifier is reduced to m′ 2 By properly setting the value of m′, the pooling effect can be maintained while reducing the number of parameters.

[0052] The implementation method of the technical solution of the present invention is:

[0053] 1. Word vector model training stage

[0054] First, the text describing the event is segmented, and then the word vector model is trained for the event-related vocabulary after segmentation through the Word2Vector algorithm. The Skip-grams network training parameters obtained through training are used to convert the event arguments into word vectors.

[0055] 2. Graph Isomorphic Network Construction Phase

[0056] Firstly, based on the event template, for each event instance in the training set and the test set, the arguments contained in the event are taken as nodes, and connections are established between the argument nodes with associated relationships to form an event representation with a graph structure; then the word vectors of the argument nodes are taken as the feature vectors of the nodes in the graph, and corresponding edges are added to the graph according to the connection relationship of the nodes to construct a graph isomorphism network.

[0057] 3. Graph Isomorphic Network Training and Output Phase

[0058] The constructed graph isomorphic network is trained by message passing, the feature vectors of the nodes are iteratively updated, and then the feature vectors of all nodes are subjected to second-order pooling and bilinear transformation to output the event representation vector.

[0059] 4. Test results

[0060] At present, there are two main types of advanced basic framework models in the field of event representation learning:

[0061] (1) Compositional Neural Network (CN) model: Use discrete tuples to represent an event, concatenate the word vectors of the trigger word and the argument, and then pass them through a fully connected layer to obtain the event representation.

[0062] (2) Neural Tensor Network (NTN) model: uses pre-trained word vectors as the representation of event elements and learns the interaction information between event elements through bilinear tensor operations.

[0063] The embodiment of the present invention uses the DuEE-Fin dataset and adopts the event classification task as the supervised training target. The experimental results of the test set are shown in the following table:

[0064] Table 1 Comparative experimental results (unit: %)

[0065]

[0066] Through the above embodiments of the present invention, the technical solution of the present invention introduces a graph isomorphic network that supports any number of arguments and can express the relationship between arguments to perform representation learning, and effectively utilizes the information of all arguments in the event in the process of generating event representation vectors. In addition, an algorithm model that converts event text into event representation vectors can be built during the application process, which is convenient for downstream tasks to directly call, and can greatly reduce the difficulty of processing and utilizing event information.

[0067] The above specific embodiments are merely explanations of the present invention and are not limitations of the present invention. After reading this specification, those skilled in the art may make modifications to the embodiments without any creative contribution as needed. However, such modifications are protected by the patent law as long as they are within the scope of the claims of the present invention.

Claims

1. An event representation learning technology based on graph isomorphic networks, characterized by: The event arguments are converted into word vectors using the Word2Vector algorithm, trained in a graph isomorphism network, and finally pooled to generate event representation vectors. The specific steps include: S1. Word vector model training: Use the Word2Vector algorithm to convert event arguments into corresponding word vectors; S2, graph isomorphism network construction: According to S1, the word vectors of event arguments are used as feature vectors of nodes in the graph, and corresponding edges are added to the graph according to the relationship between arguments to construct a graph isomorphism network; S3, graph isomorphic network training and output: According to S2, the feature vector of the node is iteratively updated by message passing, and the representation vector of the event is generated through pooling operation.

2. The event representation learning technology based on graph isomorphic network as claimed in claim 1, characterized in that: The specific method of S1 is: to perform word segmentation on the text describing the event; to train the word vector model for the event-related vocabulary after word segmentation through the Word2Vector algorithm; and to convert the event arguments into word vectors using the Skip-grams network training parameters.

3. The event representation learning technology based on graph isomorphic network as claimed in claim 1, characterized in that: The specific method of S2 is as follows: based on the event template, for each event instance in the training set and the test set, the arguments contained in the event are used as nodes, and connections are established between argument nodes with associated relationships to form an event representation with a graph structure; according to S1, the word vectors of the argument nodes are used as the feature vectors of the nodes in the graph, and corresponding edges are added to the graph according to the connection relationship of the nodes to construct a graph isomorphism network.

4. The event representation learning technology based on graph isomorphic network as claimed in claim 1, characterized in that: The specific method of S3 is as follows: according to S2, the constructed graph isomorphic network is trained by message passing, and the feature vectors of the nodes are updated iteratively; the feature vectors of all nodes are subjected to second-order pooling and bilinear transformation, and the event representation vector is output.

Images