An event representation learning technique based on graph isomorphism network

Abstract

The application discloses an event representation learning technology based on a graph isomorphism network, relates to the technical field of event knowledge representation, and has the technical points that: the application constructs an event representation learning framework which can effectively utilize all argument information and perform distributed coding on events by accepting a variable number of arguments as input; the graph isomorphism network in the application accepts an arbitrary number of event arguments as input, effectively utilizes all argument information, and performs representation learning according to the relationship between the arguments to provide a representation vector containing complete event information for a downstream task.

Description

Technical Field

[0001] This invention relates to the field of event knowledge representation technology, and more specifically to an event representation learning technique based on graph isomorphic networks. Background Technology

[0002] With the widespread use of computers, the internet has generated massive amounts of text data, containing rich event information. Event extraction technology employs a discrete structured record representation method, using trigger words that clearly express the occurrence of an event and arguments that play a certain role in the event to represent it. Based on event templates, structured event information is extracted from natural language text. Event representation learning technology builds upon this by using distributed embedding coding to transform textual event information into event representation vectors, thereby reducing the difficulty for downstream tasks to utilize event information.

[0003] Existing event representation learning techniques typically employ models such as tensor neural networks, using trigger word semantics to model the relationship between subject and object arguments. However, real-world events usually contain multiple arguments, and the relationships between arguments are more complex. Current research neglects arguments other than subject and object, failing to fully utilize event information. The tensor neural network models used perform pairwise linear transformations between input elements, obtaining the event representation vector through multiple transformations. Their transformation matrix has a large number of parameters, making convergence difficult during training. Furthermore, they can only accept a fixed number of arguments as input, failing to handle situations where the number of arguments varies in real-world events.

[0004] Therefore, the present invention aims to provide an event representation learning technique based on graph isomorphic networks to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to solve the above-mentioned problems and provide an event representation learning technique based on graph isomorphic networks. The graph isomorphic network in the method of this invention accepts any number of event arguments as input, effectively utilizes all argument information and performs representation learning based on the relationship between arguments, and provides a representation vector containing complete event information for downstream tasks.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] This invention provides an event representation learning technique based on graph isomorphic networks. It uses the Word2Vector algorithm to convert event arguments into word vectors, trains the vectors in a graph isomorphic network, and finally uses pooling to generate event representation vectors. Specifically, it includes the following steps:

[0008] S1. Word vector model training: Use the Word2Vector algorithm to convert event arguments into corresponding word vectors;

[0009] S2. Construction of Graph Isomorphic Network: Based on S1, the word vectors of event arguments are used as feature vectors of nodes in the graph. According to the relationship between arguments, corresponding edges are added to the graph to construct a graph isomorphic network.

[0010] S3. Training and output of graph isomorphic networks: Based on S2, the feature vectors of nodes are iteratively updated using message passing, and the event representation vectors are generated after pooling operations.

[0011] Furthermore, the specific method of S1 is as follows: the text describing the event is segmented into words; the event-related words are trained using the Word2Vector algorithm; and the event arguments are converted into word vectors using the Skip-grams network training parameters.

[0012] Furthermore, the specific method of S2 is as follows: based on the event template, for each event instance in the training set and the test set, the arguments contained in the event are used as nodes, and connections are established between the argument nodes with related relationships to form a graph-structured event representation; according to S1, the word vectors of the argument nodes are used as feature vectors of the nodes in the graph, and corresponding edges are added to the graph according to the connection relationship of the nodes to construct a graph isomorphic network.

[0013] Furthermore, the specific method of S3 is as follows: based on S2, the constructed graph isomorphic network is trained by using message passing, and the feature vectors of the nodes are iteratively updated; second-order pooling and bilinear transformation are performed on the feature vectors of all nodes to output event representation vectors.

[0014] Compared with existing technologies, the beneficial effects of this solution are:

[0015] 1. The technical solution of the present invention introduces a graph isomorphic network that supports any number of arguments and can express the relationship between arguments for representation learning, which effectively utilizes the information of all arguments in the event and avoids information loss;

[0016] 2. The technical solution of the present invention is based on a graph isomorphic network model using message passing. The number of model parameters increases quadratically with the increase of input elements. It requires less training resources, has a faster convergence speed, and can achieve better training results in application scenarios with a large number of arguments.

[0017] 3. Through the technical solution of the present invention, an algorithm model for converting event text into event representation vectors can be built during the application process, which is convenient for downstream tasks to call directly and can greatly reduce the difficulty of processing and utilizing event information. Attached Figure Description

[0018] Figure 1 This is a flowchart of the event representation learning method in an embodiment of the present invention;

[0019] Figure 2 This is a Word2Vector model diagram in an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be described in further detail below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the embodiments.

[0022] Example:

[0023] The event arguments are transformed into word vectors using the Word2Vector algorithm, trained in a graph isomorphic network, and finally pooled to generate the event representation vectors. The specific steps include:

[0024] S1. Word vector model training: Use the Word2Vector algorithm to convert event arguments into corresponding word vectors;

[0025] S2. Construction of Graph Isomorphic Network: Based on S1, the word vectors of event arguments are used as feature vectors of nodes in the graph. According to the relationship between arguments, corresponding edges are added to the graph to construct a graph isomorphic network.

[0026] S3. Training and output of graph isomorphic networks: Based on S2, the feature vectors of nodes are iteratively updated using message passing, and the event representation vectors are generated through pooling operations.

[0027] Furthermore, the specific method of S1 is as follows: the text describing the event is segmented into words; the event-related words are trained using the Word2Vector algorithm; and the event arguments are converted into word vectors using the Skip-grams network training parameters.

[0028] Furthermore, the specific method of S2 is as follows: based on the event template, for each event instance in the training set and the test set, the arguments contained in the event are used as nodes, and connections are established between the argument nodes with related relationships to form a graph-structured event representation; according to S1, the word vectors of the argument nodes are used as feature vectors of the nodes in the graph, and corresponding edges are added to the graph according to the connection relationship of the nodes to construct a graph isomorphic network.

[0029] Furthermore, the specific method of S3 is as follows: based on S2, the constructed graph isomorphic network is trained by using message passing, and the feature vectors of the nodes are iteratively updated; second-order pooling and bilinear transformation are performed on the feature vectors of all nodes to output event representation vectors.

[0030] The algorithm principle of this invention is as follows:

[0031] By using the Word2Vec model and a graph isomorphic network, event arguments in text form are transformed into word vectors. Based on the relationships between arguments, the graph isomorphic network updates the argument vectors, and finally, pooling outputs the representation vector of the entire event. This model supports any number of argument inputs and can effectively utilize the information of all arguments in the event, thus producing a more accurate event representation vector.

[0032] The Word2Vec model in this invention:

[0033] The Word2Vec word vector model is an unsupervised text corpus training model, and its basic structure diagram is as follows: Figure 2 As shown, it is a distributed representation of words, using a fixed-length vector to represent each word, which also contains the semantic information of the word. In the input layer of the Word2Vec model, the input is a one-hot vector x of the word. After passing through the shared weight matrix W... V×N The hidden layer vector h is obtained, that is:

[0034] h = x·Wv×n

[0035] Then it goes through the weight matrix W' V×N The prediction of the output word is obtained, that is:

[0036] y = h·W′v×n

[0037] In this embodiment of the invention, the focus is not on predicting the context of words, but rather on the byproducts generated during the training process of this task—the model parameters. Each vector in the model parameters corresponds to a word, and these parameters are used as word vector representations of the words.

[0038] Graph isomorphic networks take graph structure data as input and learn the representation vectors of nodes and the representation vector h of the entire graph based on the topology and node characteristics. G Graph isomorphic networks employ a neighborhood aggregation strategy, which iteratively updates the representation of the central node by aggregating the representations of neighboring nodes. After k rounds of aggregation iterations, the representation vector of a node contains the features of its neighboring nodes within its k-hop range and the topological structure information of the graph.

[0039] Graph isomorphic networks recursively update the feature vectors of each node to learn the network structure and features. The message-passing function is the core of this process, operating on a multiset consisting of the center node and its neighbors. Its output is independent of the order of elements in the set, depending only on the type and number of elements. The message-passing method in graph isomorphic networks is similar to the first-order Weisfeiler-Lehman test; only when two nodes have the same features and the same neighbor structure are their features updated to the same vector. This means that the multiset function used is injective.

[0040] In this embodiment of the invention, word vectors are used as node labels, and the injective message passing function employed is as follows:

[0041]

[0042] Where σ(·) is an activation function such as sigmoid or tanh, and A, η, and b are the trainable parameters of the model.

[0043] The second-order pooling method in this invention:

[0044] The present invention places two main requirements on the pooling output method. First, different graphs have different numbers of nodes, while downstream machine learning models such as classifiers typically require a fixed-size graph representation vector as input. This necessitates that graph pooling have the ability to transform a variable number of node representations into a fixed-size graph representation. Second, there is no inherent ordering relationship between nodes in the graph structure. The pooling operation requires input from any order of these nodes to produce the same graph representation vector. The second-order pooling method can satisfy the above conditions and produce a graph representation vector that effectively utilizes all node information.

[0045] Suppose that the node features output after learning by the GIN network are represented as follows:

[0046] H = GIN(A,X) = [h1,...,h n ] T ∈R n×m

[0047] Writing a second-order pooling method:

[0048] SOPOOL(H) = H T H∈R n×m

[0049] However, compared to the dimension m of the node representation vector, the number of nodes n is usually small. Directly using the second-order pooling method will result in the number of parameters in the downstream classifier reaching m. 2The magnitude of the matrix is ​​much larger than the m×n output of GIN, making it more difficult for the classifier to converge during training and prone to overfitting to the training dataset. To address this issue, a bilinear mapping method is used to reduce the matrix dimension before the pooling operation:

[0050] h G =SOPOOL(HW)=W T H2 T H2W∈R m′×m

[0051] Where m′<m, W∈R m′×m It is a trainable parameter matrix representing a bilinear mapping operation on the matrix. The additional training parameters introduced are m′×m, but the number of parameters in the downstream classifier is reduced to m′. 2 By setting the value of m′ appropriately, the pooling effect can be maintained while reducing the number of parameters.

[0052] The implementation method of the technical solution of the present invention:

[0053] 1. Word vector model training phase

[0054] First, the text describing the event is segmented into words. Then, the Word2Vector algorithm is used to train a word vector model on the segmented event-related words. The Skip-grams network training parameters are then used to transform the event arguments into word vectors.

[0055] 2. Graph Isomorphic Network Construction Stage

[0056] First, based on the event template, for each event instance in the training and test sets, the arguments contained in the event are used as nodes, and connections are established between the argument nodes with related relationships to form a graph-structured event representation. Then, the word vectors of the argument nodes are used as feature vectors of the nodes in the graph, and corresponding edges are added to the graph according to the connection relationship of the nodes to construct a graph isomorphic network.

[0057] 3. Training and Output Stages of Graph Isomorphic Networks

[0058] The constructed graph isomorphic network is trained using a message-passing method, the feature vectors of the nodes are iteratively updated, and then second-order pooling and bilinear transformation are performed on the feature vectors of all nodes to output event representation vectors.

[0059] 4. Test Results

[0060] Currently, the advanced basic framework models in the field of event representation learning mainly fall into the following two categories:

[0061] (1) Compositional Neural Network (CN) model: Use discrete tuples to represent an event, concatenate the word vectors of the trigger word and the argument, and then pass them through a fully connected layer to obtain the representation of the event.

[0062] (2) Tensor Neural Network (NTN) model: It uses pre-trained word vectors as representations of event elements and learns the interaction information between event elements through bilinear tensor operations.

[0063] This invention uses the DuEE-Fin dataset and employs an event classification task as the supervised training objective. The experimental results for the test set are shown in the table below:

[0064] Table 1. Comparative Experiment Results (Unit: %)

[0065]

[0066] Through the above embodiments of the present invention, the technical solution of the present invention introduces a graph isomorphic network that supports an arbitrary number of arguments and can express the relationships between arguments for representation learning, effectively utilizing the information of all arguments in the event during the generation of event representation vectors. Furthermore, an algorithmic model for converting event text into event representation vectors can be built during application, facilitating direct invocation by downstream tasks and greatly reducing the difficulty of processing and utilizing event information.

[0067] The above specific embodiments are merely explanations of the present invention and are not intended to limit the present invention. After reading this specification, those skilled in the art can make modifications to these embodiments without contributing any inventive step, but as long as they are within the scope of the claims of the present invention, they are protected by patent law.

Claims

1. An event representation learning technique based on graph isomorphic networks, characterized by: The event arguments are transformed into word vectors using the Word2Vector algorithm, trained in a graph isomorphic network, and finally pooled to generate the event representation vectors. The specific steps include: S1. Word vector model training: Use the Word2Vector algorithm to convert event arguments into corresponding word vectors; S2. Construction of Graph Isomorphic Network: Based on S1, the word vectors of event arguments are used as feature vectors of nodes in the graph. According to the relationship between arguments, corresponding edges are added to the graph to construct a graph isomorphic network. S3. Training and output of graph isomorphic networks: Based on S2, the feature vectors of nodes are iteratively updated using message passing, and the event representation vectors are generated after pooling operations.

2. The event representation learning technique based on graph isomorphic networks as described in claim 1, characterized in that: The specific method of S1 is as follows: segment the text describing the event into words; train a word vector model on the event-related words after segmentation using the Word2Vector algorithm; and use the Skip-grams network to train parameters to convert event arguments into word vectors.

3. The event representation learning technique based on graph isomorphic networks as described in claim 1, characterized in that: The specific method of S2 is as follows: Based on the event template, for each event instance in the training set and the test set, the arguments contained in the event are used as nodes, and connections are established between the argument nodes with related relationships to form a graph-structured event representation; according to S1, the word vectors of the argument nodes are used as feature vectors of the nodes in the graph, and corresponding edges are added to the graph according to the connection relationship of the nodes to construct a graph isomorphic network.

4. The event representation learning technique based on graph isomorphic networks as described in claim 1, characterized in that: The specific method of S3 is as follows: Based on S2, the constructed graph isomorphic network is trained by using message passing, and the feature vectors of the nodes are iteratively updated; second-order pooling and bilinear transformation are performed on the feature vectors of all nodes to output event representation vectors.

Images