Graph neutral networks with attention

Abstract

Methods and apparatus are provided for generating a graph neural network (GNN) model based on an entity-entity graph. The entity-entity graph comprising a plurality of entity nodes in which each entity node is connected to one or more entity nodes of the plurality of entity nodes by one or more corresponding relationship edges. The method comprising: generating an embedding based on data representative of the entity-entity graph for the GNN model, wherein the embedding comprises an attention weight assigned to each relationship edge of the entity-entity graph; and updating weights of the GNN model including the attention weights by minimising a loss function associated with at least the embedding; wherein the attention weights indicate the relevancy of each relationship edge between entitynodes of the entity-entity graph. The entity-entity graph may be filtered based on the attention weights of a trained GNN model. The filtered entity-entity graph may be used to update the GNN model or train another GNN model. The trained GNN model may be used to predict link relationship between a first entity and a second entity associated with the entity-entity graph.

Description

[0001] The present application relates to devices, systems and methods for graph neural networks (GNNs) and GNN models with attention. Background technique [0002] Many related reasoning problems with direct practical applications can be formulated as link prediction tasks exploiting prior graph-structured information. For example, predicting the winner in football, a possible therapeutic target for a disease, or reasoning about entities on Wikipedia can all be projected as link prediction tasks that leverage graph-structured data to inform predictions. A large-scale database of facts can be represented as an entity-entity knowledge graph, which is a graph structure describing the relationship between entity nodes, used to form the basis of prediction or increase the quality of prediction, and is represented by the relationship edge r The general form of the graph triple e=(s,r,o) associating the principal entity node s with the object entity node o of the knowledge graph pro...

AI Technical Summary

Problems solved by technology

The second type of method, non-spectral, defines convolution directly on the graph, but the problem is still to provide operations that are invariant to edge order and node degree

[0005] Although GCNNs are more effective than conventional methods, most conventional GCNNs are usually susceptible to noisy or incorrect graph-based datasets, which are more difficult than traditional neural network methods. Sets lead to performance degradation

Due to the sheer size of datasets derived from large-scale fact databases, this problem is exacerbated for deep learning that requires extremely large datasets that are difficult or prohibitively expensive to validate

While traditional approaches are capable of ingesting large-scale fact databases in a non-graph-structured manner, they are prohibitively expensive in terms of computational complexity

Images