Graph Neural Networks with Attention Weights for Noisy Data Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional graph convolutional neural networks (GCNNs) are susceptible to noisy or incorrect graph-based datasets, leading to worsened performance, especially in large-scale datasets where verifying data accuracy is difficult, and they struggle to ignore or remove noise from entity-entity knowledge graphs, affecting the robustness of link prediction models.
Innovation Solution
Introduce attention weights to each relationship edge in the entity-entity graph during training, modifying the GCNN technique to filter out noisy connections and improve model robustness by minimizing a modified loss function, allowing the model to learn the relevancy of each connection and suppress incorrect edges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional GCNN models are used to process large-scale graph datasets, then computational efficiency is improved, but model performance deteriorates due to inability to filter noisy or incorrect graph data
Solution Approach 1:
The patent extracts and removes noisy or incorrect edges from the graph data through attention mechanisms. The model learns to identify and suppress spurious connections, effectively separating useful information from harmful noise in the graph structure, which resolves the contradiction by maintaining computational efficiency while improving reliability through data filtering
Solution Approach 2:
The patent introduces attention weights as an intermediary mechanism between the graph data and the model's predictive output. These attention weights act as a mediator that modulates the influence of each edge, allowing the model to down-weight noisy connections and up-weight reliable ones, thus improving performance without sacrificing computational efficiency
2Speed
If conventional GCNN models process entity-entity knowledge graphs, then processing speed is improved, but the models cannot ignore or remove noise from less useful relationship edges
Solution Approach 1:
The patent extracts harmful noise from the graph by learning to identify and suppress incorrect edges. The attention mechanism enables the model to separate and discard spurious relationships, maintaining fast processing speeds while effectively removing harmful factors from the data
Solution Approach 2:
The patent converts the harmful noise in the graph data into a learning opportunity. By training the model to explicitly identify and suppress incorrect edges, the harmful noise becomes a signal that improves the model's ability to distinguish reliable from unreliable connections, thereby transforming a disadvantage into an advantage
3Measurement precision
If deep learning approaches are used for link prediction, then prediction accuracy is improved, but the computational cost increases due to requirement for extremely large-scale datasets that are difficult to verify
Solution Approach 1:
The patent extracts and filters high-quality reliable edges from large-scale graph datasets using attention mechanisms. By pre-filtering the data to remove noisy connections, the model can achieve high prediction accuracy on a smaller effective dataset, reducing the computational cost associated with processing and verifying extremely large-scale data
Data Source
Figure 1a
Figure 1b
Figure 1c
AI summary
Methods and apparatus are provided for generating a graph neural network (GNN) model based on an entity-entity graph. The entity-entity graph comprising a plurality of entity nodes in which each entity node is connected to one or more entity nodes of the plurality of entity nodes by one or more corresponding relationship edges. The method comprising: generating an embedding based on data representative of the entity-entity graph for the GNN model, wherein the embedding comprises an attention weight assigned to each relationship edge of the entity-entity graph; and updating weights of the GNN model including the attention weights by minimising a loss function associated with at least the embedding; wherein the attention weights indicate the relevancy of each relationship edge between entity nodes of the entity-entity graph. The entity-entity graph may be filtered based on the attention weights of a trained GNN model. The filtered entity-entity graph may be used to update the GNN model or train another GNN model. The trained GNN model may be used to predict link relationship between a first entity and a second entity associated with the entity-entity graph.