DOLPHIN Policy Network Navigating Knowledge Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-hop methods for traversing knowledge graphs to infer therapeutic targets for diseases are limited by focusing on spurious paths, failing to explore multiple targets, and being biased towards pre-trained tensor factorization models, with memory constraints truncating the action space to highly connected neighbors.
Innovation Solution
The DOLPHIN system uses a trained policy network to navigate knowledge graphs, optimizing latent policies to rank targets and paths by minimizing entropic differences and considering all possible hops, thereby avoiding spurious paths and exploring a broader action space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If action dropout methods are used to mask links in the knowledge graph, then the agent is forced to explore multiple paths and avoid spurious paths, but the agent still becomes focused on spurious paths early in training
Solution Approach 1:
The system pre-computes and stores action masks in the replay buffer before training begins. These masks are generated based on the graph structure and are reused during training, eliminating the need for real-time mask computation and allowing the agent to immediately benefit from path diversity without delaying training
Solution Approach 2:
The system creates multiple copies of the same action mask for different training steps and stores them in the replay buffer. Instead of generating new masks continuously, the agent samples from these pre-computed mask copies, reducing computational overhead while maintaining the exploratory benefit of action masking
2Quantity of substance
If the action space is truncated to N neighbours due to memory constraints, then the algorithm can run within memory limits, but the agent is restricted to only consider the N most highly connected neighbours and biases towards travelling through highly connected entities
Solution Approach 1:
The action space is segmented into multiple parts: commonly used actions are stored in a compact format in the replay buffer, while less frequent actions are handled on-demand. This allows the system to store a large diverse action space without requiring all actions to be simultaneously accessible in memory during training
Solution Approach 2:
The system pre-computes and stores a diverse set of actions including less-connected neighbours in the replay buffer before training begins. This preliminary preparation ensures that the agent has access to a broader range of actions during training without requiring large memory allocations during the training loop itself
3Adaptability or versatility
If reward shaping using a pre-trained tensor factorisation model is used to incentivise out-of-benchmark targets, then the agent's ability to learn outside benchmark targets improves, but the model is biased towards making the same prediction as the pre-trained tensor factorisation model
Solution Approach 1:
The system introduces the replay buffer as an intermediary between the reward signal and the agent's learning process. The replay buffer stores diverse actions with their associated rewards, allowing the agent to learn from a broader distribution of outcomes rather than directly following the biased reward signal, thus reducing the bias toward tensor factorisation predictions
Solution Approach 2:
The reward signal is made dynamic by sampling from the replay buffer at different training steps rather than using a static biased reward. This allows the agent to experience varied rewards for different action types over time, encouraging exploration of diverse paths and targets rather than converging on the pre-trained model's preferred predictions
Data Source
AI summary
Embodiments of present disclosure provide a system, apparatus and method(s) for determining one or more target nodes and associated paths from a query of a graph structure. The method receives the query to the graph structure, where the query comprises a data representation of at least one query node. The method identifies one or more target nodes in response to the query based on a policy network, where the policy network is configured to determine the one or more target nodes in accordance with a latent policy distribution associated with the policy network. The method traverses the graph structure by a search in relation to the policy network, where the search is configured to navigate from the query node to the one or more identified target nodes to determine the associated paths. The method outputs a list of the one or more target nodes and the associated paths for the query, where the list are ranked in relation to the latent policy distribution.