Persistent Message Passing for Graph Neural Network State Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional graph neural networks (GNNs) struggle to perform well on tasks requiring efficient data structure queries dependent on previous states and connectivity, as they lack the ability to track the evolution of data structures over time.
Innovation Solution
Implement a persistent message passing (PMP) mechanism that endows GNNs with the capability to query past states by persisting node representations, creating new nodes (hidden states) instead of overwriting existing ones, applicable to both static and dynamic graphs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional GNNs overwrite existing hidden states with new states at each time step, then computational efficiency is improved, but the ability to query past states and track data structure evolution is lost
Solution Approach 1:
The patent creates persistent copies of hidden states at each time step instead of overwriting them. Each time step t maintains its own copy of node representations h_v^(t), which are preserved and can be queried later. This copying mechanism allows the system to retain historical state information while继续进行 computations at new time steps.
Solution Approach 2:
The patent adds a time dimension to the hidden state representations by introducing time step indices. Instead of a single hidden state h_v, the system maintains h_v^(t) where t denotes the time dimension. This dimensional extension allows simultaneous access to multiple temporal states without interference.
2Loss of information
If persistent copies of hidden states are maintained for all time steps, then the ability to query past states is improved, but memory consumption increases
Solution Approach 1:
The patent applies local quality by making persistence selective rather than universal. The system uses attention mechanisms to determine which hidden states from previous time steps are relevant and should be persisted. Not all historical states are maintained with equal quality - only those deemed important by the attention mechanism receive persistent storage.
Solution Approach 2:
The patent dynamically adjusts the persistence parameter φ_v^(t) for each node v at time step t. This parameter controls whether a hidden state should be persisted based on learned importance scores. By changing this parameter adaptively, the system optimizes the balance between retaining necessary historical information and managing memory resources efficiently.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing persistent message passing using graph neural networks.