Deep-Graph-Sprints for Low-Latency Dynamic Graph Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for handling Continuous Time Dynamic Graphs (CTDGs) face computational and memory constraints, limiting their practicality for real-time applications, and struggle to capture long-term dependencies effectively.
Innovation Solution
A novel architecture, Deep-Graph-Sprints (DGS), employs forward-mode automatic differentiation (RTRL) within a tailored recurrent cell structure to efficiently compute and update node embeddings in real-time, using vectorized forgetting and proximity coefficients to capture long-term dependencies while minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If reverse-mode backpropagation is used for training recurrent models on CTDGs, then gradient computation is efficient, but memory complexity increases to O(l×d2) making it infeasible for long sequences
Solution Approach 1:
The patent segments the backpropagation process into two distinct phases: a forward pass that computes node states and stores only essential intermediate values, and a backward pass that computes gradients using stored summaries rather than complete intermediate states. This segmentation reduces memory complexity from O(l×d2) to O(d2) while maintaining computational efficiency.
Solution Approach 2:
The patent extracts and stores only the essential intermediate representations needed for gradient computation during the forward pass, rather than storing all intermediate states. This extraction approach retains the computational efficiency of full backpropagation while dramatically reducing memory requirements by eliminating redundant storage.
2Quantity of substance
If Truncated Backpropagation Through Time (TBPTT) is used to reduce memory and computational demands, then resource usage is optimized, but the ability to capture long-term dependencies is restricted
Solution Approach 1:
The patent performs preliminary action by computing and storing aggregated summary statistics of intermediate states during the forward pass. These pre-computed summaries contain condensed information about long-term patterns, enabling the backward pass to capture long-term dependencies without storing complete intermediate states, thus avoiding the truncation limitation.
Solution Approach 2:
The patent changes the representation parameters from storing complete intermediate states (high memory cost) to storing compressed summary statistics (low memory cost). This parameter transformation maintains the essential information needed for long-term dependency capture while reducing memory usage to feasible levels.
3Quantity of substance
If Forward-mode AD (RTRL) is used to facilitate online updates and capture long-term dependencies, then memory overhead is reduced to O(d2), but computational demand increases to O(l×d4)
Solution Approach 1:
The patent inverts the traditional approach by performing the computationally intensive operations during the forward pass when data is available, and using the stored summaries during the backward pass. This inversion achieves RTRL-like memory efficiency (O(d2)) while avoiding its computational burden (O(l×d4)) by strategically distributing computational work.
4Speed
If asynchronous operation is employed to mitigate computational constraints, then real-time processing is enabled, but representation accuracy is compromised
Solution Approach 1:
The patent enables continuous real-time processing by computing node embeddings incrementally as new edges arrive, rather than waiting for complete graph snapshots. The continuous backpropagation mechanism maintains representation accuracy by continuously updating embeddings with new information, achieving both real-time processing capability and accurate representations.
Data Source
AI summary
Computer-implemented method and computer device for low-latency dynamic embedding generation for transactional graphs related to providing a task specific score regarding a time-dependent graph, preferably a Continuous Time Dynamic Graph (CTGD), for classifying data records from a real-time transactional data record stream. In particular, a memory-constrained computer-implemented method and computer device for low-latency dynamic embedding generation for a transactional graph from a real-time transaction stream having transaction data records, wherein each transaction corresponds to an edge, for obtaining a feature embedding for at least a portion of the plurality of nodes, by using a computer system for recurrently updating a state array corresponding to a feature embedding of each node of the at least a portion of the plurality of nodes.