Dynamic Graph Sampling with Probabilistic Edge Decay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing models for analyzing dynamic interaction graphs struggle to balance recency and continuity, leading to processing and storage issues, as they either maintain all interactions regardless of age or abruptly remove older interactions, resulting in loss of important past relationships.
Innovation Solution
Generating multiple sample graphs from a current snapshot of a dynamic graph, using a probabilistic edge decay function to subsample edges, and incrementally maintaining these graphs to reduce storage consumption and execution time, while ensuring recency and continuity of analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the snapshot model is used to maintain all interactions regardless of age, then continuity of historical relationships is preserved, but storage consumption and processing time increase significantly
Solution Approach 1:
The patent segments the graph data into multiple snapshots taken at different time points. Instead of maintaining all interactions in a single unbounded graph, the system divides the historical data into discrete time-based segments (snapshots), allowing efficient storage and retrieval while preserving continuity through the sequence of snapshots.
Solution Approach 2:
The system dynamically adjusts the retention and processing of graph data based on time. Older snapshots can be archived or summarized, while recent snapshots are maintained in detail for active analysis. This dynamic approach allows the system to balance storage constraints with the need for historical continuity.
2Productivity
If the sliding window model is used to consider only recent interactions within a specified time window, then processing efficiency improves, but continuity of past relationships is lost
Solution Approach 1:
The system performs preliminary actions by pre-processing and archiving older snapshots before they fall out of the active analysis window. This includes computing aggregate statistics, summarizing relationship patterns, and storing these pre-computed results for quick retrieval, thus maintaining continuity without requiring full re-processing of historical data.
Solution Approach 2:
The system maintains continuous analysis across time by seamlessly transitioning between snapshots. As new data arrives, the system continuously updates the current snapshot while preserving access to previous snapshots, ensuring that the analysis trajectory remains continuous and historical relationships are not abruptly severed.
3Loss of information
If all interactions are maintained in the graph regardless of age, then complete historical analysis is possible, but the graph size becomes unbounded causing processing issues
Solution Approach 1:
The patent extracts and separates older interactions from the active graph by archiving them in historical snapshots. This extraction removes the burden of managing all historical data in the main graph structure, allowing the active graph to remain manageable in size while preserving access to complete historical information through the archived snapshots.
4Quantity of substance
If interactions outside the specified time window are removed in the sliding window model, then storage requirements are reduced, but important past relationships are lost
Solution Approach 1:
The system creates copies of historical graph states in the form of snapshots at different time points. Instead of deleting old interactions, the system maintains copies of the entire graph state at specific moments in time. This allows the system to reduce storage requirements in the active graph while preserving important past relationships in the snapshot archives.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach allows for efficient analysis by controlling the sampling probability based on edge age, ensuring recent and past interactions are preserved, reducing storage needs and computation time, and maintaining the integrity of dynamic graph insights.
Implementation Method 1
subsampling each of the edges of each of the sample graphs at a given time by applying a Bernoulli trial
Implementation Method 2
use a hash function to generate one or more integers corresponding to one or more of the edges in one or more of the sample graphs
Implementation Method 3
convert each of the one or more integers into a pseudorandom sample using a geometric distribution of a life span of the respective one or more of the edges
Implementation Method 4
sampling the edges of the current snapshot of the dynamic graph includes sampling the edges according to a decay function, such that sampling probability is non-increasing over time
Data Source
AI summary
A computer-implemented method, according to one embodiment, includes: generating two or more sample graphs by sampling edges of a current snapshot of a dynamic graph, generating two or more partial results by executing an algorithm on the two or more sample graphs, combining the partial results into a final result, and incrementally maintaining the sample graphs. Edges included in the current snapshot of a dynamic graph and which were added to the dynamic graph in a most recent update thereto are included in each of the generated two or more sample graphs. Moreover, incrementally maintaining the sample graphs includes: subsampling each of the edges of each of the sample graphs at a given time by applying a Bernoulli trial, and combining a result of the subsampling with new edges received in a batch corresponding to the given time to form new sample graphs.