Social Network Graph Querying via Static Fixed-Size Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex querying of social and online professional network data is time- and resource-intensive due to the exponential increase in the number of links to be scanned with the number of hops in the query, leading to significant computational resources and latency issues, especially when dealing with nodes having a large number of connections.
Innovation Solution
A system that processes social network data by maintaining a static representation of the graph on a single computer system, using fixed-size blocks and an index to efficiently retrieve data, and a dynamic representation in memory to track changes, reducing the need for multiple machines and minimizing latency through contiguous data storage and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex queries scan all links between individuals to find paths in the social network, then query completeness is improved, but the number of links to be scanned increases exponentially with the number of hops, causing time and resource consumption to worsen
Solution Approach 1:
The patent segments the social network graph into fixed-size blocks organized in a grid structure, where each block contains a subset of nodes and their connections. This segmentation allows queries to be processed in distributed fashion across multiple machines, with each machine handling a specific block, thereby reducing the exponential time complexity by dividing the search space into manageable segments that can be processed in parallel.
Solution Approach 2:
The patent introduces a spatial dimension by organizing graph blocks in a two-dimensional grid layout with coordinates. This dimensional transformation enables efficient routing and localization of data blocks across distributed machines, allowing the system to quickly navigate to relevant blocks without scanning the entire graph, thus reducing query execution time while maintaining completeness.
2Power
If the system uses a cluster-based architecture to distribute graph processing, then computational capacity is improved, but system complexity and deployment difficulty increase
Solution Approach 1:
The patent designs each worker machine to be a universal node capable of processing any graph block type (source blocks, intermediate blocks, destination blocks) and handling various query operations. This multi-functionality allows the system to scale by simply adding identical worker machines without increasing system complexity, as each machine can perform any required operation, thereby improving computational capacity while maintaining architectural simplicity.
Solution Approach 2:
The patent uses identical copies of the graph data distributed across multiple worker machines, where each machine holds a complete or partial copy of the graph blocks it needs to process. This copying approach enables parallel processing and improves computational capacity while keeping each node's software stack simple and uniform, reducing deployment complexity compared to specialized cluster architectures.
3Productivity
If the system stores graph data distributed across multiple machines, then processing parallelism is improved, but network communication overhead and latency increase
Solution Approach 1:
The patent pre-computes and caches intermediate results during the graph traversal process, storing them in local memory on worker machines. This preliminary action reduces the need for repeated network communications during query execution, as intermediate findings are already available locally, thereby improving throughput while reducing network overhead and energy consumption.
Solution Approach 2:
The patent introduces coordinator machines that act as intermediaries between client queries and worker machines. The coordinator receives queries, determines which worker machines need to be contacted, and manages the aggregation of results. This intermediary layer optimizes network communication by batching requests and reducing the total number of network hops, thereby improving throughput while minimizing communication overhead.
Data Source
AI summary
The disclosed embodiments provide a system for processing data. During operation, the system obtains a graph of a social network, wherein the graph includes a set of nodes representing users in the social network and a set of edges representing relationships between pairs of the users. Next, the system stores, on a single computer system, a static representation of the graph, wherein the static representation includes a first set of fixed-size blocks representing the nodes and the edges and a first index that maps a set of identifiers for the nodes and the edges to offsets of the first set of fixed-size blocks. The system then uses the static representation of the graph to process, by the single computer system, one or more queries of the graph.