Social Graph Data Ordering for Cache Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional social networking services face inefficiencies in data access due to the random or sequential storage of social graph data across data shards, leading to ineffective caching as related user data is not stored together, resulting in suboptimal memory and CPU cache performance.
Innovation Solution
The social graph reordering system reassigns user IDs based on social similarity, storing data in a way that socially related users have numerically close IDs, allowing their data to be accessed together, thereby improving cache efficiency and reducing storage space requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If social graph data is stored randomly or sequentially by user account creation order, then data can be stored in a simple manner, but cache efficiency deteriorates because related user data is not stored together
Solution Approach 1:
The patent changes the storage parameter from sequential user account creation order to graph-based connected components order. Users are reordered based on their social graph relationships, placing users in the same connected component adjacent to each other in storage. This parameter transformation enables related user data to be stored together, significantly improving cache efficiency while maintaining storage simplicity.
2Power
If data shards are distributed across separate database server instances, then data access load is spread, but cache effectiveness deteriorates because related users' data are not stored together
Solution Approach 1:
The patent applies graph-based connected components analysis to reorder users within data shards, changing the storage parameter from arbitrary assignment to relationship-based grouping. This ensures that when data is sharded across database server instances, each shard contains locally connected users who are likely to be accessed together, thereby improving cache effectiveness while maintaining load distribution across instances.
3Ease of operation
If mapping data structure is used to map user IDs to data shards, then user data locations can be determined, but memory and CPU cache performance deteriorates due to scattered storage of related user data
Solution Approach 1:
The patent transforms the mapping parameter from sequential user ID assignment to graph-based connected components assignment. The mapping data structure continues to function for locating user data, but the underlying storage order is changed so that users in the same connected component are stored adjacently. This parameter transformation enables the mapping structure to remain operational while dramatically improving memory and CPU cache performance through localized data access patterns.
Data Source
AI summary
Embodiments are disclosed for improving cache or memory efficiency of a social network system. A method according to some embodiments includes steps of: receiving an instruction to improve cache or memory efficiency of social graph data of a social graph; generating based on the social graph a partitioning tree including multiple bottom-level buckets, the partitioning tree dividing the vertices of the social graph into the bottom-level buckets and ordering the bottom-level buckets such that a social network metric regarding the vertices is optimized; assigning user IDs to the vertices of the social network in a numerical sequence based on the ordering of the bottom-level buckets; storing the social graph data of the users in storage locations in an order according to the numeral sequence of the assigned user IDs of the vertices.