Directed Graph Clustering for Skewed Device Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed computing systems face challenges in efficiently clustering large amounts of device identifiers due to computational burdens, particularly when handling skewed data sets from users with a large number of device identifiers, leading to degraded customization of online services across different devices.
Innovation Solution
The use of a directed graph representation and specific rules for iteratively updating the graph, including removal of leaf nodes, reconnection of chain and split nodes, and key-value pair management, reduces data processing and identifies data skews, enabling efficient clustering and customization across devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If distributed computing is used to cluster device identifiers, then computational burden is reduced, but data processing efficiency deteriorates when handling skewed data sets
Solution Approach 1:
The patent segments the device identifier dataset by identifying and separating high-degree nodes (web robots with thousands of identifiers) from regular users. This segmentation allows the system to handle skewed data by processing different node types through different pathways, preventing any single computer from receiving an overwhelming subset of data and thereby maintaining processing efficiency while distributing the computational burden.
2Reliability
If device identifiers are clustered to enable customization, then service quality improves, but computational resources are overwhelmed
Solution Approach 1:
The patent performs preliminary actions by pre-identifying high-degree nodes and their associated device identifiers before the main clustering process. By预先 marking and separating these nodes, the system prepares the data in advance to avoid computational bottlenecks during the actual clustering operation, thereby maintaining service customization quality while reducing the peak computational resource requirements.
3Measurement precision
If all device identifiers are processed uniformly, then clustering accuracy is maintained, but processing time increases significantly
Solution Approach 1:
The patent applies local quality by treating different types of nodes differently based on their characteristics. High-degree nodes (web robots) are identified and handled with specialized logic, while regular user nodes undergo standard clustering. This differentiated approach maintains clustering accuracy for both groups while significantly reducing processing time by avoiding uniform treatment of all identifiers.
Data Source
AI summary
Various embodiments describe clustering of nodes of a directed graph based on the oriented edges of the directed graph and on a set of rules. In an example, each node represents a device identifier associated with a computing device. The device identifier facilitates an online activity provided by a computing service. A computing system accesses the directed graph and generates clusters that contain subsets of the nodes by at least iteratively updating the directed graph based on the set of rules. The set of rules specifies (i) removal of leaf nodes from the directed graph, (ii) reconnection of nodes that form a chain in the directed graph, and (iii) reconnection of nodes that form a split in the directed graph. The computing system also associates a client profile with a subset of the nodes contained in a cluster from the clusters.