Two-Stage Clustering for Large Social Graph Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing clustering methods, such as K-means, face challenges in efficiently segmenting large social networks with thousands of users into optimal clusters, often resulting in local minima and requiring excessive computational effort, and struggle to accurately identify naturally forming communities.
Innovation Solution
A novel clustering method that combines seeded clustering using variations of the K-means method with Density-Based Spatial Clustering of Applications with Noise (DBSCAN), producing primary and secondary centroids to refine clusters and maximize object affinity within clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If K-means method is used for clustering large number of objects, then segmentation into K clusters is achieved, but computational effort becomes prohibitively extensive and solution converges to local minimum
Solution Approach 1:
The patent applies segmentation by dividing the clustering process into two distinct stages: (1) seeded clustering that generates primary centroids, and (2) autonomous density-based clustering that refines these centroids into final clusters. This segmentation allows each stage to focus on specific aspects of clustering, reducing overall computational complexity while improving accuracy.
Solution Approach 2:
The patent implements preliminary action through the seeded clustering stage that pre-computes primary centroids before the final clustering stage. By performing this preliminary grouping based on initial seed points, the algorithm reduces the search space for the second stage, significantly decreasing computational time required for the exhaustive search that would otherwise be needed.
2Measurement precision
If number of clusters K is increased to capture more communities, then clustering detail improves, but computational complexity increases
Solution Approach 1:
The patent segments the clustering task into two phases where the first phase (seeded clustering) handles the initial grouping with a manageable number of seeds, and the second phase (autonomous density-based clustering) automatically determines the final number of clusters based on data density. This avoids the need to pre-specify a large K while still capturing detailed community structures.
Solution Approach 2:
The autonomous density-based clustering stage employs self-service by allowing the algorithm to automatically determine the optimal number of clusters through density analysis rather than requiring manual specification of K. The algorithm autonomously identifies cluster boundaries based on local density variations, eliminating the need for complex parameter tuning.
3Measurement precision
If traditional clustering methods are applied to large social graphs, then some communities are identified, but naturally forming clusters are not accurately captured
Solution Approach 1:
The patent applies parameter changes by transitioning from the parameter-driven K-means approach (which requires specifying K) to a density-based approach where cluster parameters emerge naturally from the data. The density threshold and radius parameters are adjusted to capture different community structures, allowing the algorithm to adapt to various clustering scenarios in social graphs.
Solution Approach 2:
The patent creates a composite clustering methodology that combines elements of seeded clustering (providing initial structure) with density-based clustering (providing adaptive refinement). This composite approach leverages the strengths of both methods: the structured initialization of seeded clustering and the adaptive, parameter-free nature of density-based clustering, resulting in accurate identification of natural clusters.
Data Source
AI summary
A method of complementary clustering of a vast population of objects is disclosed. The method aims at maximizing a global measure of object affinity within naturally-formed clusters. A first clustering procedure produces primary centroids of clusters of objects and a second clustering procedure produces secondary clusters of the primary centroids and corresponding secondary centroids. Refined clusters of the population of objects are formed based on object proximity to the secondary centroids. The first clustering procedure is preferably based on a variation of the K-means method, and the second clustering procedure is preferably based on the Density-Based Spatial Clustering of Applications with Noise (DBSCAN). An apparatus implementing the method is devised to facilitate conflict-free parallel processing.