Weighted Graph Community Detection via Coherence Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing community detection methods in weighted graphs face challenges with noise removal and thresholding, leading to loss of information and sensitivity to threshold values, especially when dealing with overlapping communities in weighted graphs.
Innovation Solution
A system that identifies tagsets, generates counts statistics, and creates a weighted co-occurrence graph, denoises it by removing noise based on co-occurrence consistency, and identifies overlapping communities using importance scores and coherence measures, allowing for aggressive noise removal and detection of maximally coherent communities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If thresholding is used to convert weighted graph to unweighted graph, then community detection can be performed using existing methods, but significant information is lost and results become sensitive to threshold values
Solution Approach 1:
The patent changes the parameter representation from binary (unweighted) to continuous (weighted), preserving the original weight information in the graph edges. This allows community detection algorithms to operate on the weighted graph directly, avoiding information loss while maintaining detection capability through modified objective functions that incorporate weight considerations.
Solution Approach 2:
Instead of converting weighted graph to unweighted graph (traditional approach), the patent inverts the approach by developing community detection methods that work directly on weighted graphs. This reversal eliminates the thresholding step and its associated information loss while maintaining algorithmic feasibility through specialized weighted graph formulations.
2Object-generated harmful factors
If noise removal is performed after community detection, then some noise can be eliminated, but important tags may be improperly removed and communities become poorly defined
Solution Approach 1:
The patent performs noise filtering and weight adjustment as preliminary steps before community detection, rather than after. By preprocessing the weighted graph to remove obvious noise and adjust weights based on co-occurrence statistics, the subsequent community detection operates on a cleaner graph, preventing important tags from being improperly removed while still eliminating noise.
Solution Approach 2:
The patent implements iterative feedback mechanisms where community detection results inform weight adjustments, which in turn refine community detection. This feedback loop allows the system to distinguish between noise and important tags more accurately, preserving community definition precision while removing noise through multiple refinement iterations.
3Adaptability or versatility
If traditional community detection methods are used on weighted graphs, then overlapping communities can be detected, but the methods are sensitive to threshold values and lose information
Solution Approach 1:
The patent modifies the objective function parameters to work with weighted graphs, using weight-based measures such as weighted modularity or weighted cohesion metrics. These parameter changes enable detection of overlapping communities while eliminating threshold sensitivity, as the continuous weight values provide natural differentiation without requiring binary conversion.
Solution Approach 2:
The patent employs dynamic community assignment where entities can have varying degrees of membership in multiple communities based on their weighted connections. This dynamic approach allows overlapping communities to be detected with reliability, as membership is determined by weight thresholds derived from the data itself rather than fixed external parameters.
Data Source
AI summary
The disclosure includes a system and method for detecting communities in a weighted graph. The community detection module includes a tagset data aggregator, a counts statistics engine, a weighted graph generator, a coherence engine, a community detector and a tag recommendation engine. The tagset data aggregator receives tagset data. The counts statistics engine determines counts statistics for the tagset data. The weighted graph generator generates and denoises weighted tag occurrence graph based on the counts statistics. The coherence engine determines importance score for all tags and coherence score for all tagsets in the tagset data. The community detector determines maximally coherent communities in the weighted tag co-occurrence graph. The tag recommendation engine recommends tags in real time using the maximally coherent communities.