Graph Partitioning Module Iterative Edge Exclusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph clustering techniques are inefficient for massive datasets due to high computational and storage requirements, as they consider all edges and nodes globally, making them impractical for datasets with millions of clusters.
Innovation Solution
A graph partitioning module iteratively excludes edges based on the degree of overlap between nodes and their neighbors, allowing for localized evaluation and scalable partitioning of nodes into clusters without requiring concurrent loading of all nodes and edges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If global clustering techniques evaluate all edges and nodes to find optimal partitions, then clustering accuracy is improved, but computational complexity and storage requirements increase dramatically
Solution Approach 1:
The patent segments the graph evaluation process into localized operations around each node. Instead of evaluating all edges and nodes globally, the system performs clustering decisions locally for each node by examining only its incident edges and neighboring nodes. This segmentation transforms the intractable global optimization problem into manageable local decisions that can be processed independently and in parallel.
Solution Approach 2:
The patent extracts and removes edges from the graph based on local clustering decisions. As nodes are assigned to clusters, edges connecting nodes in different clusters are identified and removed from further consideration. This extraction process gradually simplifies the graph structure, reducing computational complexity while maintaining clustering accuracy through iterative refinement.
2Measurement precision
If global clustering techniques consider all edges and nodes, then clustering quality is improved, but storage requirements increase for massive datasets
Solution Approach 1:
The patent segments the data storage requirements by storing and processing information locally for each node rather than maintaining global graph structures in memory. Each node's clustering decision depends only on its incident edges and neighbors, allowing the system to process massive graphs with limited storage by loading only relevant local subgraphs into memory during evaluation.
Solution Approach 2:
The patent applies local quality by making clustering decisions based on local graph properties rather than global characteristics. Each node's cluster assignment is determined by examining local edge weights and neighbor relationships, allowing high-quality clustering without requiring storage of entire massive datasets. The system processes nodes independently using only locally relevant information.
3Productivity
If iterative edge exclusion is performed for each node, then scalability is improved, but processing time per node increases
Solution Approach 1:
The patent performs preliminary actions by pre-calculating and storing edge weights and neighbor relationships for each node before the iterative clustering process. This preliminary preparation allows the iterative edge exclusion algorithm to proceed efficiently by accessing pre-computed local graph properties rather than calculating them repeatedly during each iteration, reducing processing time per node while maintaining scalability.
Data Source
AI summary
Methods, machines, and stored instructions are provided for partitioning a graph of nodes into clusters of nodes by iteratively excluding edges in the graph. For each node of at least a subset of nodes in the graph, a graph partitioning module determines whether to exclude edges for the node and, if so, selects for exclusion edge(s) to at least a subset of the node's neighbor(s). The module selects edge(s) to the node's neighbor(s) for exclusion based at least in part on a degree of overlap between the node's neighbor(s) and neighbor(s) of the node's neighbor(s). For any subset(s) that are yet not sufficiently partitioned into clusters, the module repeats the step of determining whether to exclude edges and, if so, selecting nodes for exclusion, and determining whether or not the nodes are sufficiently partitioned. Subset(s) of nodes that are already sufficiently partitioned may be skipped during the repeated steps.