Graph Duplicate Detection via Neighborhood Subgraph Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for duplicate determination in large graphs, such as those used in master data management systems, face challenges in efficiently identifying and merging duplicate records due to the complexity of relationships between millions of nodes and edges, often resulting in suboptimal performance and false positives.
Innovation Solution
A method that involves identifying neighborhood subgraphs for nodes, comparing their similarity using a calculated metric, and determining duplicates based on predefined thresholds, with additional criteria for twin detection to avoid false positives, and utilizing an index structure with edge descriptors for efficient candidate list generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used for duplicate determination in large graphs, then comprehensive comparison can be performed, but processing time and computational complexity increase significantly
Solution Approach 1:
The patent segments the graph into neighborhood subgraphs centered at each node, comparing only relevant local structures rather than performing exhaustive global comparisons. This segmentation reduces the comparison scope from O(n²) to O(k) where k is the neighborhood size, significantly reducing processing time while maintaining detection accuracy through localized structural analysis
Solution Approach 2:
The patent performs preliminary actions by calculating and storing index structures (such as canonical labels or hash values) for neighborhood subgraphs before actual duplicate detection. This preprocessing enables rapid comparison during the detection phase, reducing real-time processing time while preserving comprehensive comparison capabilities for accuracy
2Reliability
If exhaustive node comparison is performed to ensure accurate duplicate detection, then false positives are reduced, but the number of comparisons and computational resources required increase
Solution Approach 1:
The patent applies local quality by focusing comparison efforts on the neighborhood subgraphs where duplicate relationships are most likely to manifest. By analyzing local structural properties (edges, paths, and connections within k-hop neighborhoods) rather than global graph properties, the method achieves high reliability in duplicate detection while improving productivity through reduced comparison scope
Solution Approach 2:
The patent introduces intermediary structures such as canonical labels, hash values, or fingerprint representations of neighborhood subgraphs that mediate the comparison process. These intermediaries enable efficient similarity assessment without requiring exhaustive direct node-by-node comparison, thus maintaining detection reliability while enhancing processing efficiency
3Productivity
If simple comparison methods are used to improve processing speed, then computational complexity is reduced, but false positives increase
Solution Approach 1:
The patent transitions from comparing nodes in a single dimension (direct attribute matching) to comparing neighborhood subgraphs in multiple dimensions (structural relationships, path patterns, connection topologies). This dimensional expansion enables more precise duplicate detection through multi-faceted structural analysis while maintaining processing speed by working with pre-computed neighborhood representations rather than exhaustive comparisons
Data Source
AI summary
Embodiments of the present invention determines duplicates in a graph. The graph comprises nodes representing entities and edges representing relationships between the entities. The method comprises: identifying at least two nodes in the graph. A neighborhood subgraph may be determined for each of the two nodes. The neighborhood subgraph includes the respective node. The method further comprises determining whether the two nodes are duplicates with respect to each other, based on a result of a comparison between the two subgraphs.