Graph Data Distribution via Sub-Graph Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed computing methods for graph data processing are inefficient due to the lack of consideration for graph structure, leading to high network costs and suboptimal data distribution in distributed environments.
Innovation Solution
A method and apparatus that search for common sub-graphs in graph data, generate partitions by merging or arranging nodes based on adjacency, and calculate processing costs to allocate data to partitions with minimal processing costs, considering factors like hop number, time complexity, and adjacent vertices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hashing method is used to distribute graph data, then data can be distributed to servers, but processing performance is limited and network costs increase due to not considering graph structure
Solution Approach 1:
The patent segments graph data into partitions based on graph structure characteristics (common sub-graphs, vertex degrees, connectivity patterns) rather than simple hashing. This segmentation preserves local graph structures within partitions, reducing cross-partition edges and network communication requirements while enabling efficient parallel processing across multiple servers.
Solution Approach 2:
The patent applies local quality by creating partitions with heterogeneous characteristics tailored to specific graph structures. Each partition is optimized based on its local graph properties (density, connectivity, vertex degrees), allowing different processing strategies for different partitions and improving overall processing performance while minimizing network costs.
2Ease of manufacture
If graph data is distributed without considering graph structure, then distribution is simple, but processing performance is limited
Solution Approach 1:
The patent performs preliminary analysis of graph structure (identifying common sub-graphs, calculating vertex degrees, determining connectivity patterns) before distribution. This preliminary action enables the system to understand graph characteristics and create optimized partitions that preserve local structures, thereby improving processing performance without significantly complicating the distribution process.
Solution Approach 2:
The patent changes the distribution parameters from simple hash values to graph-structure-based parameters such as common sub-graph identifiers, vertex degree distributions, and connectivity metrics. This parameter change allows the distribution algorithm to consider graph structure while maintaining computational efficiency and scalability.
3Productivity
If common sub-graphs are merged into partitions, then processing cost is reduced, but partition generation complexity increases
Solution Approach 1:
The patent extracts common sub-graphs from the overall graph structure and identifies them as distinct units for partitioning. By taking out these recurring structural patterns, the system can merge them into single partitions, reducing the number of cross-partition edges and minimizing network communication requirements, thereby improving processing efficiency.
Solution Approach 2:
The patent uses copying by identifying and replicating common sub-graph patterns across different parts of the graph. Once a common sub-graph is identified, the system can efficiently partition all instances of this sub-graph together, reducing partition generation complexity through pattern recognition and reuse rather than analyzing each sub-graph individually.
Data Source
AI summary
A method and apparatus distributes graph data in a distributed computing environment. The method of distributing graph data in a distributed computing environment includes searching graph data for common sub-graphs, generating a partition by merging the common sub-graphs if, as a result of the search, the common sub-graphs are present and generating a partition by arranging the graph data if, as a result of the search, the common sub-graphs are not present, and calculating a processing cost of the graph data and allocating the partition based on the calculated processing cost.