Graph Data Processing with Cost-Based Policy Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graph pattern matching algorithms for large-scale graph data processing are inefficient, failing to meet the processing requirements due to high execution costs and redundant operations.
Innovation Solution
A graph data processing system with processing units equipped with decision modules that determine set operations for extracting subgraphs, select execution policies based on cost values, and store results for reuse, along with dynamic partitioning to optimize processing tasks across multiple units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If graph pattern matching algorithm is used to extract subgraphs from large-scale graph data, then subgraph extraction capability is achieved, but execution efficiency deteriorates
Solution Approach 1:
The patent divides the graph data into multiple partitions and processes each partition independently using multiple processing units. This segmentation allows parallel processing of graph pattern matching tasks, significantly improving execution efficiency while maintaining subgraph extraction accuracy through consistent application of the matching algorithm across all partitions.
Solution Approach 2:
The patent performs preliminary optimization by determining optimal execution policies for set operations before actual graph pattern matching. Cost values are calculated in advance for different execution policies, and the best policy is selected and cached. This preliminary action reduces the computational overhead during actual pattern matching, improving execution efficiency without compromising extraction accuracy.
2Measurement precision
If multiple set operations are executed on neighboring node sets, then subgraph matching completeness is improved, but redundant operations increase
Solution Approach 1:
The patent calculates cost values for different execution policies of set operations in advance and determines the optimal policy before execution. By preliminarily analyzing the cost associated with intersection and difference operations on neighboring node sets, the system selects the most efficient execution path, avoiding redundant operations while ensuring complete subgraph matching.
Solution Approach 2:
The patent dynamically adjusts execution parameters by selecting different execution policies based on calculated cost values. For set operations involving neighboring node sets, the system changes execution parameters such as operation order, data structure selection, and memory access patterns to minimize redundant operations while maintaining matching completeness.
3Productivity
If execution policies are optimized for each set operation, then processing speed is improved, but system complexity increases
Solution Approach 1:
The patent optimizes processing speed by changing execution parameters through policy selection. For each set operation, the system evaluates different execution policies and selects the optimal one based on cost values. This parameter optimization improves processing speed while managing complexity through automated cost-based selection rather than manual configuration.
Solution Approach 2:
The patent implements self-service through automated policy selection. The decision module automatically determines the optimal execution policy for each set operation by calculating cost values and selecting the best option without external intervention. This self-service mechanism handles the complexity internally while presenting a simplified interface, improving processing speed without burdening the user with system complexity.
4Speed
If on-chip caches are used to store execution results, then memory access speed is improved, but chip area increases
Solution Approach 1:
The patent applies local quality by using on-chip caches specifically for storing execution results of set operations that benefit from fast access. Rather than uniformly increasing memory speed across the entire system, the on-chip caches are strategically placed to store frequently accessed data such as intermediate results and execution policies, improving memory access speed for critical operations while minimizing the overall chip area impact.
Data Source
AI summary
Systems, methods, devices and storage media for graph data processing are provided. In one aspect, a graph data processing system includes a memory and a plurality of processing units, and each processing unit is provided with a decision module. Each processing unit is configured to determine set operations required for extracting one or more subgraphs matching a specified graph pattern from target graph data according to a preset graph pattern matching algorithm. Then, for each set operation, the decision module is configured to determine a cost value corresponding to a performance of the processing unit occupied to execute the set operation in accordance with different execution policies, and further select a target execution policy with a smallest cost value to execute the set operation.