Distributed Graph Query Concurrency via Dynamic Worker Reassignment

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Efficient execution of graph queries on distributed graph databases is challenging due to irregular access patterns and limited locality, leading to inefficient resource utilization and throughput, especially when multiple workers are utilized for query execution.

Innovation Solution

A runtime engine assigns multiple workers to execute portions of graph queries within a parallel-while construct, allowing for asynchronous execution and reassignment based on performance history and resource requirements, enabling concurrent execution of multiple queries and optimizing worker utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple workers are utilized to execute graph queries on distributed graph database, then throughput is increased, but resource utilization efficiency deteriorates due to irregular access patterns and limited locality

Engineering Contradiction:

ImprovethroughputVSAvoidresource utilization efficiency

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The system dynamically assigns and reassigns workers to graph queries based on real-time performance metrics and resource availability. The runtime engine monitors worker performance history and adjusts assignments to optimize resource utilization while maintaining high throughput, resolving the contradiction between productivity and resource efficiency

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The runtime engine collects feedback from worker performance history and uses this information to make informed decisions about worker assignment and reassignment. This feedback mechanism enables the system to optimize resource allocation continuously, improving both throughput and resource utilization efficiency

Inventive Principle:

Principle #23Feedback

2Device complexity

If workers are assigned to execute graph queries without reassignment, then execution simplicity is maintained, but worker utilization optimality deteriorates

Engineering Contradiction:

Improveexecution simplicityVSAvoidworker utilization optimality

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The runtime engine performs preliminary analysis of worker performance history before making assignment decisions. By pre-evaluating performance metrics and preparing optimal assignment strategies in advance, the system achieves high worker utilization optimality without introducing excessive complexity to the execution process

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11947539B2Concurrency and cancellation in distributed asynchronous graph processing

Publication Date: 2024.04.02 ORACLE INT CORP

AI summary

Techniques to efficiently assign available workers to executing multiple graph queries concurrently on a distributed graph database are disclosed. The techniques comprise a runtime engine assigning multiple workers to executing portions of multiple graph queries, each worker in each assignment asynchronously executing a portion of a graph query within a parallel-while construct that includes return statements at different locations, and the runtime engine reassigning a worker to executing another portion of the same or a different graph query to optimize the overall performance of all workers.