Graph Computation Component Folding via Virtual CPUs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graph-based computations often face inefficiencies due to high overhead and latency associated with multiple processes running concurrently, as each process incurs significant overhead, leading to increased resource usage and reduced scalability.
Innovation Solution
The approach of 'component folding' where multiple graph components are executed within a single process, eliminating the need for inter-process buffering and reducing context-switching, thereby improving performance by minimizing data movement and startup time while maintaining expressibility and ease of use.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple processes are used to execute graph components, then parallelism and modularity are improved, but overhead and resource consumption increase
Solution Approach 1:
The patent merges multiple graph component executions into a single process by implementing a work stealing scheduler where one process manages multiple virtual CPUs and steals work from other processes' queues. This combines the benefits of parallelism through virtual CPUs while avoiding the overhead of multiple actual processes, as all operations occur within a single process context with shared memory access.
2Productivity
If multiple processes are used to execute graph components, then concurrency is improved, but context-switching and latency increase
Solution Approach 1:
The patent introduces virtual CPUs as an abstract dimension layer between the single physical process and multiple graph component executions. This allows the system to simulate multi-process concurrency without actual process context switches, as the virtual CPU switching occurs through software management within the single process rather than through operating system process context switches.
3Device complexity
If components are consolidated into fewer processes, then overhead is reduced, but expressibility and ease of use may be sacrificed
Solution Approach 1:
The patent creates a universal execution model where a single process can dynamically execute any graph component through the work stealing scheduler. The virtual CPU abstraction provides a unified interface for component execution, maintaining expressibility through the graph model while achieving overhead reduction through single-process execution. The system is versatile enough to handle any DAG-based computation graph.
Data Source
AI summary
Executing graph-based computations includes: accepting a specification of a computation graph in which data processing elements are joined by linking elements; dividing the data processing elements into sets, at least one of the sets including multiple of the data processing elements; assigning to each set a different computing resource; and processing data according to the computation graph, including performing computations corresponding to the data processing elements using the assigned computing resources.