Hybrid Task Scheduling for Sparse Tensor Parallelization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing irregular computations, particularly sparse tensor computations, on modern multi-core systems is challenging due to dependencies on input data known only at runtime, memory locality issues, and the complexity of non-uniform memory access (NUMA) architectures, which complicates parallelization and performance optimization.
Innovation Solution
Implementing a hybrid static-plus-dynamic task scheduling scheme that maps operations across multiple processors, utilizing locality-aware scheduling decisions and load balancing to optimize data locality and reduce overhead, while dynamically partitioning operations based on sparse data patterns to enhance parallel execution efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If irregular computations are parallelized on multi-core systems, then computational throughput is improved, but scheduling complexity and overhead increase
Solution Approach 1:
The patent segments irregular computations into regular computational patterns and irregular data access patterns. By separating these concerns, the system can apply standard parallelization techniques to the regular portions while handling the irregular portions through specialized data structures and scheduling approaches, thereby improving throughput without proportionally increasing scheduling complexity.
Solution Approach 2:
The patent changes the representation parameters of sparse tensor data from traditional formats to coordinate-based formats (COO, CSR, CSC) that enable more regular memory access patterns. This parameter change allows irregular computations to be expressed in terms that are more amenable to parallel processing, improving computational throughput while reducing the complexity of scheduling irregular operations.
2Speed
If data locality optimizations are applied to irregular codes, then memory access efficiency is improved, but optimization difficulty increases due to runtime data dependencies
Solution Approach 1:
The patent performs preliminary actions by pre-processing sparse tensor data into coordinate-based formats and pre-computing data access patterns before the main computation begins. This preliminary organization of data enables subsequent memory access optimizations without requiring complex runtime decisions, thereby improving memory access efficiency while keeping the optimization process manageable.
Solution Approach 2:
The patent introduces coordinate-based data structures as intermediaries between the irregular computation logic and the underlying memory system. These intermediate representations (COO, CSR, CSC formats) act as a buffer that regularizes memory access patterns, enabling efficient memory access without requiring direct optimization of the irregular computation code itself.
3Productivity
If sparse tensor computations are performed on multi-core systems with NUMA architecture, then parallel processing capability is improved, but memory access latency varies significantly
Solution Approach 1:
The patent applies local quality by assigning data partitions to specific processor cores based on their NUMA node affinity. Sparse tensor data is partitioned and stored in memory locations that are locally accessible to the cores that will process them, ensuring that each core primarily accesses memory from its local NUMA node. This reduces memory access latency variability while maintaining parallel processing capability across multiple cores.
Data Source
AI summary
A scheduling system can schedule several operations for parallel execution on a number of work processors. At least one of the operations is not to be executed, and the determination of which operation or operations are not to be executed and which ones are to be executed can be made only at run time. The scheduling system partitions a subset operations that excludes the one or more operation that are not to be executed into several groups based on, at least in part, an irregularity of operations resulting from the one or more operation that are not to be executed. In addition, the partitioning is based on, at least in part, locality of data elements associated with the subset of operations to be executed or loading of the several work processors.