Thread Scaling Ratios for Heterogeneous Compute Cluster Workload Orchestration

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing workload orchestration systems face challenges in efficiently executing machine learning applications across heterogeneous compute clusters with varying core configurations, leading to underutilization and asynchronous learning parameter exchanges due to the inability to accurately estimate execution times and scale workloads based on available resources.

Innovation Solution

A workload orchestration system that includes a discovery subsystem to identify compute resources, a manifest subsystem to describe resource demands, a placement subsystem to assign workloads based on thread scaling ratios, and an adaptive modeling subsystem to dynamically define hyperparameters, ensuring optimal resource utilization across clusters with different core counts.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If workloads are distributed across heterogeneous compute clusters with different core configurations, then parallel processing capability is improved, but execution time estimation accuracy deteriorates due to variability in core counts

Engineering Contradiction:

Improveparallel processing capabilityVSAvoidexecution time estimation accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system changes the parameter of execution time estimation by introducing thread scaling ratios that account for different core configurations. Instead of using fixed estimation methods, the system dynamically adjusts execution time predictions based on the relationship between thread count and core count, allowing accurate estimation across heterogeneous clusters with varying core configurations.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system performs preliminary discovery of compute resources and calculation of thread scaling ratios before workload assignment. By pre-characterizing each compute cluster's core configuration and determining how threads scale with cores, the system establishes accurate execution time estimates in advance, eliminating the need for runtime adjustments and enabling efficient workload distribution.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If thread scaling ratios are calculated based on measured execution times, then execution time estimation accuracy is improved, but system complexity increases due to additional measurement and calculation requirements

Engineering Contradiction:

Improveexecution time estimation accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system implements self-service by automatically discovering compute resources, measuring execution times, and calculating thread scaling ratios without manual intervention. The workload orchestration system autonomously characterizes each compute cluster and maintains thread scaling ratio data, eliminating the need for manual system configuration and reducing operational complexity despite the enhanced measurement capabilities.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system uses feedback from measured execution times to continuously refine thread scaling ratio calculations. By monitoring actual execution performance and adjusting scaling ratios accordingly, the system maintains accurate execution time estimates while automating the complexity of measurements and calculations through iterative optimization.

Inventive Principle:

Principle #23Feedback

3Productivity

If workloads are assigned without considering thread scaling ratios, then assignment speed is improved, but resource utilization deteriorates leading to straggler issues

Engineering Contradiction:

Improveworkload assignment speedVSAvoidresource utilization efficiency

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The system performs preliminary calculation of thread scaling ratios and execution time estimates before workload assignment. By pre-computing these parameters for each compute cluster, the system enables rapid workload assignment decisions that consider resource characteristics, achieving both fast assignment and efficient resource utilization without straggler issues.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes the assignment methodology by incorporating thread scaling ratios as a key parameter in workload distribution decisions. Instead of simple round-robin or random assignment, the system uses scaling-aware parameter-based assignment that matches workloads to compute clusters based on their core configurations and measured performance characteristics, optimizing both speed and efficiency.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20230145437A1Execution prediction for compute clusters with multiple cores

Publication Date: 2023.05.11 HEWLETT PACKARD DEVELOPMENT COMPANY LP

AI summary

Systems and methods are described herein to estimate or calculate an execution time for a compute cluster to execute a task based on the number of cores the compute cluster has relative to the number of cores present in a heterogeneous compute cluster for which the time to complete the task was previously measured. In some examples, minimum and maximum scaling ratios are calculated for compute clusters having a different number of cores than a compute cluster for which the time to complete the task has been measured.