Dynamic Workload Steering Between Hardware Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud management systems lack the ability to dynamically select the most suitable hardware accelerators for executing kernels in high-performance computing and cloud environments, leading to suboptimal performance due to rigid and static workload steering that does not consider the dynamicity of the execution environment.
Innovation Solution
A dynamic workload steering service that includes a server, agent, and client component, which centrally coordinates the selection of hardware accelerators based on analytics and insights, allowing for real-time workload steering between accelerators and supporting machine learning-based algorithms to optimize kernel execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If static workload steering is used to assign kernels to accelerators, then system simplicity is maintained, but application performance deteriorates due to inability to adapt to dynamic execution environments
Solution Approach 1:
The patent implements dynamic workload steering that adapts to changing execution environments by monitoring performance metrics and workload characteristics in real-time, allowing the system to select optimal accelerators dynamically rather than using static assignment rules
Solution Approach 2:
The system collects performance metrics and execution data from accelerator operations, uses this feedback to analyze and predict optimal accelerator selections, and adjusts workload steering decisions based on learned patterns from historical performance data
2Measurement precision
If dynamic workload steering with machine learning algorithms is implemented, then accelerator selection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent introduces a workload steering service as an intermediary layer between the application and accelerators, which contains the machine learning algorithms and performance analysis logic, isolating the complexity from both the application and the accelerator hardware
Solution Approach 2:
The system performs preliminary analysis of workload characteristics and accelerator capabilities using machine learning models before actual kernel execution, pre-determining optimal accelerator assignments to avoid runtime complexity
3Ease of operation
If rigid accelerator assignment is used, then ease of operation is maintained, but cloud system efficiency deteriorates due to inability to optimize resource allocation
Solution Approach 1:
The workload steering service automatically analyzes workload requirements and accelerator capabilities to make intelligent assignment decisions without requiring manual configuration or user intervention, enabling the system to self-optimize resource allocation
Data Source
AI summary
A system and method determine accelerators to execute a function of an application in a computing node, where the application includes a workload steering client. The method includes receiving, by the workload steering client, a workload request from the application identifying a first task, and determining, by the workload steering client, whether the first task has an assigned accelerator. The method further includes requesting, by the workload steering client, the first task to be executed on the assigned accelerator, and returning, by the workload steering client, a result of the first task to the application.