Hybrid Acceleration Discovery Service for Dynamic Workload Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In machine learning and deep learning environments, existing acceleration methods are inefficient in dynamically discovering and routing work to appropriate accelerators, leading to suboptimal processing times for model training and inference, especially when dealing with large volumes of data.
Innovation Solution
A system and method for hybrid acceleration that involves a first computing system requesting access to a plurality of accelerators, receiving access information, analyzing it to identify the most suitable accelerator, and offloading processing jobs to that accelerator, allowing for dynamic allocation of work based on performance thresholds and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If existing acceleration methods are used, then processing speed is improved, but dynamic discovery and routing efficiency deteriorates
Solution Approach 1:
The patent introduces a discovery service as an intermediary component that mediates between computing systems and accelerators. This service dynamically discovers available accelerators, manages access information, and routes workloads appropriately, resolving the contradiction by improving both processing speed through efficient accelerator utilization and dynamic discovery efficiency through centralized management.
Solution Approach 2:
The system implements feedback mechanisms where the discovery service continuously monitors accelerator availability, performance metrics, and workload status. This feedback loop enables dynamic adjustment of routing decisions, ensuring that processing jobs are directed to the most suitable accelerators based on real-time conditions, thereby improving both speed and routing efficiency.
2Productivity
If multiple accelerators are available, then processing capability is improved, but access management complexity increases
Solution Approach 1:
The discovery service provides universal access management functionality that works across multiple different accelerator types and computing systems. It implements a standardized interface for requesting, analyzing, and granting access to accelerators, thereby managing multiple accelerators without proportionally increasing complexity.
Solution Approach 2:
The discovery service acts as a mediator between computing systems and multiple accelerators, abstracting away the complexity of direct accelerator management. It maintains access information for multiple accelerators and handles all routing decisions centrally, allowing processing capability to scale with multiple accelerators while keeping access management complexity contained within the service itself.
3Adaptability or versatility
If accelerators are dynamically discovered, then adaptability is improved, but information analysis time increases
Solution Approach 1:
The discovery service performs preliminary actions by pre-discovering and cataloging available accelerators before workloads arrive. It maintains up-to-date access information for multiple accelerators in advance, so when processing jobs need to be routed, the analysis can be performed quickly against pre-collected information rather than discovering accelerators from scratch at the time of routing.
Solution Approach 2:
The discovery service maintains continuous operation, continuously discovering and updating accelerator availability information. This continuous action ensures that adaptability is improved through real-time awareness of accelerator status while minimizing analysis time because the information is already current and ready for immediate use in routing decisions.
Data Source
AI summary
Embodiments of the invention include methods and systems for hybrid acceleration in a processing environment. Aspects of the invention include transmitting, by a first computing system to a second computing system, a request for access to an accelerator. The first computing system receives access information for a plurality of accelerators from the second computing system responsive to the request. The first computing system analyzes the access information for the plurality of accelerators to identify a first accelerator from the plurality of accelerators and offloads a first processing job to the first accelerator utilizing the access information.