Heterogeneous Accelerator Switching for Data Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems using heterogeneous hardware accelerators for data model training either underutilize the potential of available accelerators by continuing to use a processor until a threshold is reached or by assigning a single accelerator to handle the entire load, failing to leverage the full potential of multiple accelerators.
Innovation Solution
A method and system that initiate data model training using a first hardware accelerator and iteratively switch to the next in a sequence based on accuracy thresholds across epochs, ensuring maximum accuracy is achieved and all accelerators are utilized effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single hardware accelerator with high specifications is used for the entire training load, then processing speed and accuracy are improved, but cost and hardware requirements increase significantly
Solution Approach 1:
The patent segments the model training workload across multiple heterogeneous hardware accelerators with varying specifications. Different accelerators handle different portions of the training epochs based on their capabilities, allowing the system to achieve high accuracy through coordinated effort rather than relying on a single high-end accelerator, thus reducing individual hardware requirements and overall cost.
Solution Approach 2:
The system creates a universal training framework that can accommodate multiple types of hardware accelerators with different specifications. The training workflow is designed to be adaptable across heterogeneous devices, allowing various accelerators to perform the same training function at different performance levels, thereby reducing dependency on any single high-specification hardware type.
2Ease of operation
If a single hardware accelerator is assigned to handle the entire training load, then simplicity of management is improved, but utilization efficiency of heterogeneous accelerators deteriorates
Solution Approach 1:
The patent implements a dynamic workload distribution mechanism where the training epochs are automatically allocated across multiple accelerators based on their current availability and capabilities. The system dynamically adjusts which accelerator handles which portion of the training, optimizing utilization efficiency while maintaining straightforward management through automated decision-making.
Solution Approach 2:
The training management system operates autonomously by automatically distributing training epochs across available accelerators without requiring manual intervention. The system self-manages the allocation and coordination of heterogeneous accelerators, improving utilization efficiency while keeping management simple through automation rather than complex manual procedures.
3Stability of the object's composition
If utilization threshold-based switching is used between heterogeneous accelerators, then system stability is improved, but complete potential of accelerators is not utilized
Solution Approach 1:
Instead of using a fixed utilization threshold for switching between accelerators, the patent changes the switching criterion to be based on training epoch completion and model accuracy metrics. This parameter change allows the system to maintain stability through structured epoch-based transitions while fully utilizing accelerator potential by assigning specific training portions to each accelerator based on their capabilities rather than arbitrary threshold crossings.
4Ease of manufacture
If heterogeneous accelerators with varying specifications are used, then cost-effectiveness is improved, but coordination complexity increases
Solution Approach 1:
The patent assigns specific training epochs and workloads to individual accelerators based on their local capabilities and specifications. Each accelerator operates with optimized parameters suited to its specific hardware characteristics, allowing cost-effective use of heterogeneous devices while the central coordination system manages the distribution logic to minimize coordination complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Existing approaches for switching between different hardware accelerators in a heterogeneous accelerator approach have the disadvantage that complete potential of the heterogeneous hardware accelerators do not get used as the switching relies on load on the accelerators or a random switching in which entire task gets reassigned to a different hardware accelerator. The disclosure herein generally relates to data model training, and, more particularly, to a method and system for data model training using heterogeneous hardware accelerators. In this approach, the system switches between hardware accelerators when a measured accuracy of the data model after any epoch is below a threshold of accuracy.