Hardware-Agnostic Virtualized ML Accelerator for Cross-Vendor Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current approaches for virtualizing ML/AI accelerators are hardware-specific, limiting the migration of ML/AI workloads across different physical hardware and vendors, and requiring custom plug-ins and libraries, which makes live migration impossible and cold migration between vendors challenging.
Innovation Solution
A hardware-agnostic virtualized ML/AI accelerator (vAIA) is introduced, which runs ML applications on a virtual computing instance, generates tensors and computational graphs, and executes computations using physical hardware ML accelerators, abstracting hardware specifics and enabling migration across different hardware and vendors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If hardware-specific virtualization approaches are used for ML/AI accelerators, then the accelerators can be virtualized and shared across multiple workloads, but migration of ML/AI workloads across different physical hardware and vendors becomes limited and requires custom plug-ins and libraries
Solution Approach 1:
The patent introduces a hardware abstraction layer (HAL) as an intermediary between the virtualized ML/AI accelerator and the physical hardware. This HAL provides a standardized interface that decouples the virtualization software from hardware-specific implementations, allowing workloads to migrate across different vendors' hardware without requiring custom plug-ins. The abstraction layer translates hardware-specific operations into vendor-neutral commands, resolving the contradiction between hardware adaptability and system complexity.
Solution Approach 2:
The patent creates a universal virtualized ML/AI accelerator interface that can work with multiple types of physical hardware from different vendors. By designing a vendor-agnostic abstraction layer with standardized data structures and communication protocols, the system achieves multi-functionality where a single virtualization platform can support diverse hardware backends, eliminating the need for vendor-specific customizations.
2Reliability
If hardware-specific virtualization is implemented, then ML/AI accelerators can be virtualized, but live migration of workloads becomes impossible and cold migration between vendors is challenging
Solution Approach 1:
The hardware abstraction layer serves as a mediator that enables workload migration by translating hardware-specific operations into standardized commands. This allows workloads to be moved between different physical hosts and vendors without requiring customization, as the abstraction layer handles the hardware-specific details uniformly across all platforms.
Solution Approach 2:
The patent employs standardized data structures and communication protocols that can be dynamically configured to work with different hardware parameters. By using parameterized interfaces rather than hard-coded hardware-specific code, the system can adapt to different hardware configurations and enable seamless migration by changing configuration parameters rather than code.
3Productivity
If ML/AI applications are tightly coupled with specific hardware, then optimization for that hardware can be achieved, but vendor lock-in occurs and operational management becomes complex
Solution Approach 1:
The patent segments the ML/AI system into distinct layers: the hardware-specific computation layer and the vendor-neutral virtualization layer. This segmentation allows hardware optimization to occur at the computation layer while the virtualization layer maintains vendor independence. The abstraction layer acts as a boundary that preserves both hardware-specific performance optimizations and vendor-agnostic portability.
Solution Approach 2:
The patent creates a virtual copy of the hardware interface that emulates the behavior of physical accelerators through software. This virtualization allows ML/AI applications to run on standardized virtual hardware interfaces while the underlying physical hardware can be optimized independently. The virtual copy decouples application performance from vendor-specific hardware implementations.
Data Source
AI summary
The disclosure provides an approach for a hardware agnostic paravirtualized machine learning (ML) and artificial intelligence (AI) accelerator in a virtual environment. A method includes running a ML application on a first virtual computing instance (VCI) on a host; generating a set of tensors based on one or more computations associated with the ML application; generating a computational graph for the ML application based on the set of tensors; providing, by a paravirtualized ML accelerator running on the host, the computational graph to a driver VCI running on the host; and executing the one or more computations associated with the ML application using at least one of the one or more physical hardware ML accelerators associated with the paravirtualized ML accelerator.