Multi-Die Reconfigurable Processor Virtualization for Multi-Tenant AI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing reconfigurable processors face challenges in supporting multi-tenancy and dynamic workload scenarios in cloud computing environments, leading to hardware underutilization due to the lack of practical virtualization support for accelerators.
Innovation Solution
A data processing system with a reconfigurable processor having two dies in a package, enabling the execution of multiple applications with isolation and minimal overhead through virtual function support, utilizing a compiler to map operations to CGR units for parallel and pipelined computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If reconfigurable processors are used to accelerate machine learning and AI workloads, then computational performance is improved, but hardware utilization efficiency deteriorates due to lack of virtualization support
Solution Approach 1:
The reconfigurable processor is divided into multiple virtual instances through virtualization, allowing each virtual instance to be independently allocated to different workloads. This segmentation enables multiple clients to share the same physical hardware resources, improving overall utilization while maintaining the high computational performance of the underlying reconfigurable architecture
Solution Approach 2:
The reconfigurable processor is designed with universal interfaces and control mechanisms that support multiple virtual functions simultaneously. The hardware can dynamically reconfigure to serve different computational tasks while maintaining a unified resource pool, enabling the same physical device to fulfill multiple roles and improve hardware utilization efficiency
2Reliability
If multiple applications are executed on dedicated hardware, then isolation and security are improved, but hardware resource utilization deteriorates
Solution Approach 1:
A virtualization layer is introduced as an intermediary between multiple applications and the underlying reconfigurable processor. This virtualization layer provides isolation and security guarantees to each application while dynamically managing hardware resource allocation, allowing multiple applications to share the same physical hardware without compromising isolation or utilization efficiency
3Productivity
If reconfigurable processors support virtualization for multi-tenancy, then hardware utilization is improved, but system complexity increases
Solution Approach 1:
The virtualization system incorporates automated resource management and allocation mechanisms that operate autonomously to handle multi-tenancy. The system automatically provisions, configures, and manages virtual instances based on workload demands, reducing the need for manual intervention and minimizing operational complexity while improving hardware utilization
Data Source
AI summary
A data processing system is presented that includes a communication link, a runtime processor, and one or more reconfigurable processors. A reconfigurable processor includes first and second dies arranged in a package, having respective K and L arrays of coarse-grained reconfigurable (CGR) units, and respective first and second communication link interfaces coupled to the communication link. The runtime processor is adapted for configuring the first communication link interface to provide access to the K arrays of CGR units through the communication link from a first physical function driver and from up to M virtual function drivers, and for configuring the second communication link interface to provide access to the K arrays of CGR units of the first die and to the L arrays of CGR units of the second die through the communication link from a second physical function driver and from up to N virtual function drivers.