Heterogeneous AI Processor Task Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current heterogeneous AI processor architectures face bottlenecks due to inefficient task scheduling and coordination between different computation units, leading to high power consumption and limited adaptability, especially in handling AI algorithms that require parallel computation and diverse application scenarios.
Innovation Solution
A configurable heterogeneous AI processor with a controller that manages and schedules different architectural types of computation units, including ASIC, GPGPU, and FPGA, through task scheduling, synchronization, and access control modules, allowing for flexible operation modes and efficient distribution of computation subtasks based on unit capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a heterogeneous architecture with CPU and AI chip is used, then AI computation performance is improved, but the CPU becomes a bottleneck that constrains the performance of the computation acceleration chip
Solution Approach 1:
The patent extracts the CPU's task scheduling function from the heterogeneous system and implements it directly in the AI chip through an integrated task scheduling module. This removes the CPU bottleneck by self-scheduling computation tasks within the AI accelerator itself, allowing the AI chip to independently manage and execute multiple computation graphs without CPU intervention.
Solution Approach 2:
The AI chip is designed with multi-functional capabilities including an instruction parsing module that can interpret multiple instruction types (computation instructions, data transfer instructions, branch instructions), a task scheduling module for managing multiple computation graphs, and support for various neural network operations. This universal design allows the single AI chip to replace both CPU and dedicated AI accelerator functions.
2Productivity
If a multi-core CPU is used to compensate for system performance, then computation performance is improved, but power consumption and cost increase unnecessarily
Solution Approach 1:
The patent extracts the complex multi-core CPU functionality and replaces it with a simplified AI chip that has integrated task scheduling capabilities. The AI chip independently handles task distribution, resource allocation, and computation execution without requiring multiple CPU cores, thereby achieving high computation performance with significantly reduced power consumption.
Solution Approach 2:
The system transitions from using multiple high-power CPU cores to a single AI chip with optimized architectural parameters including dedicated computation units, on-chip memory, and hardware-accelerated neural network operations. This parameter change enables the system to achieve equivalent or superior computation performance with lower power consumption.
3Adaptability or versatility
If a conventional heterogeneous architecture driven by variety of different software is used, then flexibility is maintained, but coordination efficiency is low
Solution Approach 1:
The patent replaces the software-driven coordination mechanism with a hardware-based task scheduling module integrated into the AI chip. The scheduling module uses hardware logic to parse instructions, manage task queues, and coordinate computation units directly in hardware, eliminating the inefficiencies of software-based coordination while maintaining adaptability through configurable scheduling algorithms.
Solution Approach 2:
The task scheduling module acts as an intermediary between the host processor and computation units within the AI chip. It receives computation graphs from the host, parses them into executable tasks, and distributes them to appropriate computation units using hardware-based scheduling algorithms, thereby improving coordination efficiency while maintaining system flexibility.
4Productivity
If dedicated AI chips (GPU, FPGA, ASIC) are used, then specific application performance is improved, but adaptability to different application scenarios is limited
Solution Approach 1:
The AI chip is designed with universal computation units that can execute multiple types of neural network operations including convolution, fully connected layers, recurrent neural networks, and attention mechanisms. The integrated task scheduling module can dynamically allocate these universal units to different computation graphs, enabling the chip to adapt to various AI applications without requiring application-specific customization.
Solution Approach 2:
The system implements dynamic task scheduling where the scheduling module can adjust task allocation, resource distribution, and computation unit configuration in real-time based on the characteristics of different computation graphs. This dynamic capability allows the AI chip to optimize performance for different application scenarios while maintaining a single unified architecture.
Data Source
AI summary
Embodiments described herein provide a configurable heterogeneous Artificial Intelligence (AI) processor comprising at least two different architectural types of computation units, a storage unit and a controller. Each of the computation units has a respective task queue. The controller is configured to partition a computation graph of a neural network into a plurality of computation subtasks and distribute the computation subtasks to the task queues of the computation units. The controller is also configured to set a dependency among the computation subtasks, synchronize the computation subtasks according to the set dependency, and control access to data involved in the computation subtasks. Different application tasks are processed by uniformly managing and scheduling the various architectural types of computation units in an on-chip heterogeneous manner, so that the AI processor can flexibly adapt to different application scenarios.