Heterogeneous Computing Task Scheduling for AI Edge Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial intelligence algorithms require significant computing power, which CPUs alone cannot provide, especially for edge-side applications, necessitating the use of heterogeneous hybrid acceleration modules like GPUs, FPGAs, and ASICs to enhance processing capabilities.
Innovation Solution
A heterogeneous computing-based task processing method and framework that breaks down AI tasks into sub-tasks, utilizing a hierarchical data flow and analysis function unit services to manage and schedule multiple units efficiently, including GPUs, FPGAs, and ASICs, with features like RDMA and HAL for unified data storage and transportation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If heterogeneous hybrid acceleration modules (GPUs, FPGAs, ASICs) are adopted to enhance computing power, then processing capability is improved, but system complexity increases
Solution Approach 1:
The system segments heterogeneous computing units into standardized functional modules that can be independently managed and scheduled. Each acceleration module (GPU, FPGA, ASIC) is divided into discrete task-execution units that interface through standardized protocols, reducing the complexity of managing heterogeneity while preserving computing power benefits
Solution Approach 2:
A unified task scheduling and management system acts as an intermediary layer between diverse heterogeneous units and application tasks. This mediator translates various task types into standardized operations that different acceleration modules can execute, simplifying the interface complexity while maintaining high computing power utilization
2Productivity
If multiple heterogeneous units are deployed to process AI tasks, then task execution efficiency is improved, but model loading time increases
Solution Approach 1:
The system performs preliminary actions by pre-loading and caching model components into a shared memory pool before they are actually needed for task execution. When multiple heterogeneous units require the same model, it is already available in the shared pool, eliminating repeated loading delays and improving both execution efficiency and reducing time loss
Solution Approach 2:
The patent merges the model storage and loading functionality across multiple heterogeneous units into a unified shared memory system. Instead of each unit maintaining separate model copies, they share a common model pool that reduces redundant loading operations while maintaining fast access for all units simultaneously
3Productivity
If heterogeneous units share resources to improve efficiency, then resource utilization is improved, but service stability decreases
Solution Approach 1:
The system implements flexible isolation mechanisms (analogous to thin films) that separate service contexts while allowing controlled resource sharing. Each service runs in an isolated context with defined resource allocations, preventing interference between services while still enabling efficient sharing of underlying heterogeneous computing resources through the unified scheduling layer
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A heterogeneous computing-based task processing method and a software-hardware framework system. The task processing method comprises: breaking down an artificial intelligence analysis task into sub-tasks at one stage or multiple stages (301); and completing the artificial intelligence analysis task by means of one or more analysis function unit services corresponding to the sub-tasks at one stage or multiple stages using a hierarchical data flow (302), wherein the types of sub-tasks at different stages are different, one type of sub-task corresponds to one analysis function unit service, and each analysis function unit service uniformly schedules multiple heterogeneous units to execute a corresponding sub-task.