Neural Network Task Manager for Heterogeneous Resource Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural network systems face inefficiencies in processing tasks due to the lack of effective management of heterogeneous resources, leading to suboptimal performance and processing speed.
Innovation Solution
A system and method that utilize a model analyzer to generate sub-graph information and a task manager to output request signals for heterogeneous resources, allowing for efficient task partitioning and distribution across resources, including specific task-dedicated and general-purpose resources, to optimize task processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tasks are processed using heterogeneous resources without effective management, then resource utilization is low, but processing speed and performance are suboptimal
Solution Approach 1:
The patent segments tasks into sub-graphs and partitions them across heterogeneous resources. The model analyzer divides the neural network model into sub-graphs, and the task manager allocates these sub-graphs to appropriate resources (GPU, NPU, CPU) based on their capabilities, enabling parallel processing and improving throughput without overwhelming management complexity
Solution Approach 2:
The patent introduces a task manager as an intermediary component that mediates between the model analyzer and heterogeneous resources. This intermediary handles resource allocation, task scheduling, and coordination, abstracting the complexity of managing multiple resource types while optimizing processing speed and resource utilization
2Productivity
If tasks are allocated to resources without dynamic allocation, then resource utilization is simple, but operation efficiency decreases
Solution Approach 1:
The patent implements dynamic resource allocation where the task manager continuously monitors resource availability and task characteristics, adjusting allocations in real-time. Resources can be dynamically assigned or reassigned based on current system state, improving operation efficiency while maintaining adaptability to changing workloads
3Productivity
If batch mode processing is enforced, then processing throughput increases, but resources that do not support batch mode cannot be utilized
Solution Approach 1:
The patent creates a universal task management system that can handle both batch mode and non-batch mode resources. The task manager detects resource capabilities and adapts its allocation strategy accordingly, allowing batch processing for resources that support it (like GPU) while individually processing tasks on resources that don't (like certain NPUs), thereby utilizing all heterogeneous resources effectively
Data Source
AI summary
An artificial neural network system for managing a task to be performed by heterogeneous resources executing an artificial neural network, the artificial neural network system including a model analyzer that receives an artificial neural network model and outputs sub-graph information generated based on the artificial neural network model including at least one of sub-graph, a detector that outputs awareness information about the heterogeneous resources, and a task manager that outputs a first request signal for performing a task with respect to each layer of first resource of the heterogeneous resources based on the sub-graph information and the awareness information, and a second request signal for performing an task with respect to each depth of a second resource of the heterogeneous resources.