Neural Network Profiling for Parallel Heterogeneous Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for processing neural networks on electronic devices lead to overutilization of pre-defined processors and underutilization of unused processors due to sequential processing, limiting flexibility and efficiency across heterogeneous processors.
Innovation Solution
A unified neural network profile is created for all processors, incorporating metadata and individual processor profiles, allowing parallel processing and pipelining based on task type to optimize resource allocation across heterogeneous processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sequential processing is used for neural network layers on pre-defined processors, then processing order is maintained, but processor utilization becomes inefficient with overutilization of selected processors and underutilization of unused processors
Solution Approach 1:
The system dynamically selects processors for processing neural network layers based on current device state and task requirements, rather than using fixed pre-defined processor assignments. The processor selection is made flexible and adaptive, allowing the system to choose from multiple available processors (CPU, GPU, NPU, DSP) depending on workload, hardware capabilities, and task characteristics, thereby improving both productivity and adaptability.
Solution Approach 2:
The system implements a unified neural network profile that can be executed across multiple types of processors (CPU, GPU, NPU, DSP), making the neural network framework universal and not framework-dependent. This multi-functionality allows the same neural network to be processed on different processor types, enabling efficient resource utilization across heterogeneous hardware while maintaining processing flexibility.
2Productivity
If neural network information is compiled separately for each processor and task, then processor-specific optimization is achieved, but cross-processor and cross-task information reuse is prevented
Solution Approach 1:
The system creates a unified neural network profile that serves as a universal representation of the neural network structure and parameters, which can be executed on multiple processor types (CPU, GPU, NPU, DSP) and for multiple tasks. This unified profile eliminates the need for separate compilations for each processor and task, reducing management complexity while maintaining processor-specific optimization through selective execution.
Solution Approach 2:
The system merges the neural network information into a single unified profile that combines characteristics and parameters relevant to multiple processors and tasks. This consolidation replaces the separate compilation approach, reducing the overall complexity of managing multiple processor-specific and task-specific profiles while enabling efficient information reuse across different processors and tasks.
3Ease of operation
If pre-defined processors are used for specific tasks, then task-specific processing is optimized, but flexibility to use other available processors is lost
Solution Approach 1:
The system employs dynamic processor selection where the choice of processor (CPU, GPU, NPU, DSP) is made based on current device state, task requirements, and available hardware capabilities rather than being fixed in advance. This dynamic approach maintains ease of operation for task-specific processing while simultaneously providing flexibility to select from multiple available processors, resolving the contradiction between operational simplicity and adaptability.
Data Source
AI summary
A method, an apparatus, and a system for configuring a neural network across heterogeneous processors are provided. The method includes creating a unified neural network profile for the plurality of processors; receiving at least one request to perform at least one task using the neural network; determining a type of the requested at least one task as one of an asynchronous task and a synchronous task; and parallelizing processing of the neural network across the plurality of processors to perform the requested at least one task, based on the type of the requested at least one task and the created unified neural network profile.