Neural Network Model Partitioning for Heterogeneous Accelerator Deployment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware-based and software-based design methods for neural networks in heterogeneous computing platforms face issues such as low resource utilization, lack of flexibility, and compatibility with new hardware, leading to inefficient execution and maintenance.
Innovation Solution
A method that partitions a neural network model into sub-models based on operational characteristics, verifies their performance, and deploys them on specific accelerators to optimize execution, using a heterogeneous graph to connect and execute sub-models for optimal performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hardware-based design method is used to improve neural network performance, then execution efficiency is improved, but resource utilization rate becomes low and hardware cost increases
Solution Approach 1:
The patent segments the neural network model into multiple sub-models based on operation types (e.g., convolutional layers, fully connected layers, activation functions). Each sub-model is independently deployed to the most suitable processor type (GPU, NPU, CPU), avoiding the need to allocate entire hardware resources to a single monolithic model and improving overall resource utilization while maintaining execution efficiency.
2Productivity
If models are directly programmed for specific hardware to improve execution performance, then execution speed is improved, but flexibility is reduced and re-programming is needed when models change
Solution Approach 1:
The patent implements a dynamic model deployment architecture where the system automatically analyzes the operational characteristics of different neural network models and dynamically assigns sub-models to appropriate processors. This dynamic allocation mechanism allows the system to adapt to different models without manual re-programming, maintaining high execution speed while providing the flexibility to handle model changes and updates.
3Adaptability or versatility
If software-based design method using OpenCL is used to enable cross-platform execution, then compatibility is improved, but device complexity increases and not all hardware is supported
Solution Approach 1:
The patent introduces an intermediary model analysis and assignment module that acts as a mediator between the neural network model and the heterogeneous processors. This intermediary automatically analyzes model characteristics, determines the most suitable processor types for each sub-model, and manages the deployment without requiring complex cross-platform programming. This reduces software architecture complexity while maintaining broad hardware compatibility through automatic adapter selection.
Data Source
AI summary
A method and apparatus for implementing a neural network model in a heterogeneous computing platform are disclosed. The method includes partitioning a neural network model into first sub-models based on a partition standard, obtaining second sub-models by merging at least a portion of the first sub-models based on characteristics of the first sub-models, and deploying the second sub-models.