Neural Network Model Conversion for Streaming Architecture Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models developed under conventional instruction set architectures face inefficiencies when deployed on streaming architectures due to mismatched operator granularities, leading to suboptimal computation efficiency and limited customization for deep learning applications.
Innovation Solution
A neural network model conversion method that parses and reconstructs initial model information to generate target model information suitable for streaming architectures, enabling efficient deployment and operation of neural network models on streaming architecture-based chips.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If neural network models are developed under conventional instruction set architecture, then development is straightforward with fine-grained operators, but computation efficiency is low when deployed on streaming architecture
Solution Approach 1:
The patent changes the operator granularity parameter from fine-grained (instruction set level) to coarse-grained (basic layer operator level) through automatic conversion. This parameter transformation enables the model to adapt from conventional architecture to streaming architecture, resolving the contradiction between development ease and computation efficiency.
Solution Approach 2:
The patent introduces an automatic model conversion tool as an intermediary between conventional instruction set architecture and streaming architecture. This intermediary automatically transforms models developed under conventional architecture into formats suitable for streaming architecture, eliminating the need for manual redevelopment while enabling efficient execution.
2Adaptability or versatility
If operator granularity is at tensor operator level in conventional instruction set architecture, then model development is flexible, but computation power is insufficient for various application scenarios
Solution Approach 1:
The patent merges multiple fine-grained tensor operators into coarse-grained basic layer operators during the conversion process. This merging operation consolidates computation tasks into larger operational units that better utilize the streaming architecture's computation power while maintaining the original model's functional flexibility.
3Productivity
If streaming architecture with larger operator granularity is used, then computation efficiency is higher, but additional processing is required due to operator granularity mismatching
Solution Approach 1:
The conversion tool performs self-service by automatically analyzing the original model structure, identifying operator granularity mismatches, and performing the necessary transformations without requiring manual intervention. This automation reduces the perceived complexity for users while achieving high computation efficiency.
Solution Approach 2:
The patent performs the model conversion action in advance before deployment to the streaming architecture. By completing the granularity alignment transformation beforehand, the system eliminates the need for complex runtime processing and ensures optimal computation efficiency during actual execution.
Data Source
AI summary
A neural network model conversion method, a server, and a storage medium are provided according to embodiments of the present disclosure. The neural network model conversion method includes: parsing a neural network model to obtain initial model information; reconstructing the initial model information to obtain streaming model information; generating a target model information file according to the streaming model information; and running, under a streaming architecture, the neural network model according to the target model information file.