Graph-Based ANN Conversion with Calibration for NPU Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network models face issues of high-power consumption, heat generation, memory bandwidth bottlenecks, latency, and computational inefficiencies, particularly in edge devices, leading to reduced performance and difficulty in optimizing neural processing units (NPUs) for various electronic devices.
Innovation Solution
A method to convert non-graph based neural network models into graph-based models by converting functions into graph modules, analyzing input-output relationships, generating a directed acyclic graph (DAG), adding markers, and determining scale and offset values to optimize the model for low-power neural processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural network models are converted to graph-based models with DAG structure, then computational efficiency and NPU optimization are improved, but model complexity and conversion process difficulty increase
Solution Approach 1:
The neural network model is segmented into discrete graph modules representing computational operations, with explicit input-output relationships mapped as edges. This segmentation enables systematic optimization for NPU while maintaining computational equivalence.
Solution Approach 2:
A conversion process acts as an intermediary between the original neural network model and the optimized graph-based model. This intermediary systematically transforms the model structure, analyzes input-output relationships, and generates the optimized DAG representation.
2Adaptability or versatility
If graph modules are added to represent neural network functions, then NPU optimization capability is improved, but device complexity and processing steps increase
Solution Approach 1:
The conversion process performs preliminary analysis of input-output relationships and generates graph module representations before the actual NPU optimization. This preliminary structuring simplifies subsequent optimization steps and enables systematic NPU selection and configuration.
3Measurement precision
If calibration data collection and scale/offset determination are performed, then model accuracy on specific NPU is improved, but processing time and computational overhead increase
Solution Approach 1:
The method determines optimal scale and offset parameters by analyzing calibration data from graph modules. These parameter transformations enable the model to achieve high accuracy on specific NPU hardware while maintaining efficiency through systematic parameter optimization rather than brute-force search.
Data Source
AI summary
A method comprises: converting a plurality of functions or function call instructions of a first neural network (NN) model into a plurality of graph modules; analyzing a relationship between one or more inputs and one or more outputs of the plurality of graph modules; generating a second NN model in a form of a directed acyclic graph (DAG) using the plurality of graph modules corresponding to the first NN model, by mapping the one or more inputs and the one or more outputs of the plurality of graph modules to each other based on the relationship; adding a plurality of markers to the plurality of graph modules in the second NN model; generating calibration data by collecting input values and output values of each of the plurality of graph modules using the plurality of markers; and determining a scale value and an offset value applicable to the second NN model.