Neural Network Model Compilation for Processor-Specific Data Formats
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network models have fixed input data shapes, leading to low flexibility in data processing.
Innovation Solution
Obtain the data format of input data, perform mapping processing based on the neural network model's operator to determine a data sub-format, and compile an initial model file using attribute information of the processing circuitry to create a final model file that adapts to the processor's capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the input data shape is fixed for neural network model processing, then the processing is simple and straightforward, but the flexibility in handling various data formats is reduced
Solution Approach 1:
The patent applies preliminary action by performing data format mapping and model file compilation before actual neural network inference. The system pre-processes the input data to match the expected data sub-format of the operator, and pre-compiles the model file according to processor attributes, thereby avoiding complex format conversions during runtime and achieving flexible handling of various data formats without increasing runtime complexity
Solution Approach 2:
The patent introduces an intermediary data sub-format that bridges the gap between the original input data format and the fixed expected format of the neural network operator. This intermediate representation allows the system to accept various input formats while maintaining compatibility with the operator's requirements, effectively resolving the contradiction between adaptability and processing simplicity
2Productivity
If manual intervention is required for data format processing, then processing accuracy can be ensured, but processing efficiency is reduced
Solution Approach 1:
The patent implements self-service by enabling the system to automatically perform data format mapping and model compilation without manual intervention. The mapping module automatically determines the appropriate data sub-format based on the operator type and input data characteristics, and the compilation module automatically generates the processor-specific model file, thereby maintaining high processing accuracy while significantly improving productivity
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting data format parameters (such as data type, dimension, and layout) based on the operator requirements and processor capabilities. This automated parameter adaptation ensures that the data format processing maintains the same level of accuracy as manual intervention would provide, while eliminating the time-consuming manual configuration process
Data Source
AI summary
In a data processing method, an indication of a data format of input data of a neural network model is obtained. An initial model file for the neural network model is obtained, the initial model file indicating an operator of the neural network model. A data sub-format of input data of the operator is obtained by processing circuitry based on both the data format and the operator. Attribute information of the processing circuitry is obtained. A final model file for the processing circuitry is obtained based on compiling the initial model file according to both the attribute information and the data sub-format. A processing result is obtained based on running the final model file, by the processing circuitry, with data to be processed by the neural network model. Apparatus and non-transitory computer-readable storage medium counterpart embodiments are also contemplated.