Deep Learning Model Conversion for Heterogeneous Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Universal deep learning models are not customizable to meet specific user requirements, and generated models are often limited to specific hardware devices, making them incompatible with other devices.
Innovation Solution
A method and apparatus for generating a deep learning prediction model by converting existing models to be applicable on target devices, using topology descriptions and device information to adjust parameters and operators, and providing software development kits for model prediction interfaces, enabling model compression and heterogeneous computing chip acceleration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a universal deep learning model is used, then it can meet general AI requirements, but it cannot meet user customized requirements for specific scenarios
Solution Approach 1:
The patent applies parameter changes by allowing users to customize model parameters such as precision (e.g., FP32, FP16, INT8), model structure (adding/removing layers), and operational characteristics. This enables the same model framework to adapt to different specific scenarios by modifying these parameters rather than creating entirely new models, thus improving adaptability while controlling complexity.
2Adaptability or versatility
If a model is generated using existing model generation tools, then it can be applied to specific hardware devices, but it cannot be applied to devices other than the specific hardware devices
Solution Approach 1:
The patent implements universality by creating a model that can run on multiple different hardware devices (CPU, GPU, NPU, DSP, etc.) through a unified model structure. The system generates device-specific code from a single model definition, allowing the same model to be deployed across heterogeneous hardware platforms without requiring separate model generation for each device type.
Solution Approach 2:
The patent uses an intermediary approach by introducing a model conversion system that acts as a mediator between the universal model definition and device-specific implementations. This conversion system translates the model into device-appropriate formats and optimizations, eliminating the need for users to manually convert models for each hardware platform.
3Productivity
If model parameters are reduced in precision, then model size and computing resources are reduced, but prediction accuracy may be affected
Solution Approach 1:
The patent applies parameter changes by systematically adjusting precision parameters (FP32, FP16, INT8) to find the optimal balance between inference speed and accuracy. The system allows dynamic selection of precision levels based on hardware capabilities and performance requirements, enabling users to trade off accuracy for speed when necessary while maintaining high accuracy when resources are available.
Data Source
AI summary
A method and apparatus for generating a model, and a method and apparatus for recognizing information are provided. An implementation of the method for generating a model includes: acquiring a to-be-converted model, a topology description of the to-be-converted model, and device information of a target device; converting, based on the topology description and the device information, parameters and operators of the to-be-converted model to obtain a converted model applicable to the target device; and generating a deep learning prediction model based on the converted model. This embodiment enables the conversion of an existing model to a deep learning prediction model that can be applied to a target device.