Unified Backend Engine for Deep Learning Framework Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Domestic AI chips face challenges in computing performance and adaptability due to incompatibility with mainstream deep learning frameworks, hindering the development of self-controlled equipment and intelligent application scenarios.
Innovation Solution
A method for adapting a deep learning framework to a hardware device using a unified backend engine, which involves registering the hardware device, adding a compiler, converting computational graphs into intermediate representations, compiling, and managing memory to enable seamless execution on the hardware device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a unified backend engine is introduced to improve hardware adaptability, then compatibility with domestic AI chips is improved, but system complexity increases due to additional compilation and conversion layers
Solution Approach 1:
The unified backend engine serves as an intermediary layer between the deep learning framework and hardware devices. It receives computational graphs from the framework, converts them to intermediate representations, and compiles them for execution on various hardware platforms. This mediator approach enables broad hardware compatibility while maintaining a clean interface with the framework, resolving the contradiction between adaptability and complexity.
Solution Approach 2:
The system is segmented into distinct functional modules: the deep learning framework layer, the unified backend engine layer (with graph conversion and compilation sub-functions), and the hardware execution layer. This segmentation allows each component to be optimized independently and facilitates easier maintenance and adaptation to new hardware, managing overall system complexity.
2Productivity
If computational graphs are converted to intermediate representations and compiled on hardware devices, then execution efficiency is improved, but compilation time and processing overhead increase
Solution Approach 1:
The unified backend engine performs preliminary conversion of computational graphs to intermediate representations and pre-compilation to executable code before runtime execution. This preliminary processing enables optimized execution on target hardware while separating the compilation overhead from the actual inference time, improving execution efficiency at the cost of upfront compilation time.
Data Source
AI summary
Disclosed is a method for adapting a deep learning framework to a hardware device based on a unified backend engine, which comprises the following steps: S1, adding the unified backend engine to the deep learning framework; S2, adding the unified backend engine to the hardware device; S3, converting a computational graph, wherein the computational graph compiled and generated by the deep learning framework is converted into an intermediate representation of the unified backend engine; S4, compiling the intermediate representation, wherein the unified backend engine compiles the intermediate representation on the hardware device to generate an executable object; S5, running the executable object, wherein the deep learning framework runs the executable object on the hardware device; S6: managing memory of the unified backend engine.