Computational Graph Interface for Heterogeneous AI Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning frameworks face challenges in adapting to diverse heterogeneous hardware due to differences in instruction set IP and chip architecture, leading to high development costs and suboptimal execution efficiency.
Innovation Solution
The method involves optimizing a computational graph based on access device information and generating a universal interface for different hardware types, allowing for efficient operation of deep learning models across various hardware platforms by abstracting and encapsulating common interfaces and optimizing computational graph units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep learning frameworks are adapted to diverse heterogeneous hardware through hardware-specific implementations, then execution efficiency on each hardware platform is improved, but development cost and system complexity increase significantly
Solution Approach 1:
The patent implements a universal computational graph interface that can be executed across multiple heterogeneous hardware platforms (CPU, GPU, NPU, FPGA) without requiring separate hardware-specific implementations. The computational graph serves as a hardware-agnostic intermediate representation that enables the same deep learning model to run efficiently on different hardware through a unified execution engine, thereby reducing development costs while maintaining execution efficiency.
Solution Approach 2:
The patent introduces a computational graph as an intermediary layer between the deep learning framework and heterogeneous hardware devices. This computational graph acts as a mediator that translates high-level model operations into hardware-specific execution plans, allowing the framework to interact with diverse hardware through a standardized interface while still leveraging hardware-specific optimizations when needed.
2Productivity
If hardware-specific optimizations are implemented for each access device, then execution efficiency on that device is improved, but adaptability to new hardware types deteriorates
Solution Approach 1:
The computational graph execution engine is designed with universal hardware support, allowing it to execute computational graphs on any heterogeneous hardware platform through a unified interface. The system maintains hardware-agnostic computational logic while supporting multiple hardware types (CPU, GPU, NPU, FPGA), enabling easy adaptation to new hardware without requiring hardware-specific code modifications.
Solution Approach 2:
The patent implements dynamic hardware selection and execution plan generation based on the target device characteristics. The system can dynamically adapt the execution strategy by analyzing hardware capabilities at runtime and selecting appropriate execution paths, allowing it to efficiently support both existing and new hardware types without sacrificing execution efficiency on any specific platform.
3Productivity
If computational graphs are optimized for specific hardware architectures, then performance on that architecture is improved, but portability across different hardware platforms deteriorates
Solution Approach 1:
The patent segments the deep learning execution pipeline into distinct components: a hardware-agnostic computational graph construction phase and a hardware-specific execution optimization phase. The computational graph itself remains portable and hardware-independent, while execution optimizations are applied separately during the execution phase based on target hardware characteristics, thereby maintaining both portability and performance.
Solution Approach 2:
The computational graph serves as a portable intermediary representation that preserves the model's computational logic across different hardware platforms. Hardware-specific optimizations are applied to the execution of this intermediate representation rather than to the model definition itself, allowing the same computational graph to be executed efficiently on various hardware architectures without sacrificing portability.
Data Source
AI summary
The disclosure provides an access method, an access apparatus, an electronic device and a computer storage medium, and relates to a field of computer technologies, in particular to a field of artificial intelligence technologies such as chip and deep learning. The method includes: determining a computational graph for calling an access device based on operator representations in a target model; optimizing the computational graph based on information of the access device; and performing relevant running operations of the target model on the access device based on the computational graph and an interface for the access device to access to a model framework of the target model, the interface being determined based on kit data of the access device.