AI Inference Code Generation via DSL Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI inference systems in embedded environments require significant time and effort to adapt to different hardware environments, leading to high maintenance costs and reduced reusability due to their dependency on specific acceleration hardware.
Innovation Solution
An AI inference method that converts neural network applications into executable code in a high-level language independent of learning frameworks, separating the code into GPL and DSL components, and generating target code optimized for various hardware environments, including CPUs and accelerators, using DSL separation rules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If the inference system is designed to be optimized for specific hardware acceleration environments, then operation performance is improved, but device complexity and maintenance cost increase due to hardware dependency
Solution Approach 1:
The patent segments the inference system into three distinct modules: a learning framework-independent front-end that converts applications to executable code, a domain-specific language (DSL) layer that separates hardware-independent logic from hardware-specific operations, and a code generation module that targets specific hardware. This segmentation allows performance optimization for specific hardware while maintaining reusability through the modular architecture, directly resolving the contradiction between performance and complexity.
Solution Approach 2:
The patent introduces a domain-specific language (DSL) as an intermediary layer between the high-level application logic and hardware-specific implementations. The DSL captures computation graphs and operations in a hardware-agnostic manner, then translates them to hardware-optimized code. This intermediary enables the system to achieve hardware-specific performance optimization without requiring the entire system to be hardware-dependent, thus reducing complexity while maintaining performance.
2Productivity
If various types of accelerators are used simultaneously to handle parallel computational load, then processing capability is improved, but ease of operation deteriorates due to the need to construct models optimized for each hardware environment
Solution Approach 1:
The patent creates a universal front-end conversion module that can handle multiple learning frameworks (TensorFlow, PyTorch, Caffe) and translate them into a unified executable code format. This universal converter, combined with the hardware-agnostic DSL layer, allows the same high-level application to be deployed across different accelerator types (CPU, GPU, FPGA, proprietary accelerators) without requiring separate optimized models for each, thus maintaining ease of operation while supporting diverse hardware for enhanced processing capability.
Solution Approach 2:
The patent implements dynamic code generation that adapts to the target hardware environment at runtime or compile-time. The code generation module analyzes the available accelerators and dynamically generates optimized target code from the hardware-independent DSL representation. This dynamic adaptation allows the system to utilize various accelerator types simultaneously or selectively based on availability, maintaining ease of operation while maximizing processing capability across different hardware configurations.
3Power
If the inference engine is designed to be hardware-dependent, then operation performance is improved, but adaptability worsens due to the need to redesign for each hardware change
Solution Approach 1:
The patent segments the inference engine into hardware-independent components (front-end converter, DSL compiler) and hardware-specific components (code generation module, target code). The DSL layer serves as a stable intermediate representation that preserves application logic independently of hardware changes. When hardware changes occur, only the code generation module needs to be updated to generate new target code, while the majority of the inference engine remains unchanged, thus maintaining both performance and adaptability.
Solution Approach 2:
The domain-specific language (DSL) acts as an intermediary that decouples the application logic from hardware specifics. The DSL captures the computation graph and operations in a hardware-agnostic format, serving as a stable intermediate layer. This intermediary allows the inference engine to maintain high performance through hardware-optimized code generation while achieving adaptability through the hardware-independent DSL representation, enabling easy adaptation to new hardware without redesigning the entire system.
Data Source
AI summary
An embodiment relates to an artificial intelligence inference apparatus and method. The embodiment provides an artificial intelligence inference method, and may include converting an application based on a previously learned neural network into executable code in a high-level language independent of a learning framework, separating the executable code into General-Purpose Language (GPL) code and Domain-Specific Language (DSL) code depending on whether an acceleration operation is required, and generating target code optimized for hardware from the separated GPL code and DSL code.