AI Accelerator Instruction Generation With Joint IR Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compilers for AI accelerators lack correlation and collaboration between key steps, resulting in suboptimal machine instructions that do not fully utilize the performance of the AI accelerator.
Innovation Solution
A method and device that consider hardware resources during the compilation process to generate AI accelerator-executable instructions by converting high-level intermediate representation sequences into low-level sequences, performing scheduling and resource allocation, and selecting optimal execution orders to maximize hardware utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If key compilation steps are run and resolved separately, then the compilation process is simpler and easier to implement, but the final machine instruction is not a global optimal solution and AI accelerator performance cannot be fully utilized
Solution Approach 1:
The patent merges multiple independent compilation steps (code generation, instruction scheduling, resource allocation) into a unified joint optimization process. The compilation device integrates these modules to perform coordinated optimization, where code generation considers instruction scheduling constraints, and resource allocation is performed simultaneously with both, enabling global optimal machine instruction generation that fully utilizes AI accelerator performance.
2Productivity
If joint optimization of code generation, instruction scheduling and resource allocation is performed, then global optimal machine instruction is achieved and AI accelerator performance is fully utilized, but the compilation process becomes more complex
Solution Approach 1:
The patent segments the joint optimization process into distinct functional modules: a code generation module that generates initial instruction sequences, an instruction scheduling module that optimizes execution order, and a resource allocation module that assigns hardware resources. These segmented modules work in coordinated sequence, making the complex joint optimization process manageable while achieving global optimal results.
Solution Approach 2:
The patent introduces intermediate representation (IR) as a mediator between the high-level neural network model and the low-level machine instructions. The code generation module converts the model to IR, which then serves as input for instruction scheduling and resource allocation. This intermediary representation simplifies the overall compilation process by providing a standardized intermediate form that facilitates coordinated optimization across all modules.
Data Source
Figure 1
Figure 2
Figure 3~5
AI summary
Disclosed are a method, device and electronic apparatus for generating instructions of an artificial intelligence accelerator. The method includes: determining a first intermediate representation instruction sequence corresponding to a neural network model that needs to be processed by an artificial intelligence AI accelerator; converting, based on a hardware resources of the AI accelerator, the first intermediate representation instruction sequence into one or more second intermediate representation instruction sequences respectively by using one or more code generation manners; and generating an AI accelerator-executable instruction based on the hardware resources of the AI accelerator and the one or more second intermediate representation instruction sequences. The neural network model is thus compiled to generate the AI accelerator-executable instruction on the basis of fully considering the hardware resources of the AI accelerator. Therefore, the finally generated AI accelerator-executable instruction is enabled to better utilize performance of the AI accelerator, thereby improving user experience.