Neural Compiler Optimization for Deep Learning Accelerator Instructions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently implementing Artificial Neural Networks (ANNs) on Deep Learning Accelerators (DLAs) due to limitations in computation time and energy consumption, particularly when dealing with large vector and matrix operands, and insufficient data storage capabilities.
Innovation Solution
A compiler is used to optimize instructions for DLAs, leveraging an ANN to predict optimal compilation strategies based on hardware characteristics, utilizing a high-bandwidth connection between the DLA and random access memory to reduce data access bottlenecks and enable autonomous execution of ANN operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional compilation methods are used to generate instructions for Deep Learning Accelerators, then the implementation can be achieved with conventional tools, but computation time and energy consumption increase significantly when dealing with large vector and matrix operands
Solution Approach 1:
The patent changes the fundamental parameter of how compilation decisions are made - transitioning from rule-based compilation to machine learning-based compilation. The DLA compiler uses trained machine learning models to predict optimal compilation strategies, instruction configurations, and resource allocation parameters, enabling significantly faster computation and lower energy consumption for large-scale neural network operations on DLAs
Solution Approach 2:
The patent substitutes the mechanical/rule-based compilation system with an intelligent system using machine learning models. Instead of relying on predetermined compilation rules and manual optimization, the system uses trained neural networks to automatically generate optimized instructions and configurations, replacing conventional compilation mechanics with AI-driven decision making
2Ease of operation
If CPU intervention is used to manage data access and instruction execution, then flexibility and control are maintained, but data access bottlenecks and computation time increase
Solution Approach 1:
The patent implements self-service by enabling the Deep Learning Accelerator to autonomously execute instructions and manage data access without continuous CPU intervention. The DLA compiler generates self-contained instruction sequences that the DLA can execute independently, allowing the accelerator to service itself and reducing dependency on the CPU, thereby eliminating data access bottlenecks and reducing computation time
3Speed
If high-bandwidth connections are implemented between DLA and memory, then data access speed improves, but hardware complexity and cost increase
Solution Approach 1:
The patent applies preliminary action by using the DLA compiler to pre-process and optimize data layouts, instruction sequences, and memory access patterns before execution. The compiler performs preliminary transformations on the neural network computation graph, optimizing data organization and access patterns to maximize utilization of the high-bandwidth connection between DLA and memory, thereby achieving fast data access while managing hardware complexity through software optimization
Data Source
AI summary
Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, an integrated circuit device may be configured to execute instructions with matrix operands and configured with random access memory (RAM). A compiler has an artificial neural network configured to identify an optimized compilation option for an artificial neural network to be compiled by the compiler and/or for a hardware platform of Deep Learning Accelerators. The artificial neural network of the compiler can be trained via machine learning to identify the optimized compilation option based on the features of the artificial neural network to be compiled and/or features of the hardware platform on which the compiler output will be executed.