Runtime Library ANN for DLA Execution Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compiler technologies for Artificial Neural Networks (ANNs) on Deep Learning Accelerators (DLAs) face challenges in optimizing energy consumption and computation time, as they rely solely on the ANN description without considering the dynamic patterns of input data, leading to suboptimal performance.
Innovation Solution
A compiler with an embedded Artificial Neural Network (ANN) in the runtime library that analyzes current data patterns to predict and dynamically adjust execution optimizations, optimizing instruction execution on the DLA by selecting the best hardware options and execution paths based on the input data patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compiler optimization is based solely on ANN description, then compilation process is simple, but energy consumption and computation time cannot be optimized
Solution Approach 1:
The runtime library contains an embedded ANN that autonomously analyzes input data patterns and selects optimized execution paths without external intervention. The system serves itself by using the embedded ANN to make runtime decisions about computation optimization based on actual data characteristics.
Solution Approach 2:
The compilation process transitions from static (based only on ANN description) to dynamic (based on actual input data patterns). The embedded ANN in the runtime library adapts the execution strategy according to real-time data analysis, enabling dynamic optimization of computation paths.
2Adaptability or versatility
If fixed execution paths are used, then device complexity is low, but adaptability to different data patterns is poor
Solution Approach 1:
The runtime library is designed with multi-functionality, incorporating both the original ANN execution functions and an embedded ANN for pattern analysis and optimization selection. This universal runtime library can handle various data patterns and execution scenarios through a single integrated system.
Solution Approach 2:
The embedded ANN acts as an intermediary between the input data and the execution paths. It analyzes data patterns and mediates the selection of appropriate execution strategies, enabling flexible adaptation without direct complex interactions between data and execution hardware.
3Use of energy by moving object
If dynamic optimization based on data patterns is implemented, then energy consumption decreases, but computation time for pattern analysis is added
Solution Approach 1:
The system performs partial pattern analysis only when necessary, using the embedded ANN to identify optimization opportunities selectively rather than continuously analyzing all data. This reduces unnecessary computation time while still achieving significant energy savings through targeted optimization.
Data Source
AI summary
Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, an integrated circuit device may be configured to execute instructions with matrix operands and configured with random access memory (RAM). A compiler is configured to generate instructions executable by the Deep Learning Accelerator from a description of a target artificial neural network. The instructions may call routines in a runtime library that has an embedded artificial neural network configured to predict optimized execution options available to implement the routines. The prediction is based at least in part on a pattern of data being processed in the target artificial neural network and/or a pattern of usages of the routines by the instructions.