Instruction Generation for Data Processing Arrays and ML Data Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models are computationally intensive to implement efficiently on hardware platforms, particularly on complex systems like data processing arrays, due to challenges in executing machine learning models with acceptable runtime performance.
Innovation Solution
A method is developed to generate tensor-level intermediate representations from machine learning models using kernel expressions, partitioning statements into kernel instructions for compute tiles and microcontroller instructions for data movement, enabling efficient execution on data processing arrays with reconfigurable neural engines and super-graph models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If machine learning models are implemented on data processing arrays, then computational capability is improved, but device complexity increases
Solution Approach 1:
The system segments the data processing array into multiple compute tiles, each containing specialized neural engine circuits. This segmentation allows the complex ML computation task to be divided across multiple simpler, identical units, improving overall computational capability while keeping individual tile complexity manageable.
Solution Approach 2:
A microcontroller unit acts as an intermediary between the host system and the data processing array. It manages data movement, configures neural engine circuits, and coordinates computation across compute tiles, thereby simplifying the interface complexity while enabling sophisticated ML workloads.
2Productivity
If compute functions are optimized for runtime performance, then productivity is improved, but device complexity increases
Solution Approach 1:
The neural engine circuits are designed to be dynamically reconfigurable through the microcontroller unit. This allows the same hardware to adapt its behavior for different ML operations and optimization strategies, achieving high runtime performance without requiring multiple specialized static circuits for each operation type.
Solution Approach 2:
The system optimizes runtime performance by dynamically changing operational parameters such as data flow configurations, computation pipelines, and resource allocation across compute tiles. These parameter changes enable adaptive optimization without adding permanent hardware complexity.
3Loss of time
If data movement is optimized through microcontroller instructions, then loss of time is reduced, but device complexity increases
Solution Approach 1:
The microcontroller unit performs preliminary actions by pre-configuring data movement paths, pre-loading data into compute tile buffers, and pre-synchronizing operations before main computation begins. This reduces actual computation wait time without requiring complex hardware data movement circuits.
Solution Approach 2:
The microcontroller unit autonomously manages data movement between memory and compute tiles, handling addressing, transfer control, and synchronization without requiring complex external memory controllers or dedicated data movement hardware circuits.
Data Source
AI summary
Instruction generation for a data processing array and microcontroller includes generating a tensor-level intermediate representation from a machine learning model using kernel expressions. Statements of the tensor-level intermediate representation are partitioned into a first set of statements and a second set of statements. From the first set of statements, kernel instructions are generated based on a reconfigurable neural engine model. The kernel instructions are executable by a compute tile of a data processing array to implement compute functions of the machine learning model. From the set of second statements, microcontroller instructions are generated based on a super-graph model. The microcontroller instructions are executable by a microcontroller of the data processing array to move data into and out from the data processing array.