Hierarchical ML Accelerator Compilation for Multi-Core Job Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems face inefficiencies in implementing computations, particularly at the edge, due to the lack of optimized distribution and execution of jobs across diverse processing cores in multi-core computing devices.
Innovation Solution
A hierarchical compiler and execution method that analyzes jobs to assign them to suitable processing cores, generating an execution graph that optimizes job distribution and execution across ARM processors, TPUs, and CPUs, utilizing a hierarchical architecture to facilitate efficient and low-power operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If jobs are distributed across diverse processing cores without optimized analysis, then device complexity is reduced, but productivity decreases due to inefficient job execution
Solution Approach 1:
The system performs preliminary analysis of jobs to determine suitable processing cores before execution. The compilation process analyzes job characteristics and pre-determines the optimal core assignment, creating an execution graph that maps jobs to cores in advance. This preliminary action ensures efficient job execution without runtime decision-making overhead.
Solution Approach 2:
The compilation process is segmented into multiple hierarchical levels: a top-level compiler that creates an initial execution graph, and device-level compilers that refine the graph for specific multi-core devices. This segmentation allows the system to manage complexity by dividing the compilation task into manageable stages, each handling specific aspects of job distribution.
2Use of energy by moving object
If communication between CPU and ASICs is minimized through optimized job distribution, then power usage decreases, but device complexity increases due to hierarchical compilation requirements
Solution Approach 1:
The system introduces a hierarchical dimension to the compilation process, with multiple levels of compilation (top-level and device-level) operating at different stages. This dimensional approach to compilation organization enables the system to minimize CPU-ASIC communication during execution by pre-resolving job-to-core mappings, thereby reducing runtime power consumption despite increased compilation complexity.
3Productivity
If jobs are assigned to specific core types through analysis, then productivity improves through optimized execution, but loss of time increases due to analysis and compilation overhead
Solution Approach 1:
The system performs job analysis and core assignment as preliminary actions during the compilation phase, before actual job execution. By determining the optimal core mapping in advance and creating a detailed execution graph, the system eliminates runtime analysis overhead and ensures efficient execution without sacrificing productivity.
Solution Approach 2:
The compilation process is designed to be adaptive and dynamic, adjusting the depth of analysis based on job characteristics and available resources. The system can dynamically balance between analysis thoroughness and compilation time, allowing flexible optimization depending on the specific execution context and requirements.
Data Source
AI summary
This disclosure describes a system and method for compiling and executing machine learning inferences in an array of multi-core computing devices. Each multi-core computing device can be an application specific integrated circuit (ASIC) or group of ASICS. In many applications, the array of computing devices changes from inference to inference, and can be adjusted based on the requirements of the inference. Additionally, each ASIC can have multiple processing cores, and multiple types of processing cores. Therefore, performing optimizations and scheduling at compile time, can dramatically increase the efficiency of the array in executing the inference. In some implementations, it is possible to select an amount of time or effort to be spent optimizing during compiling, giving the user flexibility in determining whether to spend time during compilation or during execution.