External Compiler Strategy Files for Layer-Specific ML Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compilers lack the ability to optimize code for complex ML models effectively, particularly for array-based inference engines with many processing tiles and on-chip memories, due to limitations in user customization and sub-optimal data layout, leading to inefficiencies in latency and throughput.
Innovation Solution
A compiler generates an external strategy file that allows users to modify high-level instructions, enabling customization of data mapping, tensor splitting, and other optimizations, which can be saved in formats like XML or JSON, and used to regenerate low-level executable instructions based on user modifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the compiler provides fixed optimization options developed by the compiler developer, then the compilation process is simplified, but the user lacks the ability to modify particular layers and fine-tune optimizations
Solution Approach 1:
The patent introduces an external strategy file as an intermediary between the compiler and the user's customization needs. This file allows users to modify optimization strategies without changing the compiler itself, thus maintaining ease of operation while enabling adaptability. The external strategy file serves as a mediator that bridges the fixed compiler options and user-specific optimization requirements.
Solution Approach 2:
The patent makes the optimization strategy dynamic by allowing external modification of the strategy file. Instead of fixed compiler options, the system enables users to dynamically adjust optimization parameters for specific layers by editing the external strategy file, which is then recompiled. This transforms a static compilation process into a dynamic one that can adapt to user needs.
2Productivity
If the compiler optimizes overall code automatically, then productivity is improved, but the user cannot fine-tune optimizations for specific layers
Solution Approach 1:
The patent segments the optimization process by allowing users to target specific layers within the neural network for optimization. Instead of treating the entire model uniformly, the external strategy file enables layer-by-layer optimization control. Users can identify particular layers that require specialized optimization and apply targeted strategies to those segments while leaving other layers unchanged.
Solution Approach 2:
The patent applies local quality by enabling different optimization strategies for different layers of the neural network. Each layer can have its own customized optimization parameters in the external strategy file, allowing users to apply the most appropriate optimization technique to each specific layer based on its characteristics, rather than using a one-size-fits-all approach.
3Speed
If the compiler generates optimized low-level instructions, then latency and throughput are improved, but the user lacks visibility and control over the optimization strategy
Solution Approach 1:
The external strategy file serves as an intermediary that makes the optimization strategy transparent and accessible to users. Instead of the compiler's internal optimization process being a black box, the strategy is externalized in a readable file format that users can examine, understand, and modify. This mediator provides visibility into the optimization decisions while still enabling the compiler to generate optimized low-level instructions.
Data Source
AI summary
A method includes receiving a high-level function in a first high-level code; generating an external strategy associated with the high-level function, wherein the external strategy is in a second high-level code; outputting the external strategy, wherein the external strategy is modifiable; compiling the high-level function into a first set of low-level instructions to be executed on the hardware based on the external strategy if the external strategy remains unchanged; and compiling the high-level function into a second set of low-level instructions to be executed on the hardware based on a modification made to the external strategy if the external strategy has been modified.