Deep Learning Accelerator Program Generation via Element-Wise Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning accelerator technologies are limited in generating programs for combinations of element-wise operations beyond predetermined patterns, leading to inefficiencies and performance overheads.
Innovation Solution
A method for generating programs for deep learning accelerators that involves analyzing element-wise operations, determining data dependency relationships, and selecting optimal division schedules to concatenate intermediate expressions, thereby generating optimal core operation codes for improved performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If kernel fusion is used to combine element-wise operations, then processing overhead is reduced and performance is improved, but the technique can only be executed for predetermined patterns and cannot handle undefined operation patterns
Solution Approach 1:
The patent changes the parameters of operation pattern recognition from fixed predetermined patterns to dynamic pattern extraction through AST analysis. By representing operations as abstract syntax trees and enabling flexible matching of operation sequences, the system can adapt to any element-wise operation pattern rather than being limited to predefined patterns, thus resolving the contradiction between processing efficiency and pattern flexibility.
Solution Approach 2:
The patent creates a universal kernel fusion mechanism that can handle multiple operation patterns through a single framework. The abstract syntax tree representation and division schedule generation system serves multiple functions: it can process addition, multiplication, and other element-wise operations uniformly, making the system universally applicable to various operation patterns without requiring separate handling for each pattern type.
2Ease of manufacture
If element-wise operations are executed as separate programs, then each operation can be optimized independently, but memory overhead increases due to storing and reading intermediate results
Solution Approach 1:
The patent merges multiple element-wise operations into a single fused kernel program, eliminating the need to store and read intermediate results between separate operations. By combining operations A and B into one program execution, the system removes memory access overhead while maintaining the ability to optimize each operation's computation logic independently through the division schedule mechanism.
Solution Approach 2:
The patent segments the fusion process into distinct operational phases through division schedules, which define the order and grouping of operations within the fused kernel. This segmentation allows the system to optimize the computation of each element-wise operation independently while executing them as a unified program, thus reducing memory overhead without sacrificing optimization flexibility.
Data Source
AI summary
The present disclosure relates to a method for generating a program for use in an accelerator for deep learning. The method may include receiving, by a computing device, a deep learning application, generating an element-wise operation list included in the deep learning application, generating an intermediate expression from the element-wise operation list, and generating, based on the intermediate expression, a program for use in an accelerator for the deep learning application.