Vector Code Generation for Tensor Convolutions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Developing efficient convolution program code for AI engines like the Versal AI Engine is challenging due to the need for manual optimization of complex architectures, which limits code portability and performance, especially when dealing with convolution operations that require parallel processing and specific hardware features.
Innovation Solution
A method and system that automatically compile high-level language specifications into optimized instructions for AI engines by grouping and fusing multiply-and-accumulate operations, loading tensor elements efficiently into registers, and interleaving load and MAC operations to maximize hardware utilization, without requiring intrinsic functions or manual programming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If developers manually program each core to take full advantage of architectural features, then performance can be optimized, but code complexity and difficulty of preparation increase significantly
Solution Approach 1:
The system performs self-optimization by automatically analyzing convolution operations and generating optimized code for AI engine cores without requiring manual intervention. The compiler autonomously determines instruction scheduling, data loading patterns, and core assignment based on the convolution parameters and hardware architecture.
Solution Approach 2:
The invention transforms high-level convolution parameters (kernel size, input tensor dimensions, data types) into optimized execution parameters for the AI engine. The system automatically adjusts instruction formatting, data loading patterns, and core utilization based on these parameters to achieve optimal performance.
2Productivity
If intrinsic functions are used to express computations in groups of MAC operations, then hardware utilization is improved, but code portability and understandability deteriorate
Solution Approach 1:
The system introduces a high-level convolutional neural network interface as an intermediary between the user's high-level code and the low-level AI engine hardware. This interface translates portable high-level convolution operations into architecture-specific instructions, maintaining both portability and hardware optimization.
Solution Approach 2:
The invention replaces manual mechanical programming of hardware instructions with an automated compilation process. The compiler substitutes manual instruction generation with algorithmic translation from high-level mathematical operations to optimized hardware commands, eliminating the need for developers to write architecture-specific intrinsic functions.
3Productivity
If manual optimization of convolution code is performed, then performance can be tuned, but the solution space becomes infeasible to explore manually
Solution Approach 1:
The system incorporates feedback mechanisms where the compiler analyzes the convolution operation parameters, hardware architecture, and execution constraints to automatically generate optimized code. The feedback loop continuously refines the optimization based on convolution dimensions, data types, and available hardware resources.
Solution Approach 2:
The invention performs preliminary optimization analysis during the compilation phase before execution. The system pre-calculates optimal instruction scheduling, data loading patterns, and core assignment based on the convolution parameters, eliminating the need for manual post-processing optimization.
Data Source
AI summary
Disclosed approaches for generating vector codes include inputting tensor processing statements. Each statement specifies an output variable, an initial variable, and multiply-and-accumulate (MAC) operations, and each MAC operation references the output variable, elements of a first tensor, and one or more elements of a second tensor. The MAC operations are organized into groups, and the MAC operations in each group reference the same output variable and have overlapping references to elements of the first tensor. For each group of MAC operations, at least one instruction is generated to load elements of the first tensor into a first register and at least one instruction is generated to load one or more elements of the second tensor into a second register. For each group of MAC operations, instructions are generated to select for each MAC operation in the group for input to an array of MAC circuits, elements from the first register and one or more elements from the second register.