Compiler Converts Quasi-Affine Tensor Indexing to Matrix Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks face inefficiencies in tensor indexing operations, particularly with quasi-affine expressions involving divisions or modulo operations, which are not efficiently evaluated by processing units optimized for matrix multiplications, leading to longer computation times and increased resource usage.
Innovation Solution
A compiler analyzes neural network representations and converts quasi-affine tensor indexing expressions into matrix multiplications, transposes, and summations, grouping mappings to reduce the number of operations and fuse operations where possible, allowing efficient execution on processing units optimized for matrix operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If quasi-affine tensor indexing expressions with divisions or modulo operations are used, then tensor indexing can be performed, but computation time increases and processing efficiency decreases
Solution Approach 1:
The patent transforms the mathematical parameters of tensor indexing from quasi-affine expressions (involving divisions and modulo operations) into equivalent affine expressions using only multiplications, additions, and subtractions. This parameter transformation maintains the indexing functionality while changing the computational operations to those better suited for hardware acceleration, thereby improving processing efficiency and reducing computation time.
Solution Approach 2:
The patent substitutes the mechanical evaluation of quasi-affine expressions (divisions and modulo operations) with matrix multiplication operations. By representing tensor indexing as matrix multiplications, the system leverages hardware optimized for linear algebra operations, replacing inefficient sequential division operations with parallelizable matrix operations that can be executed faster on specialized hardware.
2Productivity
If quasi-affine expressions are evaluated by processing units optimized for matrix multiplications, then processing can be performed, but resource usage increases
Solution Approach 1:
The patent changes the computational parameters from quasi-affine operations to affine operations that align with matrix multiplication capabilities. This parameter transformation allows the processing unit to utilize its optimized matrix multiplication hardware (such as systolic arrays or GEMM units) without requiring additional computational resources, thereby improving efficiency while maintaining low resource usage.
3Reliability
If tensor indexing operations are performed without conversion to matrix operations, then original indexing logic is preserved, but processing time increases
Solution Approach 1:
The patent performs preliminary transformation of the tensor indexing expressions into matrix operation form before execution. By pre-converting quasi-affine expressions into equivalent affine expressions with matrix multiplications, the system preserves the original indexing logic and accuracy while preparing the operations for efficient hardware execution, thereby eliminating the time penalty associated with unoptimized indexing operations.
Data Source
AI summary
A computer-implemented method includes generating, based on a representation of a tensor mapping between an input tensor and an output tensor, a list of mappings from elements of the input tensor to elements of the output tensor, and generating groups of mappings from the list of mappings, where each of the groups of mappings corresponds to a respective set of matrix multiplications, a matrix transpose, or both. The computer-implemented method also includes generating a respective expression for each of the groups of mappings and generating code for summing results of the respective expressions, where each respective expression includes the respective set of matrix multiplications, the matrix transpose, or both.