Tensor Engine Load-Store Unit for Multi-Dimensional Stride Walking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing tensor engines are inadequate for high-performance computing due to the inefficiency of nested loop structures and pointer arithmetic in processing multi-dimensional arrays, which limits their effectiveness in advanced applications like machine learning and deep learning.
Innovation Solution
A load/store unit (LDSU) is introduced to facilitate efficient processing of n-dimensional tensors without nested loops, utilizing a tensor walking module, striding module, and iteration tracking to manage registers and offsets, enabling parallel processing of tensors across multiple dimensions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If nested loop structures with pointer arithmetic are used to process multi-dimensional arrays, then the tensor engine can handle complex computations, but the computational overhead increases and performance decreases
Solution Approach 1:
The patent extracts the loop control and address calculation logic from the main computation flow into dedicated hardware modules (counter module, pointer module, offset module). This separates the computational kernel from the iteration management, allowing the tensor operations to execute without the overhead of software-based nested loops and pointer arithmetic.
Solution Approach 2:
The patent introduces intermediary hardware modules that mediate between the memory system and the tensor computation units. These modules (counters, pointers, offset registers) act as intermediaries that automatically manage multi-dimensional array indexing and traversal, eliminating the need for complex software loop structures while maintaining correct data access patterns.
2Adaptability or versatility
If nested loops are implemented in tensor engines, then multi-dimensional array processing is achieved, but the number of instructions increases making them inadequate for high performance computing
Solution Approach 1:
The patent segments the multi-dimensional array processing into independent modular components: a counter module that handles iteration counting, a pointer module that manages memory addressing, and an offset module that handles dimension-specific indexing. Each module operates independently and in parallel, enabling versatile multi-dimensional processing without the sequential overhead of nested software loops.
Solution Approach 2:
The patent replaces the mechanical software-based loop control mechanism with a hardware-based automated system. Instead of using software loops that require multiple instructions per iteration, the system uses hardware counters and pointers that automatically increment and manage memory access, substituting the software control mechanism with a more efficient hardware control mechanism.
Data Source
AI summary
A method for processing a tensor is described including obtaining a first register for a number of items in the tensor. One or more second registers for a number of items in a first and a second axis of the tensor are obtained. A stride in the first and the second axis is obtained A next item in the tensor is obtained using the stride in the first axis and a first offset register, when the first register indicates the tensor has additional items to process and the second registers indicate the next item resides in the first axis. A next item in the tensor is obtained using the stride in the first axis and the second axis, the first offset register, and a second offset register. The first register and a second register is modified. The first and the second offset registers are modified.