Tensor Processor ISA Using Resultant Tensors as Operands
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hardware accelerators for machine learning tasks, such as deep neural networks, face inefficiencies due to the need for large tensor register files and the latency caused by waiting for tensors, which can lead to stalls and increased costs.
Innovation Solution
Implementing multiple smaller tensor register files dedicated to specific tensor operation calculators and reducing the number of bits in machine instructions by using resultant tensors as operands, thereby reducing memory requirements and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large tensor register files are used to store tensors for machine learning operations, then tensor availability is improved, but memory requirements and costs increase
Solution Approach 1:
The patent divides the tensor register file into multiple smaller, dedicated tensor register files (e.g., first tensor register file, second tensor register file, third tensor register file). Each smaller register file is dedicated to specific tensor operation calculators, reducing the memory capacity required in each file while maintaining overall tensor availability through specialized allocation.
Solution Approach 2:
The patent assigns different tensor register files to different types of tensor operation calculators based on their specific needs. For example, first tensor operation calculators that perform first type of tensor operations are dedicated to accessing first tensor register files, while second tensor operation calculators performing second type of tensor operations access second tensor register files. This local specialization optimizes memory usage for each calculator type.
2Reliability
If tensors are stored in large register files, then tensor access is possible, but latency increases due to waiting for tensors
Solution Approach 1:
The patent pre-loads tensors into dedicated tensor register files before they are needed by specific tensor operation calculators. By having tensors already available in the appropriate register file before the calculator needs them, the system eliminates waiting time and reduces latency while maintaining reliable tensor access.
Solution Approach 2:
The dedicated tensor register files act as intermediaries between main memory and tensor operation calculators. They provide a buffer that holds tensors close to the calculators, enabling fast access without requiring the calculators to wait for tensors to be fetched from larger, more distant memory structures.
3Productivity
If dedicated tensor register files are assigned to specific tensor operation calculators, then bandwidth utilization is improved, but device complexity increases
Solution Approach 1:
The patent creates a systematic mapping where multiple tensor operation calculators of the same type share access to the same dedicated tensor register file. For example, multiple first tensor operation calculators all access the first tensor register file, and multiple second tensor operation calculators access the second tensor register file. This multi-functionality approach allows the system to serve multiple calculators through fewer register files, reducing overall complexity while maintaining high bandwidth utilization.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
A hardware accelerator having an efficient instruction set is disclosed. An apparatus may comprise logic configured to access a first and a second machine instruction. The second machine instruction may be missing a tensor operand needed to execute the second machine instruction. The logic may be further configured to execute the first machine instruction, resulting in a tensor. The logic may be further configured to execute the second machine instruction using the resultant tensor as the missing tensor operand.