Tensor Register Files for Hardware Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware accelerators face inefficiencies in performing tensor operations due to waiting times for tensors and high memory requirements, particularly in executing machine learning algorithms like deep neural networks, where large centralized tensor register files lead to stalls and increased costs.
Innovation Solution
Implementing a hardware accelerator with multiple smaller tensor register files dedicated to specific tensor operation calculators, reducing waiting times and costs by allowing simultaneous access and utilizing the results of one machine instruction as operands for the next, thereby reducing the need for external storage and memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a large centralized tensor register file is used, then memory capacity is increased, but access stalls and waiting times increase
Solution Approach 1:
The patent divides the large centralized tensor register file into multiple smaller distributed tensor register files, each associated with specific tensor operation calculators. This segmentation allows parallel access to different segments simultaneously, eliminating the bottleneck of a single large register file while maintaining sufficient memory capacity across the distributed system.
Solution Approach 2:
The patent transitions from a single-dimension centralized storage model to a multi-dimensional distributed storage model where tensor register files are distributed across multiple calculation units. This dimensional change enables simultaneous access from multiple points, reducing waiting times while preserving total memory capacity.
2Quantity of substance
If a large centralized tensor register file is used, then memory capacity is increased, but device complexity and costs increase
Solution Approach 1:
By segmenting the large centralized register file into smaller distributed units, the patent reduces the complexity burden on any single component. Each smaller tensor register file has simpler access logic and control mechanisms, while the overall system achieves the required memory capacity through aggregation of multiple simple units.
Solution Approach 2:
The patent employs multiple smaller, simpler tensor register files that can be implemented with less complex circuitry compared to a single large register file. These smaller units are more cost-effective and easier to manufacture, achieving the same functional capacity through quantity rather than individual complexity.
3Quantity of substance
If tensors are stored in external memory, then memory capacity is sufficient, but bandwidth utilization decreases and access time increases
Solution Approach 1:
The patent pre-loads tensors into distributed tensor register files that are directly associated with tensor operation calculators before computation begins. This preliminary action ensures that data is already in fast, accessible memory when needed, eliminating the need for slow external memory access during computation and maximizing bandwidth utilization.
Solution Approach 2:
The distributed tensor register files serve as intermediary storage between external memory and the tensor operation calculators. This intermediary layer provides fast, localized access to frequently used tensors, reducing the frequency and impact of external memory accesses while maintaining sufficient capacity through the distributed architecture.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
Tensor register files in a hardware accelerator are disclosed. An apparatus may comprise tensor operation calculators each configured to perform a type of tensor operation. The apparatus may also comprises tensor register files, each of which is associated with one of the tensor operation calculators. The apparatus may also comprises logic configured to store respective ones of the tensors in the plurality of tensor register files in accordance with the type of tensor operation to be performed on the respective tensors. The apparatus may also control read access to tensor register files based on a type of tensor operation that a machine instruction is to perform.