Tensor Map Cache Storage via TMAU Prefetching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing operations with tensors require significant time, power, and computing resources, necessitating improvements in efficiency.
Innovation Solution
The implementation of a parallel computing platform with a tensor memory access unit (TMAU) that prefetches tensor maps to cache, enabling asynchronous data movement and reducing computational overhead by offloading address calculations and data transfer operations from processing units to specialized hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tensor operations are performed using conventional processing units, then computational tasks can be executed, but significant time and computing resources are consumed
Solution Approach 1:
The system divides tensor operations into two distinct segments: data movement handled by the TMAU and computational operations handled by processing units. This segmentation allows concurrent execution of data transfer and computation, reducing the sequential time consumption of conventional approaches while maintaining overall processing efficiency
Solution Approach 2:
The TMAU acts as an intermediary component between memory and processing units, specifically designed to handle tensor map data transfer. This intermediary offloads the data movement burden from processing units, enabling them to focus on computation and reducing overall time consumption through specialized optimization
2Productivity
If data transfer operations are performed synchronously with computational operations, then resource utilization is maximized, but computational overhead increases
Solution Approach 1:
The patent extracts data transfer operations from the computational workflow and assigns them to a dedicated TMAU unit. This extraction removes the computational overhead associated with managing data transfer from processing units, while the TMAU handles data movement independently, enabling resource utilization without increasing computational complexity
Solution Approach 2:
The system replaces the mechanical approach of processing units managing both computation and data transfer with a specialized hardware mechanism (TMAU) dedicated to data movement. This substitution eliminates the need for processing units to handle data transfer overhead, reducing computational complexity while maintaining high resource utilization through concurrent operations
3Adaptability or versatility
If address calculations and data transfer are performed by processing units, then flexibility is maintained, but energy consumption increases
Solution Approach 1:
The TMAU is designed as a self-service unit that autonomously performs address calculations and data transfer operations without requiring processing unit intervention. This self-service capability maintains flexibility in data access patterns while significantly reducing energy consumption by eliminating redundant computational operations from processing units
Data Source
AI summary
Apparatuses, systems, and techniques to store one or more tensor maps in one or more cache storages. In at least one embodiment, a processor includes one or more tensor acceleration logic circuits to cause one or more tensor maps to be stored in one or more cache storages.