Tensor Core Activation Hardware for Low-Latency Cache Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graphics Processing Units (GPUs) face challenges in cache management due to the use of virtually indexed caches, which prevent the application of page table entry-based caching control, leading to inefficiencies in memory access and latency.
Innovation Solution
Implementing hardware support for activation functions within a matrix engine to optimize cache control, allowing for efficient cache management and reduced latency by enabling page table entry-based caching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a virtually indexed cache is used to reduce cache access latency, then cache lookup can begin before virtual to physical translation is performed, but page table entry-based caching control cannot be used
Solution Approach 1:
The cache indexing mechanism is segmented into two independent parts: virtual address indexing (for fast lookup initiation) and page table entry-based control (for caching policy management). This allows the cache to accept virtual addresses directly for indexing while separately applying page table control logic when needed, resolving the contradiction between fast access and control flexibility
Solution Approach 2:
The patent introduces an intermediary mechanism that bridges virtual address indexing and physical address translation. The cache uses virtual addresses as intermediaries to initiate lookups early, while page table entries serve as another layer of intermediary to provide control policies without blocking the initial cache access, thus maintaining both speed and control capability
2Productivity
If hardware support for activation functions is implemented within the matrix engine, then cache management efficiency is enhanced, but device complexity increases
Solution Approach 1:
The patent merges the activation function processing unit with the matrix engine, integrating cache management capabilities directly into the existing computational hardware. This consolidation allows the matrix engine to handle both matrix operations and activation functions with unified cache control, improving efficiency while minimizing the increase in overall device complexity through resource sharing
Data Source
AI summary
Embodiments described herein provide hardware support for activation functions within a matrix engine. One embodiment provides a graphics processor including a tensor core having first circuitry to perform a matrix operation and second circuitry configured to perform an element-wise operation on a result of the matrix operation before the result is output by the tensor core.