Systolic Tensor Arrays With Significance Maps for Sparse Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) face challenges in efficiently processing sparse data due to the limitations of traditional fixed function computational units and the need for improved parallel processing techniques, particularly in machine learning operations.
Innovation Solution
Implementing a systolic tensor array architecture within GPUs that optimizes processing of sparse data by bypassing zero values and using significance map encoding to enhance processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional fixed function computational units are used to process sparse data, then the processing covers all data elements including zeros, but the processing efficiency deteriorates due to wasted computations on zero values
Solution Approach 1:
The patent extracts and removes zero values from the data processing flow by using significance maps to identify and bypass zero elements. The systolic array skips computations involving zero values, extracting only the necessary non-zero data elements for processing, thereby eliminating wasted computational energy on zero values while maintaining correct results.
Solution Approach 2:
The patent applies local quality by treating different data elements differently based on their significance. Non-zero elements receive full computational processing while zero elements are bypassed. The significance map provides local information about which data elements require processing, allowing the system to allocate computational resources selectively rather than uniformly across all elements.
2Productivity
If traditional systolic arrays process all elements including zeros, then the architecture remains simple and uniform, but the processing time increases due to unnecessary computations
Solution Approach 1:
The patent introduces dynamics by making the data flow and computation paths adaptive rather than static. The significance map dynamically guides the systolic array to activate or deactivate computation paths based on the actual data content. This dynamic behavior allows the system to speed up processing for sparse data by skipping zero values while maintaining full processing capability for dense data.
Solution Approach 2:
The significance map acts as an intermediary between the input data and the systolic array computation. It mediates the data flow by preprocessing the input to mark significant elements, thereby guiding the systolic array to process only necessary elements. This intermediary layer enables the system to achieve faster processing speeds by eliminating redundant computations without requiring fundamental changes to the systolic array architecture.
3Loss of energy
If significance map encoding is implemented to bypass zero values, then computational overhead is reduced, but the device complexity increases due to additional encoding and decoding logic
Solution Approach 1:
The patent merges the significance map generation, data encoding, and systolic array processing into an integrated system. The significance map logic is combined with the data input stage of the systolic array, and the bypass control is merged into the existing computation pathways. This merging approach reduces overall system complexity by eliminating separate standalone components while achieving the benefit of reduced computational overhead through coordinated operation of integrated elements.
Solution Approach 2:
The significance map mechanism is designed to be universal and applicable to various sparse data processing tasks. The same encoding and bypass logic can handle different data formats, dimensions, and computation types within the systolic array framework. This multi-functionality reduces the need for specialized complexity for each specific application, as the core significance map approach serves multiple purposes across different sparse data processing scenarios.
Data Source
AI summary
Embodiments described herein provided for an instruction and associated logic to enable a processing resource including a tensor accelerator to perform optimized computation of sparse submatrix operations. One embodiment provides a parallel processor comprising a processing cluster coupled with the cache memory. The processing cluster includes a plurality of multiprocessors coupled with a data interconnect, where a multiprocessor of the plurality of multiprocessors includes a tensor core configured to load tensor data and metadata associated with the tensor data from the cache memory, wherein the metadata indicates a first numerical transform applied to the tensor data, perform an inverse transform of the first numerical transform, perform a tensor operation on the tensor data after the inverse transform is performed, and write output of the tensor operation to a memory coupled with the processing cluster.