Systolic Tensor Array for Sparse Submatrix Zero-Bypass Computing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current graphics processors face inefficiencies in processing sparse data due to the limitations of fixed function computational units and the need for improved parallel processing techniques, particularly in graphics pipelines with single instruction, multiple thread (SIMT) architectures.

Innovation Solution

Implementing a systolic tensor array within a graphics processing unit (GPU) that optimizes processing of sparse data through circuitry designed for general-purpose graphics processing, including logic for bypassing zero values and using significance map encoding to facilitate efficient computation on compressed or encoded data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If fixed function computational units are used in graphics processors, then the processor can perform specific graphics operations, but it cannot efficiently process sparse data with improved parallel processing techniques

Engineering Contradiction:

Improveprocessing capability for sparse dataVSAvoidprocessing efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent implements a systolic tensor array that can perform multiple operations including sparse matrix multiplication, dense matrix multiplication, and convolution operations. This universal computational unit replaces multiple fixed-function units, enabling the graphics processor to efficiently handle both sparse and dense data types with a single reconfigurable structure.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The systolic tensor array incorporates dynamic bypass logic that can adaptively skip zero values during computation based on significance map encoding. This dynamic behavior allows the processor to adjust its computation path in real-time, processing only non-zero elements when dealing with sparse data, thereby improving parallel processing efficiency.

Inventive Principle:

Principle #15Dynamics

2Productivity

If traditional parallel processing techniques are used, then graphics data can be processed in parallel, but computational overhead increases when handling sparse data

Engineering Contradiction:

Improveparallel processing capabilityVSAvoidcomputational overhead

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent extracts and processes only the non-zero elements of sparse matrices by using significance map encoding to identify and bypass zero values. The systolic tensor array is configured to skip computations involving zero elements, thereby reducing the number of computational operations required and lowering energy consumption while maintaining parallel processing throughput.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The bypass logic within the systolic tensor array applies local optimization by selectively enabling or disabling computation paths based on the significance map. Each processing element can independently bypass zero values in its local computation, reducing overall computational overhead without affecting the global parallel processing structure.

Inventive Principle:

Principle #3Local quality

3Productivity

If significance map encoding is used to process compressed data, then processing efficiency improves, but device complexity increases

Engineering Contradiction:

Improvecomputation efficiency on sparse dataVSAvoidcircuitry complexity for encoding and bypass logic

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent merges the significance map generation, data compression, and bypass logic into a unified systolic tensor array structure. The same hardware components that perform matrix multiplication also generate and utilize significance maps, eliminating the need for separate encoding/decoding hardware and reducing overall device complexity while maintaining computation efficiency.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS20250363674A1Systolic arithmetic on sparse data

Publication Date: 2025.11.27 INTEL CORP

AI summary

Embodiments described herein provided for an instruction and associated logic to enable a processing resource including a tensor accelerator to perform optimized computation of sparse submatrix operations. One embodiment provides a parallel processor comprising a processing cluster coupled with the cache memory. The processing cluster includes a plurality of multiprocessors coupled with a data interconnect, where a multiprocessor of the plurality of multiprocessors includes a tensor core configured to load tensor data and metadata associated with the tensor data from the cache memory, wherein the metadata indicates a first numerical transform applied to the tensor data, perform an inverse transform of the first numerical transform, perform a tensor operation on the tensor data after the inverse transform is performed, and write output of the tensor operation to a memory coupled with the processing cluster.