Native Tensor Processor Outer Product Contraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional parallel processing architectures face inefficiencies and high costs in handling large data volumes for machine learning tasks, particularly due to data movement patterns and the need for custom integrated circuits that become outdated quickly.
Innovation Solution
A native tensor processor that calculates tensor contractions using a sum of outer products, implemented as a single integrated circuit with a contraction engine and input buffer, which partitions matrix multiplies into localized smaller operations, allowing for scalable and reconfigurable hardware with reduced data transfer and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional parallel processing architectures are used with computing units arranged in arrays or meshes, then the system can handle basic computations, but data movement overhead increases and computing performance deteriorates for large data volumes
Solution Approach 1:
The patent segments the tensor contraction computation into multiple outer product calculations that can be performed in parallel. Each outer product unit processes a specific portion of the computation independently, allowing simultaneous execution of multiple segments without requiring extensive data movement between processing units. This segmentation enables the system to maintain high computing performance while reducing data movement overhead.
2Speed
If register files, cache and scratch memory are added to reduce memory latency, then data access speed improves, but device complexity and energy consumption increase
Solution Approach 1:
The patent extracts the memory latency issue from the overall system by using a double-buffer architecture where data retrieval from off-chip memory and feeding data to the contraction engine are optimized separately. The input buffer decouples these two operations, allowing memory access to occur independently from computation, thereby reducing the need for complex cache hierarchies and associated circuitry.
3Productivity
If custom integrated circuits are designed for specific applications, then computing efficiency improves, but manufacturing cost increases and adaptability decreases
Solution Approach 1:
The patent implements a universal tensor processor architecture that can efficiently handle various tensor contraction operations through a single integrated circuit design. The contraction engine is designed to process different tensor sizes and configurations using the same outer product computation mechanism, eliminating the need for custom hardware designs for different applications while maintaining high computing efficiency.
4Ease of operation
If spatial dataflow machines with mesh topology are used, then dataflow processing is enabled, but latency increases and scalability becomes difficult
Solution Approach 1:
The patent transitions from the traditional spatial mesh topology to a computational dimensionality change by using outer product decomposition. Instead of relying on spatial proximity for data access, the computation is restructured into outer product operations that can be executed independently and in parallel, effectively adding a computational dimension that bypasses spatial latency constraints and enables easier scaling.
Data Source
AI summary
A native tensor processor calculates tensor contractions using a sum of outer products. In one implementation, the native tensor processor preferably is implemented as a single integrated circuit and includes an input buffer and a contraction engine. The input buffer buffers tensor elements retrieved from off-chip and transmits the elements to the contraction engine as needed. The contraction engine calculates the tensor contraction by executing calculations from an equivalent matrix multiplications, as if the tensors were unfolded into matrices, but avoiding the overhead of expressly unfolding the tensors. The contraction engine includes a plurality of outer product units that calculate matrix multiplications by a sum of outer products. By using outer products, the equivalent matrix multiplications can be partitioned into smaller matrix multiplications, each of which is localized with respect to which tensor elements are required.