Matrix Computation Unit Depth Concatenation via Shift Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Special-purpose integrated circuits used for neural network computations lack direct hardware support for depth concatenation operations, leading to processing delays and the need for off-chip computations.
Innovation Solution
Generating instructions that utilize shift weight matrices and modified identity weight matrices to perform depth concatenation operations using the existing matrix computation unit, allowing the integrated circuit to produce outputs that satisfy the specifications of a depth concatenation layer without requiring additional hardware or off-chip data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the integrated circuit uses a matrix computation unit that performs vector-matrix multiplications in hardware, then computation speed for supported operations is improved, but the circuit cannot directly perform depth concatenation operations, requiring off-chip processing
Solution Approach 1:
The patent introduces an intermediary software layer (instruction generation system) that translates depth concatenation operations into sequences of vector-matrix multiplication operations. This intermediary layer enables the matrix computation unit to perform depth concatenation indirectly through multiple supported operations, resolving the contradiction between hardware specialization and operational versatility.
Solution Approach 2:
The patent changes the operational parameters by representing depth concatenation as a series of vector-matrix multiplications with specific weight matrices. By transforming the operation type and parameters, the system enables the matrix computation unit to achieve depth concatenation functionality through its native vector-matrix multiplication capability.
2Adaptability or versatility
If the integrated circuit adds depth-concatenating hardware to directly support depth concatenation operations, then operation versatility is improved, but hardware complexity and cost increase
Solution Approach 1:
The patent makes the existing matrix computation unit universal by enabling it to perform both its original vector-matrix multiplication function and the additional depth concatenation function through software instructions. This multi-functionality approach avoids adding specialized depth concatenation hardware, maintaining hardware simplicity while achieving operational versatility.
Solution Approach 2:
The system enables the matrix computation unit to serve itself by providing instructions that allow it to perform depth concatenation using its existing vector-matrix multiplication capability. The unit uses its own computational resources and instruction set to achieve functionality that would otherwise require separate dedicated hardware.
3Adaptability or versatility
If depth concatenation operations are performed off-chip in software, then operational flexibility is maintained, but processing delays occur due to data transfer between chip and host
Solution Approach 1:
The patent performs preliminary action by pre-compiling depth concatenation operations into sequences of vector-matrix multiplication instructions that can be executed directly by the matrix computation unit. This preparation enables the operations to be performed on-chip without real-time data transfer, eliminating processing delays while maintaining operational flexibility.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for depth concatenation using a matrix computation unit. One of the methods includes: receiving a request to process network inputs to a neural network using an integrated circuit, the neural network comprising a depth concatenation neural network layer; and generating instructions that, when executed by the integrated circuit, cause the integrated circuit to performing operations comprising: for each spatial location in a first input tensor to the depth concatenation layer and a second input tensor to the depth concatenation layer: multiplying, using the matrix computation unit, a second depth vector for the spatial location by a shift weight matrix for the depth concatenation layer to generate a shifted second depth vector; and adding the shifted second depth vector and a first input depth vector for the spatial location to generate a concatenated depth vector.