Vector Compute Engine Opcode Table Microoperation Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network hardware architectures, such as systolic arrays, are optimized for matrix multiplication but struggle to efficiently execute other types of computations like batch normalization and pooling, which can lead to suboptimal utilization of computational resources.
Innovation Solution
A programming architecture for a compute engine that allows complex computations by mapping machine instructions to sequences of microoperations, each containing control flow information and datapath configurations to configure computational circuit blocks in a pipeline.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a systolic array is used for matrix multiplication, then matrix multiplication throughput is improved, but the ability to execute other computations (batch normalization, pooling) efficiently deteriorates
Solution Approach 1:
The neural network computation is segmented into two distinct parts: matrix multiplication operations are executed by the systolic array, while other operations (batch normalization, pooling, activation functions) are executed by a separate vector compute engine. This segmentation allows each component to be optimized for its specific function, resolving the contradiction between matrix multiplication throughput and computation type flexibility.
Solution Approach 2:
The vector compute engine is designed with a programmable architecture that can execute multiple types of computations including batch normalization, pooling, activation functions, and data preprocessing. This multi-functionality allows the system to handle diverse computation types that cannot be efficiently executed by the systolic array alone, thereby improving overall computation versatility while preserving matrix multiplication throughput.
2Device complexity
If a fixed architecture is used for neural network computation, then hardware simplicity is improved, but the ability to handle diverse computation types deteriorates
Solution Approach 1:
The vector compute engine employs a dynamic, programmable architecture where the datapath configuration can be changed through microoperations. Each microoperation contains control flow information and datapath configurations that dynamically reconfigure the computational circuit blocks in the pipeline. This dynamic reconfigurability allows the same hardware to efficiently execute diverse computation types without requiring multiple fixed architectures.
3Productivity
If computational resources are dedicated to matrix multiplication, then matrix multiplication performance is improved, but utilization for other computations deteriorates
Solution Approach 1:
Computational resources are segmented into two separate execution units: the systolic array is dedicated to matrix multiplication operations, while the vector compute engine handles other computation types. This segmentation prevents resource contention and ensures that each unit can operate at full capacity for its designated operations, improving overall resource utilization efficiency without compromising matrix multiplication performance.
Data Source
AI summary
A technique to program a compute channel having multiple computational circuit blocks coupled in series in a pipeline can include receiving a machine instruction for the compute channel. The machine instruction is decoded to obtain an opcode, and the opcode can be used as an index to access an opcode entry in an opcode table. The opcode entry contains a pointer to a microoperation, and the pointer can be used to access a microoperation represented by a control entry in a control table and a datapath configuration entry in a datapath table. The microoperation can then be issued to the compute channel by configuring the compute channel with the control entry and the datapath configuration entry.