Flexible Compute Engine Microarchitecture for Mixed-Precision VMMs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing learning networks face inefficiencies in computation time and flexibility due to the use of hardware tools that are not sufficiently flexible to manage different types of parameters in machine learning models, particularly for larger numbers of parameters.
Innovation Solution
A compute engine architecture that includes compute-in-memory (CIM) hardware modules with vector multiplication units (VMUs) capable of performing vector-matrix multiplications (VMMs) for different weight lengths, allowing for parallel processing and efficient use of both longer and shorter precision weights without deactivating VMUs, and incorporating combiners to combine products appropriately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hardware tools are used to perform machine learning operations, then computation speed and efficiency are improved, but flexibility to manage different types of parameters is reduced
Solution Approach 1:
The compute engine employs dynamic resource allocation where compute units can be configured to handle different parameter types and precisions. The system dynamically adjusts the number of active compute units and their configurations based on the specific computational task, enabling flexible management of various parameters while maintaining high computation speed.
Solution Approach 2:
The architecture allows changing parameters such as precision levels and compute unit configurations to optimize performance for different parameter types. By adjusting these parameters, the system can efficiently manage both longer and shorter precision weights, resolving the contradiction between speed and flexibility.
2Quantity of substance
If compute units are deactivated to manage different weight lengths, then hardware resource utilization is improved, but computation time increases
Solution Approach 1:
The compute engine segments compute units into multiple groups that can independently process different portions of computations. This segmentation allows the system to process longer and shorter precision weights simultaneously using different segments, eliminating the need to deactivate compute units and reducing computation time while maintaining efficient resource utilization.
Solution Approach 2:
The system introduces an additional dimension to resource management by enabling compute units to operate in different precision modes simultaneously. This dimensional approach allows longer and shorter precision computations to proceed in parallel without deactivating any compute units, thus reducing computation time while optimizing hardware resource utilization.
3Measurement precision
If longer precision weights are used, then model accuracy is improved, but computation complexity and resource requirements increase
Solution Approach 1:
The compute engine implements partial action by selectively activating only the necessary compute units for the given precision requirements. For shorter precision weights, fewer compute units are activated, while for longer precision weights, more compute units are activated. This partial activation strategy manages computation complexity while maintaining the ability to achieve high model accuracy when needed.
Data Source
AI summary
A compute engine is described. The compute engine includes compute-in-memory (CIM) modules and may include an input buffer coupled with the CIM modules. The input buffer stores a vector. The CIM modules store weights corresponding to a matrix and perform a vector-matrix multiplication (VMM) for the matrix and the vector. The CIM modules further include storage cells and vector multiplication units (VMUs) coupled with the storage cell and, if present, the input buffer. The storage cells store the weights. The VMUs multiply, with the vector, at least a portion of each weight of a portion of the plurality of weights corresponding to a portion of the matrix. A set of VMUs performs multiplications for a first weight length and a second weight length different from the first weight length such that each VMU of the set performs multiplications for both the first weight length and the second weight length.