Configurable Connectivity Mesh for Matrix Multiplication Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing complexity of machine learning models and datasets leads to a significant demand for computational resources, particularly in matrix multiplications, which are fundamental to many machine learning algorithms. This demand results in high computational costs, energy consumption, and barriers for smaller organizations and researchers with limited resources.
Innovation Solution
A system for accelerating vector multiplications is introduced, comprising a set of multipliers, a connectivity mesh, and a set of readable cells. Each multiplier is hardwired to multiply every possible value of a vector's parameters by an input value, with the connectivity mesh linking the multipliers' outputs to the readable cells. This configuration allows for parallel reading of output values, facilitating efficient matrix multiplications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional matrix multiplication methods are used, then computational accuracy is maintained, but computational cost and energy consumption increase significantly
Solution Approach 1:
The patent segments the matrix multiplication operation into two distinct phases: a multiply phase where weight values are multiplied by input values, and a read phase where the results are read from memory. This segmentation allows the system to optimize each phase independently, reducing overall computational cost and energy consumption while maintaining accuracy.
Solution Approach 2:
The patent implements preliminary action by pre-computing and storing weight values in the readable cells during training. During inference, these pre-stored values are directly retrieved and multiplied, eliminating the need for complex real-time computations and reducing both computational cost and energy consumption.
2Speed
If the number of discrete multiplication operations is reduced, then computational speed increases, but the complexity of the computational architecture increases
Solution Approach 1:
The patent uses copying by creating a simplified computational model where weight values are stored as readable cells with associated addresses. Instead of performing complex real-time matrix multiplications, the system copies pre-computed weight values into these cells and retrieves them during inference, significantly reducing the number of discrete multiplication operations and increasing computational speed.
Solution Approach 2:
The patent introduces an intermediary structure consisting of readable cells and a connectivity mesh. The readable cells store pre-computed weight values, and the connectivity mesh provides a simplified routing mechanism. This intermediary layer mediates between the input data and the computational output, reducing the complexity of the overall architecture while maintaining computational efficiency.
3Quantity of substance
If data types are reduced to fewer bits, then memory and computational resources are reduced, but the number of possible values per parameter decreases
Solution Approach 1:
The patent applies parameter changes by transforming the representation of weight values from full-precision floating-point numbers to compressed formats stored in readable cells. The system changes the parameter representation to use fewer bits per value while maintaining the necessary computational precision through the connectivity mesh routing mechanism, thereby reducing memory resources without significantly compromising the number of effectively representable values.
Data Source
AI summary
Methods and systems which involve computer architectures are disclosed herein. A disclosed system comprises a set of multipliers input-coupled to an input, a connectivity mesh, a set of readable cells, and a read circuit. Each readable cell in the set of readable cells is uniquely associated with a parameter of a vector, the parameter having a value. Each readable cell in the set of readable cells is also coupled, via the connectivity mesh, to a multiplier, from the set of multipliers, that outputs the product of the value of the parameter and the input. The read circuit reads a product of the input and the vector from the set of readable cells.