Analog Matrix Multiplier Fabric With Reinforcement-Learned Compute Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix multiplication, a computationally intensive task in machine learning, is often performed serially on general-purpose CPUs, limiting throughput and power efficiency due to serial computations and limited memory bandwidth.
Innovation Solution
A configurable compute fabric utilizing analog-in-memory compute cells, controlled by a machine learning system, optimizes matrix multiplication through reinforcement learning to enhance performance metrics like power, latency, and throughput by dynamically configuring compute blocks and data paths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If matrix multiplication is performed serially on general-purpose CPUs, then implementation simplicity is maintained, but throughput and power efficiency are limited
Solution Approach 1:
The compute fabric is segmented into multiple independent compute blocks (CBs), each capable of performing matrix multiplication operations. These blocks are organized in a grid structure with row CBs and column CBs, allowing parallel processing of matrix elements. Each CB contains dedicated functional units for multiplication and accumulation, enabling simultaneous operations across multiple segments of the input matrices.
Solution Approach 2:
The patent transitions from serial one-dimensional processing to a two-dimensional compute fabric architecture. Data flows through the fabric along row and column dimensions, with compute blocks arranged in a grid that enables spatial parallelism. This dimensional transformation allows multiple computations to occur simultaneously across different locations in the fabric, dramatically increasing throughput.
2Use of energy by moving object
If general-purpose CPUs are used for matrix multiplication, then hardware simplicity is maintained, but power efficiency deteriorates due to limited memory bandwidth
Solution Approach 1:
The patent merges computation and memory functions into a unified compute fabric architecture. Input matrices are stored directly within the fabric structure, and computation occurs in-place as data flows through the compute blocks. This integration eliminates the need for repeated data transfers between separate memory and processing units, reducing power consumption associated with memory access operations.
Solution Approach 2:
The compute fabric enables continuous data flow and computation without interruption. As data enters the fabric, it continuously flows through successive compute blocks performing multiplication and accumulation operations. This continuous processing eliminates idle cycles and keeps computational resources constantly utilized, improving overall power efficiency by maximizing the ratio of useful computation to total operational time.
3Loss of time
If serial computation is used on CPUs, then implementation simplicity is maintained, but latency increases
Solution Approach 1:
The compute fabric is pre-configured with compute blocks arranged in optimal positions for matrix multiplication. Input matrices are pre-loaded into the fabric structure before computation begins. The data flow paths and compute block configurations are established in advance, allowing computation to start immediately without setup delays during the actual multiplication process.
Solution Approach 2:
The compute fabric employs dynamic configuration capabilities where compute blocks can be adaptively assigned to different computational tasks. The architecture allows runtime reconfiguration of data flow paths and compute block assignments based on the specific matrix dimensions and computation requirements. This dynamic adaptability optimizes latency for different matrix sizes while maintaining a relatively simple base architecture.
Data Source
AI summary
A compute fabric includes, in part, a multitude of compute blocks, a networking circuit adapted to enable communication between the multitude of compute blocks, a performance monitor, and a controller trained to configure the compute fabric. The controller may be trained using a reinforcement learning process by setting the compute fabric to a first state, receiving a measurement of the performance characteristic of the compute fabric from performance monitor, receiving a reward signal in response to the measured performance characteristic; and repeating the setting, the receiving of the measurement and the receiving of the reward signal until the received reward reaches a maximum value. Each of at least a first subset of the compute blocks may be an analog in-memory compute block. Each of at least a second subset of the compute blocks may be a digital compute block.