Matrix Multiply Accelerator Engine with Broadcast and Unicast Interconnects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in performing matrix multiplication efficiently, particularly in energy-constrained, thermally restrained, and low-cost embedded platforms, due to high energy consumption, latency issues, and reduced precision when scaling down algorithms, and existing methods like CPU-based, systolic array, and GPU-based approaches have limitations such as power inefficiency, data mapping challenges, and hardware overheads.
Innovation Solution
The proposed solution involves a hardware-software co-design approach with a dedicated matrix multiply accelerator engine that includes a micro-architecture with a multi-channel n-stage accumulator, broadcast and unicast interconnects, and a data path controller for efficient data handling, allowing for spatial and temporal reuse of operands, and in-place transpose/computation to reduce memory access and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If CPU-based matrix multiplication is used, then versatility is maintained, but power consumption increases and processing speed decreases
Solution Approach 1:
The matrix multiplication operation is segmented into multiple pipelines, each handling specific matrix dimensions. This allows parallel processing of different segments simultaneously, reducing overall power consumption while maintaining the ability to handle various matrix sizes through dynamic pipeline configuration.
Solution Approach 2:
The accelerator engine employs dynamic pipeline activation where only the necessary pipelines are activated based on the input matrix dimensions. This dynamic adaptation reduces power consumption by avoiding activation of unnecessary processing units while maintaining versatility for different matrix sizes.
2Productivity
If GPU-based matrix multiplication is used, then processing speed increases, but hardware overhead and cost increase
Solution Approach 1:
Instead of using a full GPU architecture, the invention implements a specialized matrix multiplication accelerator with locally optimized components specifically designed for this operation. This includes dedicated multipliers, accumulators, and memory structures tailored for matrix operations, achieving high processing speed without the overhead of a general-purpose GPU.
Solution Approach 2:
The invention extracts only the essential matrix multiplication functionality from a general-purpose processor or GPU, creating a dedicated accelerator engine that performs this specific operation efficiently. This extraction removes unnecessary hardware overhead while maintaining high processing speed for matrix operations.
3Productivity
If systolic array is used, then processing speed increases, but data mapping complexity and hardware overhead increase
Solution Approach 1:
The invention implements a simplified array structure that performs only the essential matrix multiplication operations without the complex data mapping and communication overhead of full systolic arrays. By focusing on the core computation and using standard memory interfaces, it achieves high processing speed while reducing hardware complexity.
4Area of stationary object
If algorithms are scaled down for embedded platforms, then device size reduces, but precision and reliability decrease
Solution Approach 1:
The accelerator engine uses parameter changes in the computational approach, implementing high-precision multipliers and accumulators that maintain full precision arithmetic even in scaled-down embedded implementations. This allows the device to be compact while preserving computational precision through optimized arithmetic logic design.
Solution Approach 2:
The invention replaces software-based matrix multiplication with a hardware-accelerated engine that performs computations directly in circuitry. This substitution maintains high precision through dedicated hardware arithmetic units while reducing device size by eliminating the need for large software execution environments.
Data Source
AI summary
A disclosed apparatus to multiply matrices includes a compute engine. The compute engine includes multipliers in a two dimensional array that has a plurality of array locations defined by columns and rows. The apparatus also includes a plurality of adders in columns. A broadcast interconnect between a cache and the multipliers broadcasts a first set of operand data elements to multipliers in the rows of the array. A unicast interconnect unicasts a second set of operands between a data buffer and the multipliers. The multipliers multiply the operands to generate a plurality of outputs, and the adders add the outputs generated by the multipliers.