SIMD Matrix Multiplication With Flexible Operand Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer architectures face inefficiencies in supplying data to multiple computation units for matrix multiplication and convolution operations, particularly in machine learning applications, due to the need for separate instructions and significant control logic, which limits the number of SIMD engines that can be used effectively.
Innovation Solution
A computer architecture that includes a plurality of single instruction multiple data (SIMD) engines, with an Operand A and Operand B register file and distribution circuits to efficiently route operand values to these engines, allowing parallel operation and flexibility in data distribution modes, including round-robin and repeated word distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple separate MAC units are used to perform parallel dot products, then computational throughput is improved, but control logic complexity and data supply difficulty increase significantly
Solution Approach 1:
The patent combines multiple MAC units into a single SIMD engine that can perform multiple dot products in parallel using a unified control logic. The SIMD engine processes multiple operand pairs simultaneously through vectorized operations, eliminating the need for separate control logic for each MAC unit while maintaining high computational throughput.
Solution Approach 2:
The SIMD engine is designed as a universal computing unit that can perform various operations including dot products, matrix multiplication, and convolution. By making the control logic universal and applicable to multiple operation types, the patent reduces overall system complexity while preserving parallel processing capabilities.
2Productivity
If multiple SIMD engines are deployed to increase parallel processing capability, then computational performance is improved, but the difficulty of supplying data to all engines increases
Solution Approach 1:
The patent segments the data supply process by introducing dedicated distribution circuits that separately handle Operand A and Operand B routing. These circuits divide incoming data streams into appropriate portions for each SIMD engine, managing the complexity of supplying multiple engines simultaneously through structured data partitioning.
Solution Approach 2:
The distribution circuits act as intermediary components between the data source and multiple SIMD engines. They buffer, route, and synchronize data flows to ensure each engine receives the correct operands at the right time, simplifying the overall data supply architecture while supporting high parallel processing capability.
3Adaptability or versatility
If separate instructions are provided to each computation unit for flexible data distribution, then operational flexibility is improved, but the number of instructions and control logic increases
Solution Approach 1:
The SIMD engine implements a universal instruction set that can handle multiple data distribution patterns including round-robin and repeated word modes. This single versatile instruction system replaces the need for separate instructions for each computation unit, maintaining operational flexibility while reducing the total number of instructions required.
Solution Approach 2:
The distribution mode is made dynamic and configurable through control signals that can switch between different data distribution patterns. This allows the system to adapt to different computational requirements without requiring separate hardwired instruction paths, achieving flexibility through software-configurable control logic.
Data Source
AI summary
A computer system including a plurality of SIMD engines and a corresponding plurality of output register sets. Operand A register file stores one or more Operand A values, each including a plurality of operand words. Operand B register file stores one or more Operand B values, each including a plurality of operand words. Operand A distribution circuit receives an Operand A value from the Operand A register file, and selectively routes one or more of the operand words of the received Operand A value to create a plurality of input Operand A values, which are selectively routed to the SIMD engines. Operand B distribution circuit receives one or more Operand B values from the Operand B register file, and selectively routes one or more of the operand words of the Operand B value(s) to create a plurality of input Operand B values, which are selectively routed to the SIMD engines.