Matrix Operation Circuitry Partitioning Lookup Tables for MAC Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer systems face inefficiencies in processing and converting data formats, such as text to speech or speech to text, due to limitations in the number of multiply-accumulate (MAC) operations performed per cycle, which affects computation speed and power consumption during general matrix multiply operations (GEMM).
Innovation Solution
The proposed solution involves partitioning data across multiple lookup tables in a device's memory, allowing for parallel access and processing by multiple MAC units, thereby increasing the number of MAC operations per cycle and reducing power consumption. This is achieved through a hardware design that includes a processor with multiple MAC units, a load store memory, and parallel-connected memory banks, enabling efficient matrix multiplication and conversion between different data formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is processed using conventional memory access methods, then the system can perform matrix multiply operations, but the computation speed is limited and power consumption is high
Solution Approach 1:
The patent divides the input data into multiple partitions, with each partition stored in a separate memory bank. This segmentation allows parallel access to multiple memory banks simultaneously, increasing the number of MAC operations that can be performed per cycle. The data partitioning strategy enables the system to overcome the limitation of single-memory-access bottlenecks while distributing the computational load across multiple parallel processing units.
Solution Approach 2:
The patent introduces a new dimension of parallelism by organizing memory banks in a multi-dimensional array structure rather than using a single linear memory structure. By arranging memory banks in rows and columns that can be accessed independently, the system enables simultaneous access to multiple memory locations across different dimensions, thereby increasing throughput without proportionally increasing power consumption.
2Productivity
If the number of MAC operations per cycle is increased, then computation speed improves, but device complexity increases
Solution Approach 1:
The patent segments the processing architecture into multiple independent MAC units, each handling a specific partition of the input data. This segmentation allows the system to perform multiple MAC operations in parallel without requiring a single complex processing unit. Each MAC unit remains relatively simple in design, but the collective system achieves high productivity through parallel execution.
Solution Approach 2:
The patent designs the MAC units and memory banks to be universal and reusable across different computational tasks. The same hardware structure can process different data formats (text, speech, audio, video) and different matrix dimensions by reconfiguring the data partitioning strategy, rather than requiring specialized hardware for each specific operation type.
3Productivity
If data is partitioned across multiple memory banks, then parallel processing efficiency increases, but memory access complexity increases
Solution Approach 1:
The patent performs preliminary data partitioning and distribution to memory banks before the actual computation begins. By pre-organizing the input data into the appropriate partition structure that matches the memory bank arrangement, the system eliminates the need for complex runtime data reorganization. This preliminary action simplifies the access pattern during computation, as the data is already positioned for optimal parallel access.
Solution Approach 2:
The patent introduces load store units as intermediary components between the MAC units and the memory banks. These intermediary units handle the complexity of data movement and partitioning management, shielding the MAC units from direct involvement in complex memory access patterns. The load store units act as mediators that translate computational requirements into optimized memory access sequences.
Data Source
AI summary
Disclosed herein includes a system, a method, and a device for processing and converting data using matrix operations. Circuitry can partition an input of a first data format across a plurality of lookup tables each residing in a respective memory. The circuitry can access weight information from a load store memory, and the partitioned input on a per column basis from the plurality of lookup tables. The circuitry can perform a number of multiply-accumulate (MAC) operations per cycle between the weight information from the load store memory and the partitioned input read on a per column basis from the plurality of lookup tables. The number of MAC operations performed per cycle can correspond to a total number of columns of the plurality of lookup tables. The circuitry can generate, responsive to the MAC operations on the partitioned input, a plurality of outputs in a second data format.