Matrix Data Reuse in MAC Units with Serial Shift Buffers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computation of convolutions in computing systems for artificial intelligence tasks, such as image processing, involves substantial memory access and communication bandwidth consumption due to the need to load and transfer data for each output feature map calculation, leading to inefficiencies in power usage and performance.
Innovation Solution
Implementing a system with data reuse techniques in multiply and accumulate units, where adjacent values from matrices are reused by multiple units, reducing duplicate memory access and allowing parallel computations, and incorporating a serial shift buffer to share input data among units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is loaded from memory for each output feature map calculation, then computation accuracy is maintained, but bandwidth utilization increases and power consumption rises
Solution Approach 1:
The patent applies preliminary action by loading and storing matrix data (weights and input feature maps) into on-chip buffer memory before the actual convolution computation begins. This pre-loading allows multiple MAC units to reuse the same data without repeated memory accesses, thereby reducing power consumption while maintaining computation accuracy. The buffer acts as a preliminary data repository that eliminates the need for repeated high-power memory accesses during parallel computations.
Solution Approach 2:
The patent introduces an intermediary buffer memory structure that sits between the main system memory and the MAC units. This buffer serves as a mediator that holds data for multiple units to share, reducing the need for each unit to directly access high-bandwidth memory. The intermediary buffer enables data reuse across multiple MAC units, thereby reducing overall bandwidth utilization and power consumption while preserving computation accuracy.
2Productivity
If multiple MAC units perform parallel computations, then productivity increases, but bandwidth utilization and power consumption increase due to duplicate data loading
Solution Approach 1:
The patent merges the data storage function for multiple MAC units into a single shared buffer memory structure. Instead of each MAC unit having its own dedicated data copies, the buffer consolidates the weight matrix and input feature map data that can be shared across multiple units. This merging allows parallel computations to proceed while reducing the total bandwidth required, as the same data can be read once and distributed to multiple MAC units simultaneously or sequentially reused.
Solution Approach 2:
The buffer memory structure serves multiple functions: it acts as a data repository for weights, stores input feature map data, and provides shared access for multiple MAC units. This multi-functional design allows the same buffer to support parallel computations across multiple units without requiring separate data paths for each unit, thereby improving productivity while controlling bandwidth utilization.
3Use of energy by moving object
If data is reused by multiple MAC units, then power consumption decreases, but device complexity increases due to buffer management
Solution Approach 1:
The patent segments the buffer memory into distinct regions or channels corresponding to different MAC units or computation streams. This segmentation allows each MAC unit to access its designated portion of the buffer independently, simplifying the management logic. Instead of requiring complex arbitration and coordination for shared access, the segmented buffer provides structured, organized data access paths that reduce management complexity while still enabling data reuse and lowering power consumption.
Data Source
AI summary
Techniques for computing matrix convolutions in a plurality of multiply and accumulate units including data reuse of adjacent values. The data reuse can include reading a current value of the first matrix in from memory for concurrent use by the plurality of multiply and accumulate units. The data reuse can also include reading a current value of the second matrix in from memory to a serial shift buffer coupled to the plurality of multiply and accumulate units. The data reuse can also include reading a current value of the second matrix in from memory for concurrent use by the plurality of multiply and accumulate units.