MAC Operator Overlapping Data Periods for Neural Network Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network engines face challenges in maintaining high bandwidth while ensuring low latency due to the exponential increase in the size of convolution filters and input data, leading to prolonged computation times.
Innovation Solution
A memory device with a MAC operator that performs convolution operations by overlapping the input and weight data periods, utilizing a buffer memory to read weight data before input data is provided, and performing multiplication and summation operations efficiently through multiple accumulators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the size of convolution filter and input data increases to meet neural network requirements, then the amount of weight data and input data increases exponentially, but the computation time increases leading to reduced utilization
Solution Approach 1:
The buffer memory reads weight data from the memory cell in advance before the input data is provided to the MAC operator. This preliminary action ensures that weight data is ready when needed, eliminating waiting time during the convolution operation and enabling continuous processing despite increasing data sizes
Solution Approach 2:
The MAC operator performs convolution operations continuously by overlapping the input data period with the weight data reading period. This continuous operation maintains high utilization of the MAC operator without idle time, addressing the productivity issue while handling increased data volumes
2Quantity of substance
If the convolution operation is performed with increased data size, then the bandwidth requirement increases, but the delay increases reducing system utilization
Solution Approach 1:
Weight data is read from the memory cell to the buffer memory in advance before the convolution operation begins. This preliminary preparation reduces the actual computation delay by ensuring data is ready when the MAC operator needs it, even as data volumes increase
Solution Approach 2:
The patent introduces a time dimension overlap between weight data reading and input data processing. By performing these operations in parallel across different time periods, the system maintains high bandwidth utilization while reducing overall computation delay
3Reliability
If the MAC operator waits for both input data and weight data before performing convolution, then data accuracy is ensured, but the operation delay increases
Solution Approach 1:
The buffer memory prepares weight data in advance before the input data arrives at the MAC operator. This preliminary action ensures that when input data arrives, the weight data is already ready, maintaining data readiness reliability while eliminating the waiting time that would otherwise occur
Solution Approach 2:
The MAC operator maintains continuous operation by receiving weight data from the buffer memory during the same period that input data is being provided. This continuous action eliminates idle waiting time while ensuring both data streams are ready for accurate convolution processing
Data Source
AI summary
A memory device is provided. The memory device includes: a memory cell configured to store weight data, a buffer memory configured to read the weight data from the memory cell, an input/output pad configured to receive input data and a multiply-accumulate (MAC) operator configured to receive the weight data from the buffer memory and receive the input data from the input/output pad to perform a convolution operation of the weight data and the input data, wherein the input data is provided to the MAC operator during a first period, and wherein the MAC operator performs the convolution operation of the weight data and the input data during a second period overlapping with the first period.