Zero Padding for Convolutional Neural Network Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models, specifically convolutional neural networks (CNNs), face inefficiencies when reading data from storage memory to operational memory due to spurious data entries and unwanted operations, leading to increased clock cycles, software complexity, and power consumption.
Innovation Solution
Implementing zero padding of feature maps during the cache read/MMA write process to avoid spurious data entries and reduce memory usage, which involves a memory controller that reads contiguous memory vectors, replaces values with zeroes based on the filter kernel, and writes zero-padded vectors to the MMA accelerator memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is read from storage memory to operational memory without zero padding, then memory usage is reduced, but spurious data entries are introduced and unwanted operations occur
Solution Approach 1:
The memory controller performs zero padding as a preliminary action during the data transfer process. Before the data is written to the MMA accelerator memory, the controller identifies and replaces spurious data entries with zeroes based on the filter kernel column index, preventing these erroneous values from entering the computational pipeline and causing unwanted operations.
2Reliability
If zero padding is applied to feature maps, then spurious data entries are avoided and unwanted operations are reduced, but additional memory operations are required
Solution Approach 1:
The memory controller performs the zero padding operation autonomously during the data transfer process from storage memory to MMA accelerator memory. By integrating the padding functionality into the existing memory controller hardware, the system eliminates the need for separate software-based padding operations, thereby reducing software complexity while maintaining data accuracy.
3Speed
If conventional memory read/write processes are used without zero padding, then processing speed is maintained, but clock cycles increase due to spurious data entries
Solution Approach 1:
The memory controller performs zero padding as a preliminary action during the data transfer process. Before the data is written to the MMA accelerator memory, the controller identifies and replaces spurious data entries with zeroes based on the filter kernel column index, preventing these erroneous values from entering the computational pipeline and causing unwanted operations.
4Reliability
If feature map dimensions are changed to accommodate padding, then data accuracy is maintained, but device area increases
Solution Approach 1:
The patent applies zero padding only to the extent necessary to accommodate the filter kernel dimensions during convolution operations. By segmenting the padding application to match the specific requirements of each filter kernel column index, the system maintains data accuracy for the active regions while minimizing the overall device area required for the padded feature maps.
Data Source
AI summary
In described examples, an integrated circuit (IC) includes a matrix multiplication accelerator including a first memory, a second memory, and a memory controller. The second memory is configured to store multiple rows of an input feature map on a single line of cells of the memory, and to store a filter kernel. The memory controller reads multiple contiguous memory vectors of the second memory, different ones of the contiguous memory vectors corresponding to different portions of the input feature map. The memory controller also replaces (with padding zeroes) values of respective ones of the contiguous memory vectors. The number and location of replaced values are selected in response to a column index of an element of the filter kernel in response to which the respective contiguous memory vector is read. Zero padded contiguous memory vectors are written to the first memory.