Batched Convolution Calculation Reducing On-Chip SRAM Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing implementation of MobileNet convolutional neural networks requires large on-chip SRAM for buffering intermediate results of depthwise convolution operations, leading to increased storage needs and power consumption due to limited data transmission bandwidth when the SRAM is insufficient, necessitating the splitting of operations and data transfer to off-chip memory.
Innovation Solution
A method that performs depthwise and pointwise convolution calculations in a neural network by reading input feature maps and kernels from DRAM, processing them in batches to store output feature values in on-chip memory, reducing the storage space required for intermediate results and minimizing data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the size of on-chip SRAM is increased to buffer intermediate results of depthwise convolution operations, then the calculation accuracy and completeness are improved, but the device complexity and power consumption increase due to larger storage requirements
Solution Approach 1:
The patent segments the depthwise convolution calculation process into multiple batches, where each batch processes a subset of input channels. This allows intermediate results to be processed in smaller portions that fit within the limited on-chip SRAM capacity, avoiding the need for large buffer memory while maintaining complete calculation accuracy.
Solution Approach 2:
The patent performs preliminary organization of input feature map data into batches before processing. By pre-grouping input channels and preparing corresponding depthwise convolution kernels in advance, the system ensures that each batch can be completely processed within the available SRAM capacity, preventing the need for frequent off-chip memory accesses.
2Device complexity
If the depthwise convolution operation is split into multiple calculations and results are written to off-chip memory, then the on-chip SRAM requirement is reduced, but the data transmission bandwidth burden increases and power consumption rises
Solution Approach 1:
The patent dynamically adjusts the batch size and processing strategy based on the available on-chip SRAM capacity. By making the batch processing configuration adaptive rather than fixed, the system optimizes the balance between on-chip buffer usage and off-chip memory access frequency, reducing unnecessary data transfers and associated power consumption.
Solution Approach 2:
The patent changes the processing parameter of batch size to match the on-chip SRAM capacity. By adjusting this parameter, the system ensures that each batch of intermediate results fits within the available buffer space, minimizing the need to spill results to off-chip memory and thereby reducing the bandwidth burden and power consumption.
3Device complexity
If the depthwise convolution operation is split into multiple calculations with off-chip memory access, then the on-chip SRAM requirement is reduced, but the calculation time increases due to repeated data transfer
Solution Approach 1:
The patent performs preliminary organization of input data into batches that are optimized for the available on-chip SRAM capacity. This pre-processing step ensures that subsequent depthwise convolution operations can proceed with minimal off-chip memory access, reducing the time loss associated with repeated data transfers while still using a smaller SRAM.
Data Source
AI summary
Disclosed is a method for convolution calculation in a neural network, comprising: reading an input feature map, depthwise convolution kernels and pointwise convolution kernels from a dynamic random access memory (DRAM); performing depthwise convolution calculations and pointwise convolution calculations by depthwise convolution calculation units and pointwise convolution calculation units, according to the input feature map, the depthwise convolution kernels and the pointwise convolution kernels to obtain output feature values of a first predetermined number p of points on all pointwise convolution output channels; storing the output feature values of a first predetermined number p of points on all pointwise convolution output channels into an on-chip memory; and repeating above operation to obtain output feature values of all points on all point wise convolution output channels. Therefore, the storage space for storing intermediate results may be reduced.