Neural Network Convolution Buffering for Dilated Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolution operations in neural networks, particularly with dilated convolutions, face challenges in data reuse and high memory access efficiency due to increased receptive field sizes, leading to computational inefficiencies and potential overfitting.
Innovation Solution
Implement a neural network device with a controller that manages data fetching and writing operations based on dilation rates, using a buffer to enable data reuse and reduce redundancy in dilated convolution operations, thereby optimizing memory access and computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If dilated convolution operations are performed to increase receptive field size, then the ability to extract spatial features is improved, but memory access efficiency deteriorates due to high bandwidth requirements
Solution Approach 1:
The patent segments the large kernel operation into multiple smaller sub-kernels. Instead of performing one large dilated convolution operation that requires high memory bandwidth, the system divides the kernel into several smaller sub-kernels that can be processed sequentially with reduced memory access requirements at each step.
Solution Approach 2:
The patent performs preliminary actions by pre-fetching and storing intermediate results in buffer memory before they are needed for subsequent convolution operations. This allows the computation to proceed without waiting for memory access, improving overall memory access efficiency while maintaining the ability to perform large receptive field operations.
2Adaptability or versatility
If larger filters are used to increase receptive field size, then feature extraction capability is improved, but computational overhead increases
Solution Approach 1:
The patent segments a large filter operation into multiple smaller sub-filter operations. By dividing the large kernel into sub-kernels, the system reduces the computational complexity of each individual operation while achieving the same overall receptive field through sequential application of sub-kernels.
Solution Approach 2:
The patent introduces dynamic buffer management that adapts to the specific dilation rate and kernel size being used. The buffer allocation and data flow are dynamically adjusted based on the operational parameters, allowing efficient computation across varying receptive field sizes without fixed overhead.
3Reliability
If data is frequently accessed from memory for convolution operations, then computation accuracy is maintained, but memory access efficiency deteriorates
Solution Approach 1:
The patent performs preliminary actions by pre-loading input data and intermediate results into buffer memory before they are needed for convolution operations. This ensures that accurate computation can proceed using cached data without requiring frequent memory accesses, maintaining both accuracy and efficiency.
Solution Approach 2:
The patent introduces buffer memory as an intermediary between the main memory and computation units. This intermediary layer allows data to be transferred and cached in advance, reducing the frequency of main memory accesses while ensuring accurate computation by maintaining proper data flow and synchronization.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A neural network device includes: a memory configured to store a first feature map and a second feature map; and a neural network processor configured to operate a neural network, and comprising: a fetcher configured to fetch input data from the first feature map of the memory; a buffer configured to store the input data; an operator configured to generate output data by performing a convolution operation between the input data and a kernel; a writer configured to write the output data in the second feature map of the memory; and a controller configured to control the fetcher to fetch the input data and control the writer to write the output data, according to one or more intervals and one or more offsets determined based on a dilation rate of the kernel in multiple steps.